From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F3602D29B7; Thu, 24 Jul 2025 14:10:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366233; cv=none; b=FXtmYKWMXVjsMOnoae34r/1yFPo2BVdw30eT/0Al0boyrZdAZywk1N0XUkIqqHG6AiDqCxL30kZHX17o4/CBUOd2U3RmgqeWlyTFXu+rbP2U2a3lM5/PykdHPslst2bCRKgo4qSyqbu3FwmYsasH5EhpPqoN/6Yyqz8s0mUZsm4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366233; c=relaxed/simple; bh=kncEHGUHBsOPEbj18mUjhjLaFdDTloHadGxnvcBRhXM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jjFrpU+buws4QMlO+YFs+7uDx1mTNSwMAZOnFTFlRzdIJ777censlNyScOA8uv1KZ2bSxpkzooMo1RAp27ZSvNdbHCG6SLkB5FZpROUp0MCb0VhccDLmNlzEbUmyqOXn2dX/egmO04kIxfuiPEq0UNh6h0YJyp7CLucxUwVQj5k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=gAaY+Dbr; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gAaY+Dbr" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4822511E6; Thu, 24 Jul 2025 16:09:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366189; bh=kncEHGUHBsOPEbj18mUjhjLaFdDTloHadGxnvcBRhXM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=gAaY+DbrTVEhRUog6Y1tHYj8lkp5ax+jymeQIs6+3uVtcn7z+YxFdusv8C4Qu/p64 gKFSZwqZFlEPAssB+7plXSZ8LZxHsnTuZ+tcQjNZ6hmfsZQOHe2cAJWD/uxI3SSIA6 8KcE9stVQFHHDqFTYV0pgQqAGPVTS+EfBy9Yw050= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:08 +0200 Subject: [PATCH v2 01/27] media: mc: Add per-file-handle data support Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-1-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi , Laurent Pinchart , Hans Verkuil X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=7105; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=iZLPf77dc0oKef/FoIrNYxuY8wrefkX+9mbiaN7w1+U=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7OvS5BlVCgiF1gQqp9wy2UdrB4acbKilKqM fhKRME9Q1WJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zgAKCRByNAaPFqFW PFwnD/9tMm5RSk04hMUf0xgme2U3HQKA5jq+FX5aJPpIj4pNU9WjhDYPOKRXvTVrPqa2a87lE0v +2KyqJj0AYVdRG8PAqLIvfLgon0ttIlMHJRpsSY2XhX/KVhNtU78imxQsj+ZmR57o7IlKEfdaBE 0NFZBgL4VmRkB0WSaM+sbMDdtsMBS+GVITGxu0FEH7bJTIGvqGADAoFfIZT3osz6ZL/ij4OS3lB Y48H7lRZXIP0m97d26Qiph5Fa6ff7BnPXdulyG0ImDbE0cD9s5ndZPq0MNqjUr8Kq/OlzMKPJhC u57FCHwiwiA8qdSK04kWtDA1cE99rYGYSYMHXJDkHjrvzHi4LEc74iCATILYLlGMT0LCh/7/Qht Oz81NzymhC8coQxkylalaxhNzT+rcLO+ylaeB/Kj3wALxleEQAXsKzhUrcKfyDq9t7h9ECOZUsa VGP9P5PX+OwHYRqZofSIToNxjcYSUpUmYWbjBd0YPdm6BfXs+fgT0bdDiPoanvhP23ifhPqWyml /JZoke8BMxkZq4gtEtCmq75MT4XgZ+tBnQF6KzRpHCxJv6Mi+U+YbdPsV1E0HxWl6/BlsN7H81u WvDeHEKvFwre4T5n12vQr6lfLAiy2wMd+8PobpzSyQNeCTeXuqK3fhXfJpGD1z23NS2r14CZLtz 7/1Ne2kdcP5LNBQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B From: Laurent Pinchart The media devnode core associates devnodes with files by storing the devnode pointer in the file structure private_data field. In order to allow tracking of per-file-handle data introduce a new media devnode file handle structure that stores the devnode pointer, and store a pointer to that structure in the file private_data field. Users of the media devnode code (the only existing user being media_device) are responsible for managing their own subclass of the media_devnode_fh structure. Signed-off-by: Laurent Pinchart Prepare struct media_device_fh to be used for maintaining file handle list to avoid shuffling things here and there right after. Signed-off-by: Sakari Ailus Acked-by: Hans Verkuil Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 14 +++++++++++++- drivers/media/mc/mc-devnode.c | 20 +++++++++----------- include/media/media-device.h | 7 +++++++ include/media/media-devnode.h | 18 +++++++++++++++++- include/media/media-fh.h | 32 ++++++++++++++++++++++++++++++++ 5 files changed, 78 insertions(+), 13 deletions(-) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index c0dd4ae5722725f1744bc6fd6282d5c765438059..f298f8f67b8a84194d66126461e= 84228c0b20394 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -22,6 +22,7 @@ #include #include #include +#include #include =20 #ifdef CONFIG_MEDIA_CONTROLLER @@ -35,7 +36,6 @@ #define MEDIA_ENT_SUBTYPE_MASK 0x0000ffff #define MEDIA_ENT_T_DEVNODE_UNKNOWN (MEDIA_ENT_F_OLD_BASE | \ MEDIA_ENT_SUBTYPE_MASK) - /* -----------------------------------------------------------------------= ------ * Userspace API */ @@ -47,11 +47,23 @@ static inline void __user *media_get_uptr(__u64 arg) =20 static int media_device_open(struct file *filp) { + struct media_device_fh *fh; + + fh =3D kzalloc(sizeof(*fh), GFP_KERNEL); + if (!fh) + return -ENOMEM; + + filp->private_data =3D &fh->fh; + return 0; } =20 static int media_device_close(struct file *filp) { + struct media_device_fh *fh =3D media_device_fh(filp); + + kfree(fh); + return 0; } =20 diff --git a/drivers/media/mc/mc-devnode.c b/drivers/media/mc/mc-devnode.c index 56444edaf13651874331e7c04e86b0a585067d38..312eb48ffc2f2a0c013e4744204= 995df0ff5b12c 100644 --- a/drivers/media/mc/mc-devnode.c +++ b/drivers/media/mc/mc-devnode.c @@ -142,6 +142,7 @@ static long media_compat_ioctl(struct file *filp, unsig= ned int cmd, static int media_open(struct inode *inode, struct file *filp) { struct media_devnode *devnode; + struct media_devnode_fh *fh; int ret; =20 /* Check if the media device is available. This needs to be done with @@ -162,17 +163,15 @@ static int media_open(struct inode *inode, struct fil= e *filp) get_device(&devnode->dev); mutex_unlock(&media_devnode_lock); =20 - filp->private_data =3D devnode; - - if (devnode->fops->open) { - ret =3D devnode->fops->open(filp); - if (ret) { - put_device(&devnode->dev); - filp->private_data =3D NULL; - return ret; - } + ret =3D devnode->fops->open(filp); + if (ret) { + put_device(&devnode->dev); + return ret; } =20 + fh =3D filp->private_data; + fh->devnode =3D devnode; + return 0; } =20 @@ -181,8 +180,7 @@ static int media_release(struct inode *inode, struct fi= le *filp) { struct media_devnode *devnode =3D media_devnode_data(filp); =20 - if (devnode->fops->release) - devnode->fops->release(filp); + devnode->fops->release(filp); =20 filp->private_data =3D NULL; =20 diff --git a/include/media/media-device.h b/include/media/media-device.h index 53d2a16a70b0d9d6e5cc28fe1fc5d5ef384410d5..2fc750efef7c43814f019f12078= e9c96c1bd6bf9 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -108,6 +108,10 @@ struct media_device_ops { * other operations that stop or start streaming. * @request_id: Used to generate unique request IDs * + * @fh_list: List of file handles in the media device + * (struct media_device_fh.mdev_list). + * @fh_list_lock: Serialise access to fh_list list. + * * This structure represents an abstract high-level media device. It allow= s easy * access to entities and provides basic media device-level support. The * structure can be allocated directly or embedded in a larger structure. @@ -180,6 +184,9 @@ struct media_device { =20 struct mutex req_queue_mutex; atomic_t request_id; + + struct list_head fh_list; + spinlock_t fh_list_lock; }; =20 /* We don't need to include usb.h here */ diff --git a/include/media/media-devnode.h b/include/media/media-devnode.h index d27c1c646c2805171be3997d72210dd4d1a38e32..6c2e253dde498779dffd103dc5d= 00e50e14a0249 100644 --- a/include/media/media-devnode.h +++ b/include/media/media-devnode.h @@ -55,6 +55,20 @@ struct media_file_operations { int (*release) (struct file *); }; =20 +/** + * struct media_devnode_fh - Media device node file handle + * @devnode: pointer to the media device node + * + * This structure serves as a base for per-file-handle data storage. Media + * device node users embed media_devnode_fh in their custom file handle da= ta + * structures and store the media_devnode_fh in the file private_data in o= rder + * to let the media device node core locate the media_devnode correspondin= g to a + * file handle. + */ +struct media_devnode_fh { + struct media_devnode *devnode; +}; + /** * struct media_devnode - Media device node * @media_dev: pointer to struct &media_device @@ -146,7 +160,9 @@ void media_devnode_unregister(struct media_devnode *dev= node); */ static inline struct media_devnode *media_devnode_data(struct file *filp) { - return filp->private_data; + struct media_devnode_fh *fh =3D filp->private_data; + + return fh->devnode; } =20 /** diff --git a/include/media/media-fh.h b/include/media/media-fh.h new file mode 100644 index 0000000000000000000000000000000000000000..6f00744b81d6000a4b0c503fe69= 68dd7adcbb1c3 --- /dev/null +++ b/include/media/media-fh.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Media device file handle + * + * Copyright (C) 2019--2023 Intel Corporation + */ + +#ifndef MEDIA_FH_H +#define MEDIA_FH_H + +#include +#include + +#include + +/** + * struct media_device_fh - File handle specific information on MC + * + * @fh: The media device file handle + * @mdev_list: This file handle in media device's list of file handles + */ +struct media_device_fh { + struct media_devnode_fh fh; + struct list_head mdev_list; +}; + +static inline struct media_device_fh *media_device_fh(struct file *filp) +{ + return container_of(filp->private_data, struct media_device_fh, fh); +} + +#endif /* MEDIA_FH_H */ --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DABDA2D5418; Thu, 24 Jul 2025 14:10:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366235; cv=none; b=U1lgUdsLMCzTO54jWbWsO1HczK1YRrCtmcA4Wl7BRYPoAYEItbR0367bl9VGNz+EnJtkVyUS7T9VJmrvvXTGCHH+Gsef0/ThMRrpQceoyo1vRd3zZ87bXafVdkSc7tbmj3TYvmyvdo9wk2fIjWoy9sSBkOrEn4ILCfQB5BSrmRE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366235; c=relaxed/simple; bh=zDLiKpcarRebLqHSrXV+LHH8PWDYb0+SfoPVzO8Rjbg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SONVz94XLcKscXZZ9mGBhO0Q5OYlnzEcN2H/o6/QwLsfEoNLybrSjxyNZhtLxGDUb/JLHOK2k1Q+YrQCNWNmkX7gyAIlrkgk/2eJudN0s9uJ2Ky+mV/zLoLVCSN1E5p/QCjKe2jmXTCho4nKegRCOjw+IvqT+Agtz//lLECgPgc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=CPY1KMtG; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CPY1KMtG" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7DF17190D; Thu, 24 Jul 2025 16:09:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366190; bh=zDLiKpcarRebLqHSrXV+LHH8PWDYb0+SfoPVzO8Rjbg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=CPY1KMtGLCOdP9TXQ0RqC350lFyac3obxfU6ySAP1MSXGMVHFV1KXmD1nWBn7Sj86 GcrrlBUno3vW4ws4Ul6n9aOXK82mh3h47P9cOwkQLnvIdema0S34Jv+eqFnckyXeW2 dUuKfNiE2KqEhwqZU5MRyCFNsRbH+cswobBEYBGw= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:09 +0200 Subject: [PATCH v2 02/27] media: mc: Maintain a list of open file handles in a media device Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-2-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3849; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=/rqN2PeYLVGrC1gEE4q9OmKFqZ01RUXwuvcLLsZvWlg=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7Ox96cpEe5xXWsnzWGaSEqnjMmgCxy9aH+k AExdRPrDUOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zgAKCRByNAaPFqFW PGARD/9IgoHSHjubJ8wckLbGKEcTrRCALPQOrJ7z58zOdGADWX8VyXQzCbLyueW5viFbsSkw2r+ xy6DYTl45p7NrA2zXYKWzNAmVZ1aGWwOd48e1UNJfB2hJHA+xLhq6ilAOhKIrYBwOQdXdeOlfG/ rk/TEo9azmYM70sVDjfIj8g+yjwywjeWx/WknjrfgfyrGfMeb3LZbXQ9mg3ujW88vYX/uAZyRMT HO0xsK0oT4rlI9FkBQMinnwqP8ERr+ux/GPvDz4fZnpQ2oCId6yD3RUGPPW+l7jNx5nA4d7ch6y qnsTrr4+KT7VamhKd7TzLVNYSt02Pu0AZFBLI6ZNe378He7HnyiqQ7wqW0y38XtTViJEcN/cwX3 7V0lBIN37VwfLQL2fC5Ll735DgmCjmhlg72deXC2x11UDN69ArR0ma67y6hNb3h5DaslQ+Y3uxQ 1rLsJjlxmKRtkmvKW488i2fwSsc7ok2tYmOSN5IYKIZW9CMnGI+0i8Ra11f5KDq802rrXl//wtA NZ9Wfu5PfIl6n+9C8f2ifEBdGGMLS9Epz8wNO/Zi1D2jaMdPlDpilAg1zqI9ZjHLblGoB5Iz1f0 JJmFiUopCdq5RFXV4Rx7J3e+Th8LPXAlkyEOvylQdISsGQbRrDTCeTPkNioYRavPBefYX30PE1Q zwnMcz8n5rcp2/g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B From: Sakari Ailus The list of file handles is needed to deliver media events as well as for other purposes in the future. Signed-off-by: Sakari Ailus [Access media_device with devnode->media_dev] Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 19 ++++++++++++++++++- drivers/media/mc/mc-devnode.c | 2 +- include/media/media-devnode.h | 3 ++- 3 files changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index f298f8f67b8a84194d66126461e84228c0b20394..e0cad87087d3863bf14207049a5= 4e5e4dea1cdd4 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -45,8 +45,9 @@ static inline void __user *media_get_uptr(__u64 arg) return (void __user *)(uintptr_t)arg; } =20 -static int media_device_open(struct file *filp) +static int media_device_open(struct media_devnode *devnode, struct file *f= ilp) { + struct media_device *mdev =3D devnode->media_dev; struct media_device_fh *fh; =20 fh =3D kzalloc(sizeof(*fh), GFP_KERNEL); @@ -55,13 +56,23 @@ static int media_device_open(struct file *filp) =20 filp->private_data =3D &fh->fh; =20 + spin_lock_irq(&mdev->fh_list_lock); + list_add(&fh->mdev_list, &mdev->fh_list); + spin_unlock_irq(&mdev->fh_list_lock); + return 0; } =20 static int media_device_close(struct file *filp) { + struct media_devnode *devnode =3D media_devnode_data(filp); + struct media_device *mdev =3D devnode->media_dev; struct media_device_fh *fh =3D media_device_fh(filp); =20 + spin_lock_irq(&mdev->fh_list_lock); + list_del(&fh->mdev_list); + spin_unlock_irq(&mdev->fh_list_lock); + kfree(fh); =20 return 0; @@ -698,10 +709,12 @@ void media_device_init(struct media_device *mdev) INIT_LIST_HEAD(&mdev->pads); INIT_LIST_HEAD(&mdev->links); INIT_LIST_HEAD(&mdev->entity_notify); + INIT_LIST_HEAD(&mdev->fh_list); =20 mutex_init(&mdev->req_queue_mutex); mutex_init(&mdev->graph_mutex); ida_init(&mdev->entity_internal_idx); + spin_lock_init(&mdev->fh_list_lock); =20 atomic_set(&mdev->request_id, 0); =20 @@ -809,6 +822,10 @@ void media_device_unregister(struct media_device *mdev) return; } =20 + spin_lock_irq(&mdev->fh_list_lock); + list_del_init(&mdev->fh_list); + spin_unlock_irq(&mdev->fh_list_lock); + /* Clear the devnode register bit to avoid races with media dev open */ media_devnode_unregister_prepare(mdev->devnode); =20 diff --git a/drivers/media/mc/mc-devnode.c b/drivers/media/mc/mc-devnode.c index 312eb48ffc2f2a0c013e4744204995df0ff5b12c..50435f102aa7cd6a7cde759eeef= 73a99b9b80239 100644 --- a/drivers/media/mc/mc-devnode.c +++ b/drivers/media/mc/mc-devnode.c @@ -163,7 +163,7 @@ static int media_open(struct inode *inode, struct file = *filp) get_device(&devnode->dev); mutex_unlock(&media_devnode_lock); =20 - ret =3D devnode->fops->open(filp); + ret =3D devnode->fops->open(devnode, filp); if (ret) { put_device(&devnode->dev); return ret; diff --git a/include/media/media-devnode.h b/include/media/media-devnode.h index 6c2e253dde498779dffd103dc5d00e50e14a0249..26b19373c6646bfd11cfded220c= 9e61c81130580 100644 --- a/include/media/media-devnode.h +++ b/include/media/media-devnode.h @@ -22,6 +22,7 @@ #include =20 struct media_device; +struct media_devnode; =20 /* * Flag to mark the media_devnode struct as registered. Drivers must not t= ouch @@ -51,7 +52,7 @@ struct media_file_operations { __poll_t (*poll) (struct file *, struct poll_table_struct *); long (*ioctl) (struct file *, unsigned int, unsigned long); long (*compat_ioctl) (struct file *, unsigned int, unsigned long); - int (*open) (struct file *); + int (*open) (struct media_devnode *, struct file *); int (*release) (struct file *); }; =20 --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06FA62DEA80; Thu, 24 Jul 2025 14:10:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366237; cv=none; b=Ipj47fX2DSHn49uoOxOvM5MVGyOdEuROkcFjgOo68hXlvXmWZaLdDDmAWa+ynTL4EP/gdTWx891iyvPYdoXOGHGq65XBF0jA4C4Rsutxid1vFwBNx3QmpL+cKhqm53nE32bGPRByQZzSXcYH9CDgeukV9I7gS3EsS7/2wMi6Q3U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366237; c=relaxed/simple; bh=MQRwA7fqy++hrn9wDtM90iaXfefqzRiYRYRIsmH0eBo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=r8mopeOtrf7+ltUzMrZ3fTs/bfwMafoMb5F5cZENzCj5IFfReUW4EqpTUSE3657RrYagpBpyjCgAjTUPgeBtKOtnn7ynrYRQ5bk1OvdFrQXsjHdr242KqpyVF5CXtJxLas6E74j3FlQbmXy7WKASZL6nUG7Yy4ZB0TPK7N/F/No= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=tS4ldUt2; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tS4ldUt2" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A6D0619E7; Thu, 24 Jul 2025 16:09:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366191; bh=MQRwA7fqy++hrn9wDtM90iaXfefqzRiYRYRIsmH0eBo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=tS4ldUt26FugC4awS6WUaNpJW+zPQ3v7VVZYkN1Dp9g3s1ijuJrQdZPRah7y4BHul KxeumX7UTLx1vXstmFoMAga+FbhZH94WqKs/kCd1Vrr49FRw9f6msA1Z8MwdLYbO1V 2cq2m+PsAk2z0KkQ2EelXuknxDZVvzSfpkPCcj8o= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:10 +0200 Subject: [PATCH v2 03/27] media: media-entity: Introduce media_entity_context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-3-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=7515; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=MQRwA7fqy++hrn9wDtM90iaXfefqzRiYRYRIsmH0eBo=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7On0HmBG/yLewcQmpRACQsBQzuVlsMjyZo/ CXVZ6zICjmJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zgAKCRByNAaPFqFW PEuKD/4nrOfR5EUSGoAbVCcqjsRgd8stMiAo2Gw9Terar4dLAlL8CKGQemfPiePHDk8GZyKvxit WzrKSz6vljkJ57GsWQZ43pNedu45i+jrnB+ZBWkU82Nw0NmQ7o40LwqfnyT9FcLV/ewhgaG3YT5 QG3mwzAtGX623qzCqfit4fwTatvOqIAs6gwOHmAdJYTUTcOgtTwuK8ElV7GAvDmRr5QoBWid9Le Qs3Yq1q3hVD0bt1VMUFTtXrUuYeDXDfiVGVWJiAyR+4lZhalK/2sUyzxQRRW7aMJOqkf4sbA33t +GNjSujZg9TpJKY4Tl8c2tGv5Rk497B58ehR5Y3/94Cu1YXA8lIWG4isFgO87EbyXhUHMygpOPR I1n9Vf02ETbIYLH7dScvsK3+fmLYDEszfpPERZ3pDbTa4V3El1ZSx01bmyhVuLAJXtOXf+YH8ow Vjg/f8mBXoMrD2F8memdd08yQ+xdRbz3JRMmDSZRM+2wIEjJUkDtMQ2z7r4D/BYNzgjPYjokRxd mJZRYgn9tQmI+9oXuZDNYOiYI0kZ7buBCONJz2kofTWZjXbzD+oMqI/wF8qmY/9dY9kRye8FOqC ulcscKizbMciqoLWBSLAbDDQMAe0Uxzn27e1CnP9eOOkYaSNelgwkWKzllbr89qMYP0BY3VbdxE MBED2pdSb089hlw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce the 'struct media_entity_context' type, which serves for reference counting and introduce two new media entity operations to allow drivers to allocate and free a media entity context. The newly introduced type will be used as a base type for the device context types (video_device_context and v4l2_subdevice_context) that will be introduced in the next patches. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 46 ++++++++++++++++++++ include/media/media-entity.h | 101 +++++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 147 insertions(+) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 045590905582054c46656e20463271b1f93fa6b4..b4a9f0a0aa7353d7a3333f20903= 980956b3df4a7 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1673,3 +1673,49 @@ struct media_link *__media_entity_next_link(struct m= edia_entity *entity, return NULL; } EXPORT_SYMBOL_GPL(__media_entity_next_link); + +static void media_entity_release_context(struct kref *refcount) +{ + struct media_entity_context *ctx =3D + container_of(refcount, struct media_entity_context, refcount); + + ctx->entity->ops->destroy_context(ctx); +} + +struct media_entity_context * +media_entity_context_get(struct media_entity_context *ctx) +{ + if (!ctx) + return ERR_PTR(-EINVAL); + + kref_get(&ctx->refcount); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_entity_context_get); + +void media_entity_context_put(struct media_entity_context *ctx) +{ + if (!ctx) + return; + + kref_put(&ctx->refcount, media_entity_release_context); +} +EXPORT_SYMBOL_GPL(media_entity_context_put); + +void media_entity_init_context(struct media_entity *entity, + struct media_entity_context *ctx) +{ + if (!ctx) + return; + + ctx->entity =3D entity; + kref_init(&ctx->refcount); + INIT_LIST_HEAD(&ctx->list); +} +EXPORT_SYMBOL_GPL(media_entity_init_context); + +void media_entity_cleanup_context(struct media_entity_context *ctx) +{ +} +EXPORT_SYMBOL_GPL(media_entity_cleanup_context); diff --git a/include/media/media-entity.h b/include/media/media-entity.h index 64cf590b11343f68a456c5870ca2f32917c122f9..32298fe8a18c6ee3c1dbcff9ef8= 69548904417a7 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -248,6 +249,37 @@ struct media_pad { struct media_pipeline *pipe; }; =20 +/** + * struct media_entity_context - A media entity execution context + * @mdev_context: The media device context this media entity is bound to. + * The field is initialized when the entity is bound to a media + * device context. + * @entity: The media entity this context belongs to + * @refcount: The kref reference counter + * list: The list entry to link the entity context in the media device con= text + * + * This type represent the 'base class' used to implement execution contex= t for + * video device contexts and subdevice contexts. Those types embedds an in= stance + * of 'struct media_entity_context' as their first member, allowing the MC= core + * to implement type polymorphism and handle video device and subdevice co= ntexts + * transparently. + * + * The main function of this type is to provide reference counting for the + * 'dervived' device context types. The video device and subdevice core + * populates the 'context_release' function pointer that implement specific + * clean-up operations, similar to what a 'virtual destructor' would do in= C++. + * + * Drivers are not expected to use this type directly, but only the MC core + * will. + */ +struct media_device_context; +struct media_entity_context { + struct media_device_context *mdev_context; + struct media_entity *entity; + struct kref refcount; + struct list_head list; +}; + /** * struct media_entity_operations - Media entity operations * @get_fwnode_pad: Return the pad number based on a fwnode endpoint or @@ -269,6 +301,15 @@ struct media_pad { * media_entity_has_pad_interdep(). * Optional: If the operation isn't implemented all pads * will be considered as interdependent. + * @alloc_context: Allocate a media entity context. Drivers are allowed to + * sub-class the entity context type by defining a driver + * specific type that embeds an instance of either a + * video_device_context or subdevice_context as first + * member, and allocate the size of a driver-specific type + * in the implementation of this operation. Returns 0 for + * success, or an error code < 0 otherwise. + * @destroy_context: Release a media entity context previously allocated by + * the driver. * * .. note:: * @@ -284,6 +325,9 @@ struct media_entity_operations { int (*link_validate)(struct media_link *link); bool (*has_pad_interdep)(struct media_entity *entity, unsigned int pad0, unsigned int pad1); + int (*alloc_context)(struct media_entity *entity, + struct media_entity_context **context); + void (*destroy_context)(struct media_entity_context *context); }; =20 /** @@ -1448,3 +1492,60 @@ struct media_link *__media_entity_next_link(struct m= edia_entity *entity, MEDIA_LNK_FL_DATA_LINK)) =20 #endif + +/** + * media_entity_context_get - Increase the media entity context reference = count + * and return a reference to it + * + * @ctx: the media entity context + * + * Increase the media entity context reference count. The reference count + * is increased by the V4L2 core when: + * + * * a new context is allocated when bounding a media entity to a media de= vice + * context (by kref_init()) + * * the media pipeline the context is part of starts streaming + * + * The entity context gets automatically decreased by the V4L2 core when: + * + * * a context is unbound + * * the pipeline stops streaming + */ +struct media_entity_context * +media_entity_context_get(struct media_entity_context *ctx); + +/** + * media_entity_context_put - Decrease the media entity context reference = count + * + * @ctx: the media entity context + * + * Decrease the media entity context reference count. The reference count + * is decreased by the V4L2 core when: + * + * * the file handle the context is associated with is closed + * * the media pipeline the context is part of is stopped + */ +void media_entity_context_put(struct media_entity_context *ctx); + +/** + * media_entity_init_context - Initialize the media entity context + * + * @entity: the media entity this context belongs to + * @ctx: the media entity context + * + * Initialize the media entity context by initializing the kref reference + * counter. The intended caller of this function are the video device cont= ext + * and subdevic context initialize functions. + */ +void media_entity_init_context(struct media_entity *entity, + struct media_entity_context *ctx); + +/** + * media_entity_cleanup_context - Cleanup the media entity context + * + * @ctx: the media entity context + * + * Cleanup the media entity context. The intended caller of this function = are + * the video device and subdevice context cleanup functions. + */ +void media_entity_cleanup_context(struct media_entity_context *ctx); --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3915F2E2660; Thu, 24 Jul 2025 14:10:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366241; cv=none; b=tXo2jueWYic9zaIPNxlfZIqdaYi4kLE0bfgWRtNifF09+zoixxH6hLcQQURJd7ISdq9FucYhz4k7hG48160eeyWWLbf+4wS52ZIkJUZZgLZg8rlwgbfGdzLLmSJvwMUx71IkYw8l4xVn5h2eT0MINPtODCjSea7C4Fx/lHkylBc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366241; c=relaxed/simple; bh=Qd19apeMZFSiMBZkqVWpWcHh8AbZaZrN82RvB+0JkQY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZvMghb3Fb9AhJbIA6aW3CcXq7Md9951/xNVhDJYaxO+glZ9whc2sLGy17Jv7N6VQZ2/jnJtf4Rny0htnXi6tJ5NcXfJUuRQiSl5zW78nrmpNJUpEzWnq9V3aTugRatnrZxQF4CB4HkdA2563OA4mE5AMxL4xG1yHOkuoDuJZn9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=uPq0K00G; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uPq0K00G" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CEC95C64; Thu, 24 Jul 2025 16:09:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366192; bh=Qd19apeMZFSiMBZkqVWpWcHh8AbZaZrN82RvB+0JkQY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=uPq0K00GYoCid+3GFeHyIglCljgmKUiETJmOzxmYqhRyVy8JZVfDBcxhaikVtN5jT x/wMe4nzr8G8FghKujYA7xOAU+zt+hf+IPtkI+uv7u8R1+gvaTtEvDZ4ydgP+fPsQq g5EnIUUC/OjOg5x8XLf2GN9wugxgafmJlCcyRrrA= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:11 +0200 Subject: [PATCH v2 04/27] media: media-device: Introduce media device context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-4-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=18790; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Qd19apeMZFSiMBZkqVWpWcHh8AbZaZrN82RvB+0JkQY=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7O7REzI7+GYnqeAUxYhCmMOPfdqAzKHVik1 LnkyuWUSFaJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zgAKCRByNAaPFqFW PCTrD/0fiyOFhnYABbwTW4zzwr3uJNFAXK877jmO2zOiv9RpYlfsK1FiuIs9BopFcl+TN3WzwTB E0x8BkWpIaoL3BKomzZi5wpkql1+c8EQHHIpAWeiCnb5cmVHMJMwye8kFRI7uyYkOUbimYPY/y7 i67YR4AHfldhw8nuCY8kFGdc2oqnO/pf9SZLxfpvu3nWV2OEjFHoK/mgpgt1kx5n5lfyLLXHbcI KJnv4VipV1knu4Sr1261wwqhyb6jO5r/3Nj2LHvOYFg2mjkEsgcALduCDWJ+0E/riQJ3h1K55q5 Rh3pwhAxRT3JsMPEO98I31td9LPmv5WkgCQB9Tyv8HeG1pRHim1NfvLOWgwMrq3K5haXDX4IyCY uhRj1mFr9j1CYjf6EUB01sMyxr7BLCwZPrB6j5FkKSMeMrQuj/bMyqoASkgB+rTk2UlLFBYxcEi NCZbi5wV5B9qZmCGJQg/c2ejNTjbF1Eci8Tu2s8lr+fE9zCBuFzyrpOlbuTC0I3MuWSUnb7GSd5 cYeD3jTX3L36IfIGbn9L2gqRco/jma5pddrMCzi0DlD470z1oVuV4qTvhfwA0A8EI3lTutNvBG2 XzFeUj+k2hg66k15g2Ka/IQtJ17bAF+LXzLhT+kRcZ5gKmyLjbHiQjNSvrMk8glxGXz2Wm+n71g 9O/+blcEZNvpCvQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce a new type in the media-fh.h header that represent a media device context. A media device context is allocated when the media device is open and released when the last reference to it is put. A new pair of media_device_ops is added to allow device drivers to allocate and release a media context. The media context groups together the media entity contexts that are associated with it to form an isolated execution context. Provide helpers in mc-device.c for drivers and for the v4l2-core to handle media device contexts and to bind/unbind entity contexts to it. Once an entity context has been bound to a media device context it is possible to retrieve it by using a pointer to the entity the device is represented by. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 168 ++++++++++++++++++++++++++++++++++ drivers/media/mc/mc-entity.c | 1 + include/media/media-device.h | 211 +++++++++++++++++++++++++++++++++++++++= ++++ include/media/media-fh.h | 5 + 4 files changed, 385 insertions(+) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index e0cad87087d3863bf14207049a54e5e4dea1cdd4..d8f12db933d22ae7466051698d8= 53f4bdc599400 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -12,7 +12,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -49,11 +51,31 @@ static int media_device_open(struct media_devnode *devn= ode, struct file *filp) { struct media_device *mdev =3D devnode->media_dev; struct media_device_fh *fh; + int ret; =20 fh =3D kzalloc(sizeof(*fh), GFP_KERNEL); if (!fh) return -ENOMEM; =20 + if (mdev->ops && mdev->ops->alloc_context) { + if (WARN_ON(!mdev->ops->destroy_context)) { + kfree(fh); + return -EINVAL; + } + + ret =3D mdev->ops->alloc_context(mdev, &fh->context); + if (ret) { + kfree(fh); + return ret; + } + + /* + * Make sure the driver implementing alloc_context has + * called media_device_init_context() + */ + WARN_ON(!fh->context->initialized); + } + filp->private_data =3D &fh->fh; =20 spin_lock_irq(&mdev->fh_list_lock); @@ -73,6 +95,8 @@ static int media_device_close(struct file *filp) list_del(&fh->mdev_list); spin_unlock_irq(&mdev->fh_list_lock); =20 + media_device_context_put(fh->context); + kfree(fh); =20 return 0; @@ -860,6 +884,150 @@ void media_device_unregister(struct media_device *mde= v) } EXPORT_SYMBOL_GPL(media_device_unregister); =20 +/* -----------------------------------------------------------------------= ------ + * Context handling + */ + +static void media_device_release_context(struct kref *refcount) +{ + struct media_device_context *context =3D + container_of(refcount, struct media_device_context, refcount); + + /* + * All the associated entity contexts should have been released if we + * get here. + */ + WARN_ON(!list_empty(&context->contexts)); + + context->mdev->ops->destroy_context(context); +} + +struct media_device_context * +media_device_context_get(struct media_device_context *ctx) +{ + if (!ctx) + return ERR_PTR(-EINVAL); + + kref_get(&ctx->refcount); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get); + +void media_device_context_put(struct media_device_context *ctx) +{ + if (!ctx) + return; + + kref_put(&ctx->refcount, media_device_release_context); +} +EXPORT_SYMBOL_GPL(media_device_context_put); + +struct media_device_context *media_device_context_get_from_fd(unsigned int= fd) +{ + struct media_device_context *ctx; + struct file *filp =3D fget(fd); + struct media_device_fh *fh; + + if (!filp) + return NULL; + + fh =3D media_device_fh(filp); + ctx =3D media_device_context_get(fh->context); + fput(filp); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get_from_fd); + +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx) +{ + ctx->mdev =3D mdev; + INIT_LIST_HEAD(&ctx->contexts); + mutex_init(&ctx->lock); + kref_init(&ctx->refcount); + + ctx->initialized =3D true; + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_init_context); + +void media_device_cleanup_context(struct media_device_context *ctx) +{ + mutex_destroy(&ctx->lock); + list_del_init(&ctx->contexts); +} +EXPORT_SYMBOL_GPL(media_device_cleanup_context); + +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context) +{ + struct media_entity_context *entry; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + + /* Make sure the entity has not been bound already. */ + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry =3D=3D context) + return -EINVAL; + } + + list_add_tail(&context->list, &mdev_context->contexts); + context->mdev_context =3D media_device_context_get(mdev_context); + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_bind_context); + +int media_device_unbind_context(struct media_entity_context *context) +{ + struct media_device_context *mdev_context =3D context->mdev_context; + struct media_entity_context *entry; + struct media_entity_context *tmp; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + list_for_each_entry_safe(entry, tmp, &mdev_context->contexts, list) { + if (entry !=3D context) + continue; + + list_del(&entry->list); + media_device_context_put(mdev_context); + entry->mdev_context =3D NULL; + + return 0; + } + + WARN(true, "Media entity context is not bound to any media context\n"); + + return -EINVAL; +} +EXPORT_SYMBOL_GPL(media_device_unbind_context); + +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity) +{ + struct media_entity_context *entry; + + guard(mutex)(&mdev_context->lock); + + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry->entity =3D=3D entity) + return media_entity_context_get(entry); + } + + return ERR_PTR(-EINVAL); +} +EXPORT_SYMBOL(media_device_get_entity_context); + #if IS_ENABLED(CONFIG_PCI) void media_device_pci_init(struct media_device *mdev, struct pci_dev *pci_dev, diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index b4a9f0a0aa7353d7a3333f20903980956b3df4a7..7bc276c725f974539ea06e3882d= 004b81be1de68 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1717,5 +1717,6 @@ EXPORT_SYMBOL_GPL(media_entity_init_context); =20 void media_entity_cleanup_context(struct media_entity_context *ctx) { + media_device_unbind_context(ctx); } EXPORT_SYMBOL_GPL(media_entity_cleanup_context); diff --git a/include/media/media-device.h b/include/media/media-device.h index 2fc750efef7c43814f019f12078e9c96c1bd6bf9..2ea8fce9ba75700286961f16225= 84372a954cb8a 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -18,10 +18,73 @@ =20 #include #include +#include =20 struct ida; struct media_device; =20 +/** + * struct media_device_context - Media device context + * @mdev: The media device this context is associated with + * @refcount: The kref reference counter + * @lock: Protects the entities contexts list + * @contexts: List of entity contexts associated with this media device co= ntext + * @initialized: Flag set to true by media_device_init_context() + * + * A media device context is created every time the media device gets open= ed by + * userspace. It is then uniquely identified for applications by the numer= ical + * file descriptor returned by a successful call to open() and is associat= ed + * with an instance of :c:type:`media_device_fh`. + * + * Media device contexts are ref-counted and thus freed once the last refe= rence + * to them is released. + * + * A media device context groups together the media entity contexts regist= ered + * on a video device or v4l2 subdevice that has been associated with a med= ia + * device context. The association between a media entity context and media + * device context is called 'bounding', and the result of bounding is to c= reate + * an 'execution context' independent from other execution contexts. + * + * An entity context is bound to a media device context by a call to the + * VIDIOC_BIND_CONTEXT ioctl on video devices and by a call to + * VIDIOC_SUBDEV_BIND_CONTEXT on subdevices by userspace. The bounding ope= ration + * groups together entity contexts to the same media device context. As vi= deo + * devices and v4l2 subdevices devnodes can be opened multiple times, each= file + * descriptor resulting from a successful open() call can be bound to a + * different media device context. + * + * Creating execution contexts by bounding video entity contexts to a media + * device context allows userspace to effectively multiplex the usage of a + * media graph and of the device nodes that are part of it. + * + * In order to create an execution context userspace should: + * + * 1) Open the media device to create a media device context identified by= the + * file descriptor returned by a successful 'open()' call + * 2) Open the video device or v4l2 subdevice and bind the file descriptor= s to + * the media device context by calling the VIDIOC_BIND_CONTEXT and + * VIDIOC_SUBDEV_BIND_CONTEXT ioctls + * + * All devices bound to the same media device context are now part of the = same + * execution context. From this point on all the operations performed on a= file + * descriptor bound to a media device context are independent from operati= ons + * performed on a file descriptor bound to a different execution context. + * + * Binding an entity context to a media device context increases the media + * device context reference count. This guarantees that references to media + * device context are valid as long as there are valid entity contexts that + * refers to it. Symmetrically, unbinding an entity context from a media + * device context decreases the media device context reference count. + */ +struct media_device_context { + struct media_device *mdev; + struct kref refcount; + /* Protects the 'contexts' list */ + struct mutex lock; + struct list_head contexts; + bool initialized; +}; + /** * struct media_entity_notify - Media Entity Notify * @@ -62,6 +125,13 @@ struct media_entity_notify { * request (and thus the buffer) must be available to the driver. * And once a buffer is queued, then the driver can complete * or delete objects from the request before req_queue exits. + * @alloc_context: Allocate a media device context. The operation allows d= rivers to + * allocate a driver-specific structure that embeds a + * media_device_context instance as first member where to store + * driver-specific information that are global to all device + * contexts part of media device context. Returns 0 on success a + * negative error code otherwise. + * @destroy_context: Release a media device context. */ struct media_device_ops { int (*link_notify)(struct media_link *link, u32 flags, @@ -70,6 +140,9 @@ struct media_device_ops { void (*req_free)(struct media_request *req); int (*req_validate)(struct media_request *req); void (*req_queue)(struct media_request *req); + int (*alloc_context)(struct media_device *mdev, + struct media_device_context **ctx); + void (*destroy_context)(struct media_device_context *ctx); }; =20 /** @@ -298,6 +371,144 @@ int __must_check __media_device_register(struct media= _device *mdev, */ void media_device_unregister(struct media_device *mdev); =20 +/* -----------------------------------------------------------------------= ------ + * media device context handling + */ + +/** + * media_device_context_get - Increase the media device context reference = count + * and return a reference to it + * @ctx: The media device context + */ +struct media_device_context * +media_device_context_get(struct media_device_context *ctx); + +/** + * media_device_context_put - Decrease the media device context reference = count + * @ctx: The media device context + */ +void media_device_context_put(struct media_device_context *ctx); + +/** + * media_device_context_get_from_fd - Get the media device context associa= ted with a + * numerical file descriptor + * + * @fd: the numerical file descriptor + * + * A media device context is created whenever the media device devnode is = opened + * by userspace. It is then associated uniquely with a numerical file desc= riptor + * which is unique in the userspace process context. + * + * This function allows to retrieve the media device associated with such + * numerical file descriptor and increases the media device context refere= nce + * count to guarantee the returned reference stays valid at least until the + * caller does not call media_device_context_put(). + * + * Caller of this function are required to put the returned media device c= ontext + * once they are done with it. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl h= andler + * which need to get the media device contexts associated to a numerical f= ile + * descriptor. + */ +struct media_device_context *media_device_context_get_from_fd(unsigned int= fd); + +/** + * media_device_init_context - Initialize the media device context + * + * @mdev: The media device this context belongs to + * @ctx: The media device context to initialize + * + * Initialize the fields of a media device context. Device drivers that su= pport + * multi context operations shall call this function in their implementati= on of + * media_device_operations.alloc_context() + */ +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx); + +/** + * media_device_cleanup_context - Cleanup the media device context + * + * @ctx: The media device context to clean up + * + * Cleanup a media device context. Device drivers that support multi conte= xt + * operations shall call this function in their implementation of + * media_device_operations.destroy_context() before releasing the memory a= llocated + * by media_device_operations.alloc_context(). + */ +void media_device_cleanup_context(struct media_device_context *ctx); + +/** + * media_device_bind_context - Bind an entity context to a media device co= ntext + * + * @mdev_context: pointer to struct &media_device_context + * @context: the entity context to bind + * + * This function creates a mapping entry in the media device context that + * associates an entity context to the media entity it belongs to and stor= es it + * in a linked list so that they can be retrieved later. + * + * Binding an entity context to a media device context increases the media + * device context refcount. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl h= andler + * that binds a newly created context to a media device context. + */ +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context); + +/** + * media_device_unbind_context - Unbind an entity context from a media dev= ice + * context + * + * @context: the entity context to unbind + * + * An entity context is unbound from a media device context when the file = handle + * it is associated with gets closed. + * + * Unbinding an entity context from a media device context decreases the m= edia + * device context refcount. + * + * Returns 0 if the context was bound to a media device context, -EINVAL + * otherwise. + */ +int media_device_unbind_context(struct media_entity_context *context); + +/** + * media_device_get_entity_context - Get the entity context associated with + * a media entity in a media device context + * + * @mdev_context: pointer to struct &media_device_context + * @entity: pointer to struct &media_entity that the entity context is + * associated with + * + * An entity context is uniquely associated with a media device context af= ter it + * has been bound to it by a call to the VIDIOC_BIND_CONTEXT ioctl. This h= elper + * function retrieves the entity context associated with a media device co= ntext + * for a specific entity that represents a video device or a v4l2 subdevic= e. + * + * The reference count of the returned entity context is increased to guar= antee + * the returned reference stays valid until the caller does not call + * media_entity_context_put(). + * + * Drivers are not expected to call this function directly but should inst= ead + * use the helpers provided by the video_device and v4l2_subdevice layers, + * video_device_context_get() and v4l2_subdev_get_context() respectively. + * Drivers are always required to decrease the returned context reference = count + * by calling video_device_context_put() and v4l2_subdev_put_context(). + * + * If no entity context has been associated with the media device context + * provided as first argument an error pointer is returned. Drivers are + * required to always check the value returned by this function. + */ +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity); + +/*------------------------------------------------------------------------= ------ + * Media entity handling + */ + /** * media_device_register_entity() - registers a media entity inside a * previously registered media device. diff --git a/include/media/media-fh.h b/include/media/media-fh.h index 6f00744b81d6000a4b0c503fe6968dd7adcbb1c3..48ec266416dd288a008bc5f93db= 5eb7ec6b8859c 100644 --- a/include/media/media-fh.h +++ b/include/media/media-fh.h @@ -13,15 +13,20 @@ =20 #include =20 +struct media_device_context; + /** * struct media_device_fh - File handle specific information on MC * * @fh: The media device file handle * @mdev_list: This file handle in media device's list of file handles + * @context: The media device context associated with the file handle */ struct media_device_fh { struct media_devnode_fh fh; struct list_head mdev_list; + + struct media_device_context *context; }; =20 static inline struct media_device_fh *media_device_fh(struct file *filp) --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C31BB2D8DB7; Thu, 24 Jul 2025 14:10:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366241; cv=none; b=espvQ4vQaLqkHcjuW39RhXyFp6g4G72E9bh/7Y5Q1tgcEf8Z5UApcKUV5HTAze4IyGhXYo+5QbYciLIDhCGopnRU0HGar9TiIKmkuSS7rn13OxvYhFyxamIXNrQAzb3ovHfOcqH0lLINnWURV7spdIKC6yUQWvsm9gkC1+wedZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366241; c=relaxed/simple; bh=DnyByKVCK9e8LIByMHqQ/KRM1C+sAIkgyijuQbTd5Yw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZLsJc1qTZpqnS0pL2X3jWw0Ebgvb438Wsl7ggRNartIXHhSVIGV/zxFyVwxitMsKP5EGdOfIlkO4ZuJT/2gYghg5HVSt221CJnfqbq/mwwqQl7E1OKxGVBrBVjmjRQUtxckeuKsrShIU0JPEfLptGwfFJBkd6PcQpnMI5OsNL6M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=b3tuXXYX; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b3tuXXYX" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 050DE1A04; Thu, 24 Jul 2025 16:09:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366194; bh=DnyByKVCK9e8LIByMHqQ/KRM1C+sAIkgyijuQbTd5Yw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=b3tuXXYXuouWggunl8jcymqaOR1B8NNCf+TDzx7vIbdx8usDXYLag6kpoGR1DI2/O MKw0rlv0Aqn4FWSol7jr6Kp9JvhbD32uSU3SSZs8QY45KiLbn5yCM1vX1g2aEA1OSP llox/f/LkxVOG/Wkp/OzaIP04E0AU1X1bKWvJS9Y= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:12 +0200 Subject: [PATCH v2 05/27] media: v4l2-dev: Introduce video device context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-5-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=9358; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=DnyByKVCK9e8LIByMHqQ/KRM1C+sAIkgyijuQbTd5Yw=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7Oi7DOv+2oMd6oHfFjIY+DwIpJibKZ0VUDA n5elJfwmGGJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zgAKCRByNAaPFqFW PIKeEACaBYId/cKMoKVepnHcF6swvItlIcef/GgpDlqeARlJvt/FrCHAFlWhJoLeuykGKDzokfh Ne15kWRuKWrXwlIp0jq49AUGspJzxuhzIzMyWeSmVB+uyLrq1C9h/9k6SCoVnQssfM+0IF1f6rQ lanV6nm66JE3FH+5ZRhyDs/EFKJv4yPs/cp9BP2GnGdK//TUc3ANqFPBlSR+si7MpuGN1J9szBL /RVhahVfEyxuspnZO/SVFjMn41RkzwCKnVhFvAM20Raceb4a6Km2PAPUX/w7b9K8+5uBGcI5DbA maWK5VoWo3NfQoS6tsXrsuBJROQCHVU9l1ohocJWnCJEETHHg+lUbP7SwEqr2W/q+znrEexUNDS 4YUsG2c9nck3HkswcWhiQl/KLvOoja3QhOVkjGMQDyY8T+puT7kEpmqmmY9oFj5Cb+RKXGqrqrf ci4aXN/LsW47SATVSGGUgwAAO4ppJ6ikvfizP0638jO97ztK9pdzUqcfCu4OZbn+KItQTNo61eE NwusXrIxK5V+Rv+NAwC/DZwr73L65NOsZC86vFOHYCmFZIICS41lIrr8bLTCVK1CYbQAVSaXgFl MuQhVqcs3MgxYMroNortb9ibtqtCYASuNHmO3Sz0ZNO9MteqiX90OHNPMyedNE42bzxnExMG+Ub roQ2H3QBLjEI+1A== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce a new type in video-device that represents a video device context. It extends 'struct media_entity_context' to include video-device specific data and configuration. The new type is intended to be extended by drivers that can store driver-specific information in their video device derived types. The next patch will introduce the VIDIOC_BIND_CONTEXT ioctl that allows to create a video device context and uniquely associate it with a media context. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 43 +++++++++++++ include/media/v4l2-dev.h | 126 +++++++++++++++++++++++++++++++++= ++++ include/media/v4l2-fh.h | 3 + 3 files changed, 172 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v= 4l2-dev.c index c369235113d98ae26c30a1aa386e7d60d541a66e..c83c37843c9e7beb899a4b2bd17= 6273c3dec381b 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1200,6 +1200,49 @@ struct media_pipeline *video_device_pipeline(struct = video_device *vdev) } EXPORT_SYMBOL_GPL(video_device_pipeline); =20 +struct video_device_context * +video_device_context_get(struct media_device_context *mdev_context, + struct video_device *vdev) +{ + struct media_entity *entity =3D &vdev->entity; + struct media_entity_context *ctx =3D + media_device_get_entity_context(mdev_context, entity); + + if (!ctx) + return NULL; + + return container_of(ctx, struct video_device_context, base); +} +EXPORT_SYMBOL_GPL(video_device_context_get); + +void video_device_context_put(struct video_device_context *ctx) +{ + if (!ctx) + return; + + media_entity_context_put(&ctx->base); +} +EXPORT_SYMBOL_GPL(video_device_context_put); + +int video_device_init_context(struct video_device *vdev, + struct video_device_context *ctx) +{ + media_entity_init_context(&vdev->entity, &ctx->base); + + ctx->vdev =3D vdev; + mutex_init(&ctx->queue_lock); + + return 0; +} +EXPORT_SYMBOL_GPL(video_device_init_context); + +void video_device_cleanup_context(struct video_device_context *ctx) +{ + mutex_destroy(&ctx->queue_lock); + media_entity_cleanup_context(&ctx->base); +} +EXPORT_SYMBOL_GPL(video_device_cleanup_context); + #endif /* CONFIG_MEDIA_CONTROLLER */ =20 /* diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index a69801274800f4f9add723b51fe0a31331e88f97..9276e095fb17414a9eb7845db0a= a81572c42ca45 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -18,6 +18,7 @@ #include =20 #include +#include =20 #define VIDEO_MAJOR 81 =20 @@ -660,6 +661,131 @@ __must_check int video_device_pipeline_alloc_start(st= ruct video_device *vdev); */ struct media_pipeline *video_device_pipeline(struct video_device *vdev); =20 +/** + * struct video_device_context - The video device context + * @base: The media entity context base class member + * @vdev: The video device this context belongs to + * @queue_lock: Protects the vb2 queue + * @queue: The vb2 queue + * + * This structure represents an isolated execution context of a video devi= ce. + * This type 'derives' the base 'struct media_entity_context' type which + * implements refcounting on our behalf and allows instances of this type = to be + * linked in the media_device_context contexts list. + * + * By storing the data and the configuration of a video device in a per-fi= le + * handle context, userspace is allowed to multiplex the usage of a single= video + * device devnode by opening it multiple times and by associating it with a + * media device context. This operation is called 'bounding' and is perfor= med + * using the VIDIOC_BIND_CONTEXT ioctl. + * + * A video device context is created and stored in the v4l2-fh file handle + * associated with an open file descriptor when a video device is 'bound' = to a + * media device context. The 'bounding' operation realizes a permanent + * association valid until the video device context is released. + * + * A video device can be bound to the same media device context once only. + * Trying to bind the same video device to the same media device context a + * second time, without releasing the already established context by closi= ng the + * bound file descriptor first, will result in an error. + * + * To create a video device context userspace shall use the VIDIOC_BIND_CO= NTEXT + * ioctl that creates the video device context and uniquely associates it = with a + * media device file descriptor. + * + * Once a video device file descriptor has been bound to a media device co= ntext, + * all the operations performed on the video device file descriptor will be + * directed on the just created video device context. This means, in examp= le, + * that the video device format and the buffer queue are isolated from the= ones + * associated with a different file descriptor obtained by opening again t= he + * same video device devnode but bound to a different media device context. + * + * Drivers that implement multiplexing support have to provide a valid + * implementation of the context-related operations in the + * media entity operations. + * + * Drivers are allowed to sub-class the video_device_context structure by + * defining a driver-specific type which embeds a struct video_device_cont= ext + * instance as first member, and allocate the driver-specific structure si= ze in + * their implementation of the `alloc_context` operation. + * + * Video device contexts are ref-counted by embedding an instance of 'stru= ct + * media_entity_context' and are freed once all the references to it are + * released. + * + * A video device context ref-count is increased when: + * - The context is created by bounding a video device to a media device c= ontext + * - The media pipeline starts streaming + * A video device context ref-count is decreased when: + * - The associated file handle is closed + * - The media pipeline stops streaming + * + * The ref-count is increased by a call to video_device_context_get() and = is + * reponsibility of the caller to decrease the reference count with a call= to + * video_device_context_put(). + */ +struct video_device_context { + struct media_entity_context base; + + struct video_device *vdev; + /* Protects the vb2 queue. */ + struct mutex queue_lock; + struct vb2_queue queue; +}; + +/** + * video_device_context_get - Helper to get a video device context from a + * media device context + * + * @mdev_context: The media device context + * @vdev: The video device the context refers to + * + * Helper function that wraps media_device_get_entity_context() and returns + * the video device context associated with a video device in a media devi= ce + * context. + * + * The reference count of the returned video device context is increased. + * Callers of this function are required to decrease the reference count of + * the context reference with a call to video_device_context_put(). + */ +struct video_device_context * +video_device_context_get(struct media_device_context *mdev_context, + struct video_device *vdev); + +/** + * video_device_context_put - Helper to decrease a video context reference + * count + * + * @ctx: The video context to release + */ +void video_device_context_put(struct video_device_context *ctx); + +/** + * video_device_init_context - Initialize the video device context + * + * @vdev: The video device this context belongs to + * @ctx: The context to initialize + * + * Initialize the video device context. The intended callers of this funct= ion + * are driver-specific implementations of the media_entity_ops.alloc_conte= xt() + * function that allocates their driver specific types that derive from + * struct video_device_context. + */ +int video_device_init_context(struct video_device *vdev, + struct video_device_context *ctx); + +/** + * video_device_cleanup_context - Cleanup the video device context + * + * @ctx: The context to cleanup. + * + * Cleanup the video device context. The intended callers of this function= are + * driver specific implementation of the media_entity_ops.destroy_context() + * function before releasing the memory previously allocated by + * media_entity_ops.alloc_context(). + */ +void video_device_cleanup_context(struct video_device_context *ctx); + #endif /* CONFIG_MEDIA_CONTROLLER */ =20 #endif /* _V4L2_DEV_H */ diff --git a/include/media/v4l2-fh.h b/include/media/v4l2-fh.h index b5b3e00c8e6a0b082d9cd8a0c972a5094adcb6f2..a8de8613a026589ede94dc3e70c= 8a49ae08f4582 100644 --- a/include/media/v4l2-fh.h +++ b/include/media/v4l2-fh.h @@ -20,6 +20,7 @@ =20 struct video_device; struct v4l2_ctrl_handler; +struct video_device_context; =20 /** * struct v4l2_fh - Describes a V4L2 file handler @@ -38,6 +39,7 @@ struct v4l2_ctrl_handler; * @sequence: event sequence number * * @m2m_ctx: pointer to &struct v4l2_m2m_ctx + * @context: The video device context */ struct v4l2_fh { struct list_head list; @@ -54,6 +56,7 @@ struct v4l2_fh { u32 sequence; =20 struct v4l2_m2m_ctx *m2m_ctx; + struct video_device_context *context; }; =20 /** --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 843022E427B; Thu, 24 Jul 2025 14:10:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366244; cv=none; b=N40FF0MNCUVCicY74d7gs+UplHgbN6bzlsMq2htVbDiMN90HwbxtRifYXZ4wTg9q60u2IoYzgt78P6KBa7pDoVSsBb+YWjiM/w3NNPnOBXyDAYBWJwmPAIzbEVsfjr7xB01+m+RhCo3tSM5D+aDeoMVjhHob6Frnv6s0MgsqYzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366244; c=relaxed/simple; bh=y6CtFSygCTQezYgM+DYFcUUx7oApPhfpK3+h4WXc9Ak=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MFzqLMz/4heK4HtAOMjSgbdSPc2tWZZfZcwSeDPLhTHokJtPgJNN+Asfe85hk84VtNPAj7JsouXgUGa/eDspC/lHazGJT3DTwVUZZjghdJkR12S8igXn7qxdDTD0t4o1Krb8+CyigLi81as6IgoXwd5GF4Ms7ObIudaTN+Haum8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=kbLncd37; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kbLncd37" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 448871ACF; Thu, 24 Jul 2025 16:09:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366195; bh=y6CtFSygCTQezYgM+DYFcUUx7oApPhfpK3+h4WXc9Ak=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kbLncd37uM6slkchMwKMFoa5Z+rme7cALBL/gmeJPvzlOC9u6V32nK4u3NdvORqC+ QCCSBLEOKBiMlHeB6tb+aEe91YybJMTCrRRgMjxG1qWBBGfR76tZLOtkdBykjJ9cyd TV5aIuX4T5Hjyjiu3L47i/pS7YwNI89GBuSOhgeo= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:13 +0200 Subject: [PATCH v2 06/27] media: v4l2-ioctl: Introduce VIDIOC_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-6-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8696; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=y6CtFSygCTQezYgM+DYFcUUx7oApPhfpK3+h4WXc9Ak=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7OCMocGJg4McaY1eSfSvbZPdFRvvqXYBeuy 1Sht/lP79eJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zgAKCRByNAaPFqFW PJVOD/0XAYjLtUFbG6lCv0mocUmp2jzA8aHY3WAyxy1LGEk0h23LFl7Ldw9t8Rtb1IWe1sR1Xey dJxntT81RpS+p+B06GzDswCgy0hyCKYYJBNyxwhCBPp/nBphdDgucoRfW8lLADNl13RfH8I/jL0 8Fq/mml98a+AsKmHxYw50yO6OhAIlhJtATpf3a8cwRSCOcoQEwHWV87nvd9MlllkoTm8lukh+Z9 owkWKDjDvfO0snm9gJsF/2P1aKgLjL+i2bCdCFpUpppbTrr727E6J6/YooWPi+0TZXUdEZueFtt w9fnf/SWhXEH+Ind9BUVG3RbmwPjDYfw8KEj+VrAKmJiHW01DrRqiWQwSSXW+l3E5fWk62TvunK Ifix0dCR+QBcvMxdkNmvZgRms+ZEmP5LVHCUF19Wcv057eUQFXR3kVh5h/IT2jSWVA1iJ9uP/ds JqbJmkA3SdDSU+sZjRnfjQIEhSeMbWD3wxBZiCOZmiRCcQo31sZQgh9xxt25Od3SnP9JsF26xDS B1S63mJl77am4UvDFs4Y6ggYAcMcEke9zPK3JytIAJo+hdy1wcKUEAkcl0vPj370wIYPq5S0sDo SQ2ei4DrQLOYiBSRyYQBJbnes93h24sC22BNbRyRCF9RqsFQTlgX9O+sylKf1GymZD5joDOW3N4 3aLqtYbXhM9+Eyg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce a new ioctl in V4L2 to allocate a video device context and associate it with a media device context. The ioctl is valid only if support for MEDIA_CONTROLLER is compiled in as it calls into the entity ops to let driver allocate a new context and binds the newly created context with the media context associated with the file descriptor provided by userspace as the new ioctl argument. The newly allocated video context is then stored in the v4l2-fh that represent the open file handle on which the ioctl has been called. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 10 +++++ drivers/media/v4l2-core/v4l2-fh.c | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 72 ++++++++++++++++++++++++++++++++= ++++ include/media/v4l2-ioctl.h | 7 ++++ include/uapi/linux/videodev2.h | 11 ++++++ 5 files changed, 101 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v= 4l2-dev.c index c83c37843c9e7beb899a4b2bd176273c3dec381b..bc6502b4ce21cc0ad53136e1637= d1c926e31dd89 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -606,6 +606,10 @@ static void determine_valid_ioctls(struct video_device= *vdev) __set_bit(_IOC_NR(VIDIOC_ENUM_FREQ_BANDS), valid_ioctls); =20 if (is_vid) { +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif + /* video specific ioctls */ if ((is_rx && (ops->vidioc_enum_fmt_vid_cap || ops->vidioc_enum_fmt_vid_overlay)) || @@ -661,12 +665,18 @@ static void determine_valid_ioctls(struct video_devic= e *vdev) SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_cap); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } else if (is_meta && is_tx) { /* metadata output specific ioctls */ SET_VALID_IOCTL(ops, VIDIOC_ENUM_FMT, vidioc_enum_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_out); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } if (is_vbi) { /* vbi specific ioctls */ diff --git a/drivers/media/v4l2-core/v4l2-fh.c b/drivers/media/v4l2-core/v4= l2-fh.c index 90eec79ee995a2d214590beeacc91b9f8f33236d..f7af444d2344541ccae1eae230b= 39d4cbc47f6bd 100644 --- a/drivers/media/v4l2-core/v4l2-fh.c +++ b/drivers/media/v4l2-core/v4l2-fh.c @@ -93,6 +93,7 @@ int v4l2_fh_release(struct file *filp) struct v4l2_fh *fh =3D filp->private_data; =20 if (fh) { + video_device_context_put(fh->context); v4l2_fh_del(fh); v4l2_fh_exit(fh); kfree(fh); diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core= /v4l2-ioctl.c index 46da373066f4ec786b87ef18b8372abee621332f..ff80a3d9a9fe66753d8c9bc8353= 5f6ad5288c3ec 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -9,6 +9,7 @@ */ =20 #include +#include #include #include #include @@ -350,6 +351,13 @@ static void v4l_print_format(const void *arg, bool wri= te_only) } } =20 +static void v4l_print_context(const void *arg, bool write_only) +{ + const struct v4l2_context *c =3D arg; + + pr_cont("context=3D%u\n", c->context_fd); +} + static void v4l_print_framebuffer(const void *arg, bool write_only) { const struct v4l2_framebuffer *p =3D arg; @@ -2151,6 +2159,69 @@ static int v4l_overlay(const struct v4l2_ioctl_ops *= ops, return ops->vidioc_overlay(file, fh, *(unsigned int *)arg); } =20 +#if defined(CONFIG_MEDIA_CONTROLLER) +static int v4l_bind_context(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + struct video_device *vdev =3D video_devdata(file); + struct media_device_context *mdev_context; + struct v4l2_fh *vfh =3D + test_bit(V4L2_FL_USES_V4L2_FH, &vdev->flags) ? fh : NULL; + struct v4l2_context *c =3D arg; + int ret; + + /* + * TODO: do not __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls) + * if V4L2_FL_USES_V4L2_FH isn't set or the driver does not implement + * alloc_context and destroy_context. + */ + if (!vfh) + return -ENOTTY; + + if (!vdev->entity.ops || !vdev->entity.ops->alloc_context || + !vdev->entity.ops->destroy_context) + return -ENOTTY; + + mdev_context =3D media_device_context_get_from_fd(c->context_fd); + if (!mdev_context) + return -EINVAL; + + /* Let the driver allocate the per-file handle context. */ + ret =3D vdev->entity.ops->alloc_context(&vdev->entity, + (struct media_entity_context **) + &vfh->context); + if (ret) + goto err_put_mdev_context; + + /* + * Bind the newly created video device context to the media device + * context identified by the file descriptor. + */ + ret =3D media_device_bind_context(mdev_context, + (struct media_entity_context *) + vfh->context); + if (ret) + goto err_put_context; + + media_device_context_put(mdev_context); + + return 0; + +err_put_context: + video_device_context_put(vfh->context); +err_put_mdev_context: + media_device_context_put(mdev_context); + + return ret; +} +#else +static int v4l_bind_context(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + return 0; +} +#endif /* CONFIG_MEDIA_CONTROLLER */ + static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -2998,6 +3069,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] =3D= { IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chi= p_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext= _ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), IOCTL_INFO(VIDIOC_REMOVE_BUFS, v4l_remove_bufs, v4l_print_remove_buffers,= INFO_FL_PRIO | INFO_FL_QUEUE | INFO_FL_CLEAR(v4l2_remove_buffers, type)), + IOCTL_INFO(VIDIOC_BIND_CONTEXT, v4l_bind_context, v4l_print_context, 0), }; #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) =20 diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index 82695c3a300a73219f262fb556ed61a8f09d273e..6d9edfd9ca912972ad15acdc070= 14dee1ed36ab6 100644 --- a/include/media/v4l2-ioctl.h +++ b/include/media/v4l2-ioctl.h @@ -18,6 +18,7 @@ #include =20 struct v4l2_fh; +struct video_device_context; =20 /** * struct v4l2_ioctl_ops - describe operations for each V4L2 ioctl @@ -149,6 +150,8 @@ struct v4l2_fh; * :ref:`VIDIOC_TRY_FMT ` ioctl logic for metadata capture * @vidioc_try_fmt_meta_out: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT ` ioctl logic for metadata output + * @vidioc_bind_context: pointer to the function that implements + * :ref:`VIDIOC_BIND_CONTEXT ` ioctl * @vidioc_reqbufs: pointer to the function that implements * :ref:`VIDIOC_REQBUFS ` ioctl * @vidioc_querybuf: pointer to the function that implements @@ -402,6 +405,10 @@ struct v4l2_ioctl_ops { int (*vidioc_try_fmt_meta_out)(struct file *file, void *fh, struct v4l2_format *f); =20 + /* Context handlers */ + int (*vidioc_bind_context)(struct file *file, void *fh, + struct video_device_context *c); + /* Buffer handlers */ int (*vidioc_reqbufs)(struct file *file, void *fh, struct v4l2_requestbuffers *b); diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 3dd9fa45dde1066d52a68581625a39e7ec92c9b7..0b9aa89e2479620dbbaa54f1aad= ff7aaa7a3d0f7 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1057,6 +1057,14 @@ struct v4l2_jpegcompression { * always use APP0 */ }; =20 +/* + * V I D E O D E V I C E C O N T E X T + */ + +struct v4l2_context { + __u32 context_fd; +}; + /* * M E M O R Y - M A P P I N G B U F F E R S */ @@ -2818,6 +2826,9 @@ struct v4l2_remove_buffers { #define VIDIOC_REMOVE_BUFS _IOWR('V', 104, struct v4l2_remove_buffers) =20 =20 +/* Context handling */ +#define VIDIOC_BIND_CONTEXT _IOW('V', 105, struct v4l2_context) + /* Reminder: when adding new ioctls please add support for them to drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ =20 --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8361A2E4275; Thu, 24 Jul 2025 14:10:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366244; cv=none; b=tpe/VVGKQwpTgyFXUljwTVCXYLCRLl/bhWuBNI3+OLppgWIsOfcN0xdmlfynekOA8pmGUecMr0zkp0kiJqtV1ebV0vu076nWkAeITfV52PXh6SwZFmjsDDfe6xN/w2M1TWdsxHfELryUYxhHvq7OGyVHtU0gukTPqZzqPKHmSzc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366244; c=relaxed/simple; bh=V29a6qX8KA0ELmVzRolMfVBPPzH85sU33AYkFGGSdRI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OPuqW/J647RVvYH7jB4uvXda9NJjUHZKyyrgZbUl8Vs0DPVZO2pjIZpgdf+3u3CTNjJRZ2p+Vr7s2urQ9FnMc0hiYvyK4gZR+0e4Vklk0t1Z+yANpMiicn2wWlIUbGstKGCxlxqU7yFXZ2yZ3z/rB79Fw3lN6TpsmG/gc1Xyvac= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=QwlwWrx9; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QwlwWrx9" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 71877C79; Thu, 24 Jul 2025 16:09:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366196; bh=V29a6qX8KA0ELmVzRolMfVBPPzH85sU33AYkFGGSdRI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QwlwWrx9/bXH6s6L686gbmUSCanbKpOJt0Rv2wgRpyNVXfVpIhnj0jsfAQhzbSmVQ qgszdOGLzxyg6BONkbJWnBR52Vby5KQwlNPaE+S3ZsrTuCk0RiKuWR6gpwKfLzy3dM RThkXLiKp7KEwHcVbreBzOaGvNQHbt/T69t4nRCg= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:14 +0200 Subject: [PATCH v2 07/27] media: Documentation: Add VIDIOC_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-7-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4469; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=V29a6qX8KA0ELmVzRolMfVBPPzH85sU33AYkFGGSdRI=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7OHWG882uXaFEbSbt0Hr5/yWbNirgfhlDO1 fH9BACuQ0GJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zgAKCRByNAaPFqFW PMiXD/9ptjYvK6OodJsJm6frucZmguwClByRCBQIzmoIHod2NI+aXnplxYiBs004EFI+isBYOf2 ofXGV85huvYTG8eBx/S7ocknJu/aU3fLMOsO+EDpLKJ3nzhz6eUp3yWo/pdv3MKQpNSu0vurBeq oVDbfgWZFi1K/HrVDfMVLptskMYg6FyAvrd9bNrAjaLO8Um8/GA1cv+xjhiq3ZY5XKgtE4/4MDU x11OIj9KnUfa0jHl9YwIThbaENit1LsIXtBoNUTUNZPBKKL9cqKLOfHfr3i5qr55rsH4opPaMsn gbvBwtxjggqTbqoZAi/nss8gvck/3VtS8FfB7LooadDdrWNA6yWhWcvEMdxEd4DWbiFpvxPyWiF TzBoePhTZjBd/FVrSDklaRJ36Gl6zq+5y7FLcvUyDfvCkMZZyYSrxOrcCdlN2sspyXtVnsWwGRv f1od1+T1q+UDQ/Xp1S6rfiHKbq9KURLqgSaarSkjjfRsUQ5xfbSBUjjcanix4U0Dilux9DOW8aF W3zHtE1JAUbNhv2auXcMpC+nOv7nizoQxUItCutHDFvjd4J2cPp74zgtRcMZ84FU9gA6XijuTd9 T4pdogp3UHtV39jF0c7B8PrhnCICx7FZBd9sf6++rODg56jYiUYG2b5QVCJHBtWLM7OSpPHBjRG dFjZ4mFhvDKCdtw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Document the newly introduced VIDIOC_BIND_CONTEXT ioctl. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/user-func.rst | 1 + .../media/v4l/vidioc-bind-context.rst | 80 ++++++++++++++++++= ++++ 2 files changed, 81 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Document= ation/userspace-api/media/v4l/user-func.rst index 6f661138801cde2810997740ee8305085fe73a43..0d9aff56ab653b2a4f6afe4828f= 88bc5637addf1 100644 --- a/Documentation/userspace-api/media/v4l/user-func.rst +++ b/Documentation/userspace-api/media/v4l/user-func.rst @@ -12,6 +12,7 @@ Function Reference =20 func-close func-ioctl + vidioc-bind-context vidioc-create-bufs vidioc-cropcap vidioc-dbg-g-chip-info diff --git a/Documentation/userspace-api/media/v4l/vidioc-bind-context.rst = b/Documentation/userspace-api/media/v4l/vidioc-bind-context.rst new file mode 100644 index 0000000000000000000000000000000000000000..74c885169e22c495056a9364e72= 2e2140d8f21c5 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/vidioc-bind-context.rst @@ -0,0 +1,80 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: V4L + +.. _vidioc_bind_context: + +************************* +ioctl VIDIOC_BIND_CONTEXT +************************* + +Name +=3D=3D=3D=3D + +VIDIOC_BIND_CONTEXT - Bind a video device file handle to a media device co= ntext + +Synopsis +=3D=3D=3D=3D=3D=3D=3D=3D + +.. c:macro:: VIDIOC_BIND_CONTEXT + +``int ioctl(int fd, VIDIOC_BIND_CONTEXT, struct v4l2_context *argp)`` + +Arguments +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`v4l2_context`. + +Description +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Applications call the ``VIDIOC_BIND_CONTEXT`` ioctl to bind a video device= file +handle to a media device context. Binding a video device file handle to a= media +device context creates an isolated execution context which allows to multi= plex +the usage of a video device. This means, in practice, that the video device +configuration (format, sizes etc) applied on a file handle bound to a media +device context won't be visible on file handles bound to a different media +device context (or not bound at all). + +By opening a media device applications create a media device context to wh= ich +video devices and subdevices file handles can be bound to. The file descri= ptor +returned by a call to :c:func:`open()` on the media device identifies uniq= uely +the media device context. Application populates the ``context_fd`` field of +:c:type:`v4l2_context` with the file descriptor of an open media device to +identify the media context to which they want to bind a video device to. + +Applications can open a video device node multiple times, and call +``VIDIOC_BIND_CONTEXT`` on each file handle returned by a successful call = to +:c:func:`open()` to isolate the operations performed on that file handle f= rom +any operation performed on other file handles bound to different contexts.= This +means, in example, that the video device format, sizes and the buffers are +isolated from the ones associated with a file descriptor, obtained by open= ing +the same video device but bound to a different media device context (or not +bound at all). + +The bounding operation realizes a permanent association valid until the vi= deo +device context is released by closing the file handle. + +A video device file handle can be bound to the same media device context o= nce +only. Trying to bind the same file handle to the same media device context= a +second time, without releasing the already established context by closing = the +bound file descriptor first, will result in an error. + +Bounding is an opt-in feature that applications are free to ignore. Any +operation directed to a non bound file handle will continue to work as it = used +to, and the video device configuration (formats, sizes etc) will be visible +across all the other non-bound file handles. + +Return Value +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes ` chapter. + +EINVAL + The media device context file handle ``context_fd`` is not valid or the + video device file handle is already bound to a context. --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE1552E54C5; Thu, 24 Jul 2025 14:10:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366247; cv=none; b=ucoW4k/MPmomFi+6mxiEYr2Sal2x6xO83UH+LXnMhtXuCxjvNbH+qFTs9dpRYXcob2q41gehf8Itg4i5XYevVGpQa/juqU5/53mfRSTdakgvwK73p/l6OsXYTWjI/NWgSacyJCemiRM6F1VHV4h7OOXoCk0OIte2Vjiji52HG4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366247; c=relaxed/simple; bh=EonmGcn1CUXqn+CIokljWT6wYvOQxvsBhcP+HctWnYE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=VVZIwatHeJYdiG6iS4KTGJCFrOWxrSSCA+sjDzWp05DGpA8rp2gbCe3Oba1LH1XuOeHKM7g7gnT0Sf8BIAp9cQSqfJDf+25eMaWJuGHgwMvr78U4susDYBv6oYfI72b1bYPrQi1dak5ETBB1auNsCxb1Sr7jbtuOVLeo07oAKiE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=AjF3sWSq; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AjF3sWSq" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9004511E6; Thu, 24 Jul 2025 16:09:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366197; bh=EonmGcn1CUXqn+CIokljWT6wYvOQxvsBhcP+HctWnYE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=AjF3sWSq68zt55kJCCUOopuuNLaJDase9nep7waIbMpdT9bKqbLp73uRP+EyHGiz/ XJ9TEzdr5eTl5Q0sTjNMB2AJPlBfWeHXonAkLD5atf2mdeHVA0HO3JWTYZ/oFLmif0 wSnGIPipdA31TvGohJgyx3kLEWwpETnc8loppMOc= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:15 +0200 Subject: [PATCH v2 08/27] media: v4l2-dev: Introduce default contexts Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-8-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6550; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=EonmGcn1CUXqn+CIokljWT6wYvOQxvsBhcP+HctWnYE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7PjcILge5KUnOifnWuZ7nZx7uQeQKECHpY2 G+SWA6TyKyJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PMX0EAC/huwTM3l8S09cYRp9M8if3pLVvCeVwHz/66atuq4vFNHvn9xZ+ko+6dIYvnruFRcScOn T9S8xd42vHSA3QJhfiMarHLXTSU5pETGAA0mwSarBKa3WfdjuoQ3h6XdC0n6IaOITsswQm9sjJU +XJvyn69hmZkfIBKKFvysvc8LdcJEsi7hJbUaCxIagliVLS17+LgAW+uClTt2DAS4/CftGDxRkF ePdUcDKqQsIwgimJ6dQnFkdj0zjQPPNlWpOHumZ3YQyMmL/8Kk7uI/1f/0hunPmbm/VN1pd4vSq 32SCmQ9pqKLwcZtAr8bWiDl7MTNxv34lEj6hJikTarDlOC8BSTs2Zhly6eBuM3N7s6om1jxc8QM PFBIQ2ZZHRG7oNqV7ruLHpdw09faHKrHELVEsGox7VLj4k6P3el3S77woBDjeWtDdKqCGLd90O0 mUQJvU+9ropfcDfwbM/s4voDWc5rmS0/UHwsVKo7KF2PhnN+hiX1L0FNVk5QSS6ziri36MA64lK QYIaUqqdXPdy6TE8bkDqvsJErM4AqQO0fZR0hDoKr5trTpSVrd2r+6cjEtCzUZFgUL8gAA5dOuL YOUZgYGdz5KLpryryVHAxvdsw8eUDHzMgjNPs69K0SjPjDkg31uymZPZ+qge3Zg7Qur4w8Iod5U krnA9TT+YTKLRKw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce the media device and video device default contexts. Drivers ported to use multi-context support that used to work with a non-context aware userspace (which doesn't call VIDIOC_BIND_CONTEXT) shall continue to work even if they are context aware. Provide a default context in the media device and in the video device structures and let drivers allocate and release them with the newly introduced operations. Bind the video device default context to the default context of the media device associated with the v4l2_dev. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 11 +++++++++ drivers/media/v4l2-core/v4l2-dev.c | 46 ++++++++++++++++++++++++++++++++++= +--- include/media/media-device.h | 5 +++++ include/media/v4l2-dev.h | 3 +++ 4 files changed, 62 insertions(+), 3 deletions(-) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index d8f12db933d22ae7466051698d853f4bdc599400..e1f34f884fee2e8c3750f9c1e85= 142ff2f6b7bf0 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -746,12 +746,23 @@ void media_device_init(struct media_device *mdev) media_set_bus_info(mdev->bus_info, sizeof(mdev->bus_info), mdev->dev); =20 + mdev->default_context =3D NULL; + if (mdev->ops && + mdev->ops->alloc_context && mdev->ops->destroy_context) { + if (mdev->ops->alloc_context(mdev, &mdev->default_context)) { + dev_err(mdev->dev, + "Failed to initialize media device default context\n"); + return; + } + } + dev_dbg(mdev->dev, "Media device initialized\n"); } EXPORT_SYMBOL_GPL(media_device_init); =20 void media_device_cleanup(struct media_device *mdev) { + media_device_context_put(mdev->default_context); ida_destroy(&mdev->entity_internal_idx); mdev->entity_internal_idx_max =3D 0; media_graph_walk_cleanup(&mdev->pm_count_walk); diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v= 4l2-dev.c index bc6502b4ce21cc0ad53136e1637d1c926e31dd89..2d78096931efd88215bc847b105= e198a54035546 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -31,6 +31,8 @@ #include #include =20 +#include + #define VIDEO_NUM_DEVICES 256 #define VIDEO_NAME "video4linux" =20 @@ -220,8 +222,13 @@ static void v4l2_device_release(struct device *cd) if (v4l2_dev->release =3D=3D NULL) v4l2_dev =3D NULL; =20 - /* Release video_device and perform other - cleanups as needed. */ + /* Release default context. */ +#ifdef CONFIG_MEDIA_CONTROLLER + video_device_context_put(vdev->default_context); +#endif + vdev->default_context =3D NULL; + + /* Release video_device and perform other cleanups as needed. */ vdev->release(vdev); =20 /* Decrease v4l2_device refcount */ @@ -1086,7 +1093,36 @@ int __video_register_device(struct video_device *vde= v, /* Part 5: Register the entity. */ ret =3D video_register_media_controller(vdev); =20 - /* Part 6: Activate this minor. The char device can now be used. */ + /* + * Part 6: Complete the video device registration by initializing the + * default context. The defaul context serves for context-aware driver + * to operate with a non-context-aware userspace that never creates + * new contexts. If the video device driver is not context aware, it + * will never implement 'context_ops' and will never use the default + * context. + */ + vdev->default_context =3D NULL; +#ifdef CONFIG_MEDIA_CONTROLLER + if (vdev->entity.ops && vdev->entity.ops->alloc_context && + vdev->entity.ops->destroy_context) { + ret =3D vdev->entity.ops->alloc_context(&vdev->entity, + (struct media_entity_context **) + &vdev->default_context); + if (ret) { + pr_err("%s: default context alloc failed\n", __func__); + goto cleanup; + } + + ret =3D media_device_bind_context(vdev->v4l2_dev->mdev->default_context, + &vdev->default_context->base); + if (ret) { + pr_err("%s: default context bind failed\n", __func__); + goto cleanup; + } + } +#endif + + /* Part 7: Activate this minor. The char device can now be used. */ set_bit(V4L2_FL_REGISTERED, &vdev->flags); mutex_unlock(&videodev_lock); =20 @@ -1094,6 +1130,10 @@ int __video_register_device(struct video_device *vde= v, =20 cleanup: mutex_lock(&videodev_lock); +#ifdef CONFIG_MEDIA_CONTROLLER + video_device_context_put(vdev->default_context); +#endif + vdev->default_context =3D NULL; if (vdev->cdev) cdev_del(vdev->cdev); video_devices[vdev->minor] =3D NULL; diff --git a/include/media/media-device.h b/include/media/media-device.h index 2ea8fce9ba75700286961f1622584372a954cb8a..b3cc6793a8b5eff4c26e57b01e1= a62ab71e8195b 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -184,6 +184,9 @@ struct media_device_ops { * @fh_list: List of file handles in the media device * (struct media_device_fh.mdev_list). * @fh_list_lock: Serialise access to fh_list list. + * @default_context: The default video device context. Used by drivers that + * support multi-context operation when operated by a + * non-context aware userspace. * * This structure represents an abstract high-level media device. It allow= s easy * access to entities and provides basic media device-level support. The @@ -260,6 +263,8 @@ struct media_device { =20 struct list_head fh_list; spinlock_t fh_list_lock; + + struct media_device_context *default_context; }; =20 /* We don't need to include usb.h here */ diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index 9276e095fb17414a9eb7845db0aa81572c42ca45..9e1cf58623acc4ab5f503a9663f= d999b38130226 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -237,6 +237,8 @@ struct v4l2_file_operations { * @ctrl_handler: Control handler associated with this device node. * May be NULL. * @queue: &struct vb2_queue associated with this device node. May be NULL. + * @default_context: &struct video_device_context associated with this dev= ice + * node * @prio: pointer to &struct v4l2_prio_state with device's Priority state. * If NULL, then v4l2_dev->prio will be used. * @name: video device name @@ -283,6 +285,7 @@ struct video_device { =20 struct vb2_queue *queue; =20 + struct video_device_context *default_context; struct v4l2_prio_state *prio; =20 /* device info */ --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE0812E54DB; Thu, 24 Jul 2025 14:10:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366247; cv=none; b=rCT6hppC9+63IvGgHMKQLhQr8A8bC+RFIhsetCfatw5AeX5AdHwomd8USFXqgMsMQJNw+l02R3pOF4rNwsS3S8LOa4Hq0BZY52ZbY2B2J1kEPG23GR5zHZL8RQCLAAmc+XSdmYEOMXjHWinfpk1rcpwgkJb0jPLAalgqyR6HR2c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366247; c=relaxed/simple; bh=7vOckOlOxZhw2fbL9SA65DEYxitygggpkHW/wn4/+ec=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tRckgvknDo8qA/t8Dt7buZFnYiwi50kOU50TjcJHN0/zH47nh4OyTAuu8DiaQYgkR/DL9ysorzk34ei9/uIAIGehDDdFC9cizztu6dM10/doErzB7brNMtgv2PUlDdoAsjO0Tm8yESlTfJUIaFulvMeQ+FgwlKE6IV2gTcUs2EE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=Nz6s0cpM; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Nz6s0cpM" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B42C11A37; Thu, 24 Jul 2025 16:09:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366198; bh=7vOckOlOxZhw2fbL9SA65DEYxitygggpkHW/wn4/+ec=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Nz6s0cpM9/owWJkU8pyOHVjwVQjPh0byz4D193b8Po+R3FJYUKasDdPI99zkVnWaB kmj28EHDAPTMV2DrzPC+kcF+abMhnpivuTycy0mIrpIjTGr3GmOZ7D3j40fql4c2VV sQLxaukkKtPE5pGKe3hGW6LJ4fNVJ/ZKnA+9kZ3U= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:16 +0200 Subject: [PATCH v2 09/27] media: v4l2-dev: Add video_device_context_from_file() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-9-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3662; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=7vOckOlOxZhw2fbL9SA65DEYxitygggpkHW/wn4/+ec=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7PkjaY/ZSj7vHkANoqvzVbGnB7c0FYtuyIL cMLSnSwBamJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PDfrD/9RfoA8wg5LkT2LdivtZ5m0NdRIiPj4RwTBL6/pslI7Ll7rDQjbzGLU7acWDOXTBkuNwfn 7Gqay0BSwp0TrsjdKjGsrtfN7TNEFyEIyYfrTui+QxxOV1HpWQBcqjKKNytoxQwNvy2wxL53vSB 7z/xn2xR22OGQZGCJXuzfljWG1S6d7aZqPnPMwIpnS8tXGHgNntryf+kWX/lQUvQCXRMl21qqL8 82EMJ2Pfi0kJl6PWpplbm6llPl6GRQbzoDKcHSPFCwgD+F6GrXm73sKQ6DBv+XcsD2AkljP0zXb 0512SNrU3YwOHDk/uWsCAggM8jR+3tQodn0KZuh2Gkw94NzMiQ2b0tUy6DWnGSVtKJxSaf8mUMj sb5wh8HNBDIVFDNtXQ5OEeDEIZf6WIWii/YwD78BKdLnfWGA9LhRJRFqYLSMwom0E+EvIP61A3j FGEuc2IyaVqn1KKPskVJ/pLtrXFvcsAau90vhWFy5csPf0cyz8ELPaARNXWzkP6udOx3qB0+7qp Lrp7WsdApMEJmRE1j4HzgAIHcnRm64rs2ggnlDmlVLkPpFGRJ2sRUpFPksY+AgI/aiAQSY0wwF5 M1rYaIvCCN0e893e99zX133miFLLHSydgckK3ASbP/mOHS2SuJjtN6XcdFuBcCMz1TlGBqqewdd tNonoFUvhDkDc1Q== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce an helper function to get a video_device_context from file. If no context has been bound to the file, return the video device default context. As a video device context lifetime is bound to the one of the file handle it is associated with, and the intended user of this helper are the v4l2_ioctl_ops handlers that operates on an open file handle, the context returned by this function is guaranteed to be valid for the whole function scope of the caller. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 15 +++++++++++++++ include/media/v4l2-dev.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v= 4l2-dev.c index 2d78096931efd88215bc847b105e198a54035546..fafd4209e2cab320a6e164d33e9= a0f73784d22db 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1182,6 +1182,21 @@ EXPORT_SYMBOL_GPL(v4l2_debugfs_root); =20 #if defined(CONFIG_MEDIA_CONTROLLER) =20 +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vfd) +{ + struct v4l2_fh *vfh =3D + test_bit(V4L2_FL_USES_V4L2_FH, &vfd->flags) ? filp->private_data + : NULL; + + /* If the file handle has been bound to a context return it. */ + if (vfh && vfh->context) + return vfh->context; + + return vfd->default_context; +} +EXPORT_SYMBOL_GPL(video_device_context_from_file); + __must_check int video_device_pipeline_start(struct video_device *vdev, struct media_pipeline *pipe) { diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index 9e1cf58623acc4ab5f503a9663fd999b38130226..a41afb81663bc9cb3bfc06dcf9b= 11ceeaf7c3415 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -789,6 +789,36 @@ int video_device_init_context(struct video_device *vde= v, */ void video_device_cleanup_context(struct video_device_context *ctx); =20 +/** + * video_device_context_from_file - Get the video device context associate= d with + * an open file handle + * @filp: The file handle + * @vdev: The video device + * + * If a video device context has been bound to an open file handle by a ca= ll + * to the VIDIOC_BIND_CONTEXT ioctl this function returns it, otherwise re= turns + * the default video device context. + * + * The intended callers of this helper are the driver-specific v4l2_ioctl_= ops + * handlers, which receive as first argument a file pointer. By using this + * helper drivers can get the context associated with such file, if any. + * Otherwise, if the video device has not been bound to any media device + * context, the default video device context is returned. + * + * As video device contexts are reference counted and their lifetime is + * guaranteed to be at least the one of the file handle they are associated + * with, callers of this function are guaranteed to always receive a valid + * context reference by this function. The reference will remain valid for= the + * whole function scope of the caller that has received the open file hand= le as + * argument. Likewise, accessing the media context from the video device c= ontext + * returned by this function is always safe within the same function scope. + * + * This function does not increase the reference count of the returned vid= eo + * device context. + */ +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vde= v); + #endif /* CONFIG_MEDIA_CONTROLLER */ =20 #endif /* _V4L2_DEV_H */ --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DF6D2E6D15; Thu, 24 Jul 2025 14:10:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366250; cv=none; b=nt5zMEj/AUcr79MuY39xyA97JGAhWzaK/Mf82EDnH66y9xZS+61mE62Jm47iT7EATwPq5xgGtrMXqWt0bKpWPCfs0n4681hmyE6OH5i4MZidqOh2dD/rIqRaD0CpgI+hESiXrc++I7VOCHJikqutd8xBmQhd2ToDmBbEa9M8JkQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366250; c=relaxed/simple; bh=mLUIKlRtGcg7V5Vy0wqmsYRGL64MqtvU55q1oBc3ZZs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pa0hxkNrHpi/bJeFQQm5bVDjGiXV8ku5hNQa41BMf4CIIgZtX6b/2yR/oGapsHdVOdXKU1H7HRbIfcM7IzXsiqaYzTwrQ+rttsTqtx5I6Yx3cSOoyauY0WI+sb9SmbvQQ22ekRDcXBzSzOrn2UqEMU2LMuVpz1wdcM1wdRdCYF4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=SbE1oFdU; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SbE1oFdU" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E020D19E7; Thu, 24 Jul 2025 16:09:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366199; bh=mLUIKlRtGcg7V5Vy0wqmsYRGL64MqtvU55q1oBc3ZZs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SbE1oFdU/9H7Yu3GahmaFUqLyaAui8STmt0HJnCtwimhy+oGAPunQC/FQrQ7DK0Ag Hk1mYaumrjMNeWRqA1UmElJKnORD20lporB9xxFuKTLWKvy+ZxPMM0EL4w7LLlySDa 3jdpxVhF5/G8x9zZZIID0HEK1SxYRA2uv+kTlFPM= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:17 +0200 Subject: [PATCH v2 10/27] media: v4l2-dev: Add video_device_context_from_queue() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-10-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3081; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=mLUIKlRtGcg7V5Vy0wqmsYRGL64MqtvU55q1oBc3ZZs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7P1lFdRKpeT4WwbjOonQa/xpkPdT4p0PKa2 Xi8BELxuYSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PK02D/sFpbStQiARNqc+O3WRoH+zz2HEs8iJBJmfbY53V1wZTRx3C3Mf1tPR6RhhSLrvYWvD/WQ /ogIA3hUZpE6/i/ZQbfOmyz3s7jWnbkOXI2XsTy3/2v7I8I4ObSjudBEGz8frjiE6q8Jtt+IzLU AXxjRzDRij9+ttBPtVM7ZpzSux7lGzFg4Ouec/d14H0RgpxEq++WvHBSXUj6m2gTeJc4Ls4ZZ1/ AKjJO4xSG1z8xNRJuQQcMgc+qEpgvDjzBQLOXh9oBOjgazvY7ufCiH/01XUwyojS/9LB5DZJ7hB wLXjctGBl5M8I/jO/PHQB+21XeJ/xIbcN5XSONqigchqlzh6HtSothepHU27Yf48O9tPRWp/YuM ZYEQbWSPNzoxjvV1RfbHt9tUJko5+bkCJlt81CuzTdfckyf7CY9RUiGR8SGK/bMdb+qmtlU/j2/ xhCglKVHD/559xTS8QSRGyalVugUHWBPrn12gECLXTROfkZUQw1XYeDvqZcWlxm3m+868PJZP/D 5Vs14E8+ugBYLOv2D1l/zluq190d5zkWObZhcd5AivkkO6E/kA9igGkDhbD4Lamlve8EWv/aJtm EDVaRd+PqQpSDCQ4EAn7DExd4x9xSwLLM2bipJtXE4jPcIp+vj+eJndeIWLa8HG9mQohTjvZaGs sG5370v6UbI38+w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce an helper function to get a video_device_context from a videobuf2 queue. Device drivers that support multi-context operations will receive in their vb2_ops callbacks implementation a vb2_queue argument that is embedded in a video device context, either a context created by binding the video device to a media device context or the video device default context. This helper allows those drivers to retrieve the context the vb2_queue is embedded with. As the intended callers of this helpers are vb2_ops callbacks implementation, called by the videobuf2 core in response to an operation on a file handle, the returned context is guaranteed to be valid for the whole duration of the callback. Signed-off-by: Jacopo Mondi --- include/media/v4l2-dev.h | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index a41afb81663bc9cb3bfc06dcf9b11ceeaf7c3415..bab4b13b109362bec84d8d16440= b6ea895206b60 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -819,6 +819,42 @@ void video_device_cleanup_context(struct video_device_= context *ctx); struct video_device_context * video_device_context_from_file(struct file *filp, struct video_device *vde= v); =20 +/** + * video_device_context_from_queue - Get a video device context associated= with + * a vb2 queue + * + * @q: the videobuf2 queue + * + * Return the video device context this videobuf2 queue belongs to. + * + * Device drivers that support multi-context operations will always be pro= vided + * with a device context to work with, either a context created by userspa= ce + * by binding the video device to a media device context or the video devi= ce + * default context. The videobuf2 queue that context-aware drivers operate= s will + * always be contained in either one of those contexts. + * + * This function should be used by driver-specific callbacks implementatio= n of + * the vb2_ops which receive a videobuf2 as argument. Being the vb2_ops ca= llback + * called by the videobuf2 core in response to a userspace operation on an= open + * file handle, callers of this function are guaranteed to always receive a + * valid context reference by this function. The reference will remain val= id for + * the whole function scope of the vb2_ops callback that has received the + * videobuf2 queue as argument. Likewise, accessing the media device conte= xt + * from the video device context returned by this function is always safe = within + * the same function scope. + * + * Drivers that do not support multi-context operations shall never use + * this function. + * + * This function does not increase the reference count of the returned vid= eo + * device context. + */ +static inline struct video_device_context * +video_device_context_from_queue(struct vb2_queue *q) +{ + return container_of(q, struct video_device_context, queue); +} + #endif /* CONFIG_MEDIA_CONTROLLER */ =20 #endif /* _V4L2_DEV_H */ --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECEC62E6D12; Thu, 24 Jul 2025 14:10:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366251; cv=none; b=XLyycLlFC5u4hGfnwR8G2AHGZ401E8du4v7BygYcJFcM0ObZpeto5S1ZZn2ImUCw0kbH+GF8kgmik8IIczOSE58n1B/HZVVyy6cQiSCyqzF6sPBUtUIcroqFgb/dVC6AdIrLEKHr6bUF9/lJvfID63KLEp6iL6kr5z10NC0ILaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366251; c=relaxed/simple; bh=xCWgxUWmumR8zNj2gqTiVA/oEQoilTZsmfWQf+Brozw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JF4GZe+wtxkzeN4HsTdC9DedZ+uOzjEETUR0BtxY/md8X4rDsZQTzdbO2L9dXR/5zALAqzy1rX8+1AMFzqnMWWTrJO98xXRhzeFKOb8wmQ2Ru6fJ6GPugcFcoI79LzBKlrj3V925U9QFoCv1fDXe6gYNkzDZdSHtAmA5sxj7IGI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=FFC/5UcT; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FFC/5UcT" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 192281AE0; Thu, 24 Jul 2025 16:10:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366201; bh=xCWgxUWmumR8zNj2gqTiVA/oEQoilTZsmfWQf+Brozw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=FFC/5UcTJ5je+ep01rbYWGenO7M3cg2X6U5P/iHuLmtHmYRU2FED9qBd5Agl+rS1V E5h8Y3CqTnavobCobLHnYDQ04n6+AOWnZ3mlP9MWR0f/M6Vf5X/OlN1b+Z6Iqr8lZ6 CaAM1faMrxAZ/ZvxRdq5ndhBbKR52Yfgz2DQk02A= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:18 +0200 Subject: [PATCH v2 11/27] media: videobuf2-v4l2: Support vb2_queue embedded in a context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-11-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12219; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=xCWgxUWmumR8zNj2gqTiVA/oEQoilTZsmfWQf+Brozw=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7PkODGmQchPp/R/KD+SQTS1LSqZTXjXhY+z HDmSU6jEg2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PIgcD/9pXOiyoLLXbLspCvvQd+I1Wvtk6wl4GCCQiMTO8K7EAxiBe50Y92MH7i+L/iOECc57mkW O+JEN4hdUDtUTVWpxz25T/u3Qb8DG8J4dlRLlN3LEK7/rtorquAcmm7ZURajHR5d+I6n03k/wQK KMKI3/BuUxZEgX1oohwB7W5bVr+EDeZ7VME9vvUFbqsGvAYMtSpSrN12Zjl9oc/rEV+4CkIY/d6 2SYb49sEqog64UJKM4NnhI/xYNiHgQTPibuP9qc3/sZ/jUwWxFb1jqc5tsS4LCB4j5YYgY/NL0A GnxnkiuF1/pYYUAyNQ2sCbUxj+ToWvGZ10h2w6N22Sa59cT1CfxIa3bXV5J6Szsh2a6XC4gQTSP OPj/9RNOOs0kUGjTNTWJCd5BmpEOn6SEzsSWR1unYFu1tqzKNn63SPxO8N6ToO9kXc/AFw9eGXq JWH8VBIuMP85MnRqNEZ5hCi9eNXz+pzTsAt8Bapoup/xRvUCXPsDwKlYZoTrJSEjSN0JzWJ+TrV i5S4CyU5P7q8YiDVgNml/cM9a7ZdGZvS4IADNMvcw5t5p4S8buJriO3FZZPRUtxfEkFNZ5tqMeu 9xEwo7zgVseUP0DyVRslwr0kQnVEG/bg/ddXx/FHr4CS/IwXnRa3d/xKn9ZoyvnyIMbZNnLO+B0 NWVw8v9Y9ZTztiA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Support device drivers that implement multi-context operations in the videobuf2 core by providing an helper to retrieve the vb2_queue from the context associated with an open file handle. If no context is associated with a file handle, retrieve it from the video device default context, created by the core for multi-context aware drivers. Fall-back to use the vb2_queue from the video_device to support existing drivers which are not context aware. Signed-off-by: Jacopo Mondi --- drivers/media/common/videobuf2/videobuf2-v4l2.c | 139 +++++++++++++++-----= ---- 1 file changed, 88 insertions(+), 51 deletions(-) diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/medi= a/common/videobuf2/videobuf2-v4l2.c index 1cd26faee50338aefeb670c6865da7c2d43f44d3..ad7892551f54c5abc7db7641bb8= fb89d77b1337f 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -998,23 +998,46 @@ EXPORT_SYMBOL_GPL(vb2_poll); * and so they simplify the driver code. */ =20 +/* + * Helper to get the vb2 queue either from: + * 1) The video context bound to the open file handle + * 2) The default context for context-aware drivers if userspace has not b= ound + * a context to the file handle + * 3) From the video device for non-context aware drivers + */ +static struct vb2_queue *get_vb2_queue(struct file *file, + struct video_device *vdev) +{ +#if defined(CONFIG_MEDIA_CONTROLLER) + struct video_device_context *ctx =3D + video_device_context_from_file(file, vdev); + + return ctx ? &ctx->queue + : vdev->default_context ? &vdev->default_context->queue + : vdev->queue; +#else + return vdev->queue; +#endif /* CONFIG_MEDIA_CONTROLLER */ +} + /* vb2 ioctl helpers */ =20 int vb2_ioctl_remove_bufs(struct file *file, void *priv, struct v4l2_remove_buffers *d) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - if (vdev->queue->type !=3D d->type) + if (q->type !=3D d->type) return -EINVAL; =20 if (d->count =3D=3D 0) return 0; =20 - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; =20 - return vb2_core_remove_bufs(vdev->queue, d->index, d->count); + return vb2_core_remove_bufs(q, d->index, d->count); } EXPORT_SYMBOL_GPL(vb2_ioctl_remove_bufs); =20 @@ -1022,21 +1045,21 @@ int vb2_ioctl_reqbufs(struct file *file, void *priv, struct v4l2_requestbuffers *p) { struct video_device *vdev =3D video_devdata(file); - int res =3D vb2_verify_memory_type(vdev->queue, p->memory, p->type); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); + int res =3D vb2_verify_memory_type(q, p->memory, p->type); u32 flags =3D p->flags; =20 - vb2_set_flags_and_caps(vdev->queue, p->memory, &flags, - &p->capabilities, NULL); + vb2_set_flags_and_caps(q, p->memory, &flags, &p->capabilities, NULL); p->flags =3D flags; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - res =3D vb2_core_reqbufs(vdev->queue, p->memory, p->flags, &p->count); + res =3D vb2_core_reqbufs(q, p->memory, p->flags, &p->count); /* If count =3D=3D 0, then the owner has released all buffers and he is no longer owner of the queue. Otherwise we have a new owner. */ if (res =3D=3D 0) - vdev->queue->owner =3D p->count ? file->private_data : NULL; + q->owner =3D p->count ? file->private_data : NULL; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_reqbufs); @@ -1045,11 +1068,12 @@ int vb2_ioctl_create_bufs(struct file *file, void *= priv, struct v4l2_create_buffers *p) { struct video_device *vdev =3D video_devdata(file); - int res =3D vb2_verify_memory_type(vdev->queue, p->memory, p->format.type= ); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); + int res =3D vb2_verify_memory_type(q, p->memory, p->format.type); =20 - p->index =3D vb2_get_num_buffers(vdev->queue); - vb2_set_flags_and_caps(vdev->queue, p->memory, &p->flags, - &p->capabilities, &p->max_num_buffers); + p->index =3D vb2_get_num_buffers(q); + vb2_set_flags_and_caps(q, p->memory, &p->flags, &p->capabilities, + &p->max_num_buffers); /* * If count =3D=3D 0, then just check if memory and type are valid. * Any -EBUSY result from vb2_verify_memory_type can be mapped to 0. @@ -1058,12 +1082,12 @@ int vb2_ioctl_create_bufs(struct file *file, void *= priv, return res !=3D -EBUSY ? res : 0; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; =20 - res =3D vb2_create_bufs(vdev->queue, p); + res =3D vb2_create_bufs(q, p); if (res =3D=3D 0) - vdev->queue->owner =3D file->private_data; + q->owner =3D file->private_data; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_create_bufs); @@ -1072,69 +1096,76 @@ int vb2_ioctl_prepare_buf(struct file *file, void *= priv, struct v4l2_buffer *p) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_prepare_buf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf); =20 int vb2_ioctl_querybuf(struct file *file, void *priv, struct v4l2_buffer *= p) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 /* No need to call vb2_queue_is_busy(), anyone can query buffers. */ - return vb2_querybuf(vdev->queue, p); + return vb2_querybuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_querybuf); =20 int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_qbuf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf); =20 int vb2_ioctl_dqbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_dqbuf(vdev->queue, p, file->f_flags & O_NONBLOCK); + return vb2_dqbuf(q, p, file->f_flags & O_NONBLOCK); } EXPORT_SYMBOL_GPL(vb2_ioctl_dqbuf); =20 int vb2_ioctl_streamon(struct file *file, void *priv, enum v4l2_buf_type i) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamon(vdev->queue, i); + return vb2_streamon(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamon); =20 int vb2_ioctl_streamoff(struct file *file, void *priv, enum v4l2_buf_type = i) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamoff(vdev->queue, i); + return vb2_streamoff(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamoff); =20 int vb2_ioctl_expbuf(struct file *file, void *priv, struct v4l2_exportbuff= er *p) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_expbuf(vdev->queue, p); + return vb2_expbuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); =20 @@ -1143,20 +1174,22 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - return vb2_mmap(vdev->queue, vma); + return vb2_mmap(q, vma); } EXPORT_SYMBOL_GPL(vb2_fop_mmap); =20 int _vb2_fop_release(struct file *file, struct mutex *lock) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 if (lock) mutex_lock(lock); - if (!vdev->queue->owner || file->private_data =3D=3D vdev->queue->owner) { - vb2_queue_release(vdev->queue); - vdev->queue->owner =3D NULL; + if (!q->owner || file->private_data =3D=3D q->owner) { + vb2_queue_release(q); + q->owner =3D NULL; } if (lock) mutex_unlock(lock); @@ -1167,7 +1200,8 @@ EXPORT_SYMBOL_GPL(_vb2_fop_release); int vb2_fop_release(struct file *file) { struct video_device *vdev =3D video_devdata(file); - struct mutex *lock =3D vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q =3D get_vb2_queue(file, vdev); + struct mutex *lock =3D q->lock ? q->lock : vdev->lock; =20 return _vb2_fop_release(file, lock); } @@ -1177,19 +1211,20 @@ ssize_t vb2_fop_write(struct file *file, const char= __user *buf, size_t count, loff_t *ppos) { struct video_device *vdev =3D video_devdata(file); - struct mutex *lock =3D vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q =3D get_vb2_queue(file, vdev); + struct mutex *lock =3D q->lock ? q->lock : vdev->lock; int err =3D -EBUSY; =20 - if (!(vdev->queue->io_modes & VB2_WRITE)) + if (!(q->io_modes & VB2_WRITE)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - err =3D vb2_write(vdev->queue, buf, count, ppos, - file->f_flags & O_NONBLOCK); - if (vdev->queue->fileio) - vdev->queue->owner =3D file->private_data; + err =3D vb2_write(q, buf, count, ppos, + file->f_flags & O_NONBLOCK); + if (q->fileio) + q->owner =3D file->private_data; exit: if (lock) mutex_unlock(lock); @@ -1201,20 +1236,21 @@ ssize_t vb2_fop_read(struct file *file, char __user= *buf, size_t count, loff_t *ppos) { struct video_device *vdev =3D video_devdata(file); - struct mutex *lock =3D vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q =3D get_vb2_queue(file, vdev); + struct mutex *lock =3D q->lock ? q->lock : vdev->lock; int err =3D -EBUSY; =20 - if (!(vdev->queue->io_modes & VB2_READ)) + if (!(q->io_modes & VB2_READ)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - vdev->queue->owner =3D file->private_data; - err =3D vb2_read(vdev->queue, buf, count, ppos, + q->owner =3D file->private_data; + err =3D vb2_read(q, buf, count, ppos, file->f_flags & O_NONBLOCK); - if (!vdev->queue->fileio) - vdev->queue->owner =3D NULL; + if (!q->fileio) + q->owner =3D NULL; exit: if (lock) mutex_unlock(lock); @@ -1225,7 +1261,7 @@ EXPORT_SYMBOL_GPL(vb2_fop_read); __poll_t vb2_fop_poll(struct file *file, poll_table *wait) { struct video_device *vdev =3D video_devdata(file); - struct vb2_queue *q =3D vdev->queue; + struct vb2_queue *q =3D get_vb2_queue(file, vdev); struct mutex *lock =3D q->lock ? q->lock : vdev->lock; __poll_t res; void *fileio; @@ -1241,7 +1277,7 @@ __poll_t vb2_fop_poll(struct file *file, poll_table *= wait) =20 fileio =3D q->fileio; =20 - res =3D vb2_poll(vdev->queue, file, wait); + res =3D vb2_poll(q, file, wait); =20 /* If fileio was started, then we have a new queue owner. */ if (!fileio && q->fileio) @@ -1257,8 +1293,9 @@ unsigned long vb2_fop_get_unmapped_area(struct file *= file, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { struct video_device *vdev =3D video_devdata(file); + struct vb2_queue *q =3D get_vb2_queue(file, vdev); =20 - return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags); + return vb2_get_unmapped_area(q, addr, len, pgoff, flags); } EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area); #endif --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49D1C18BBB9; Thu, 24 Jul 2025 14:10:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366254; cv=none; b=RLxbIiy8VTyxOsS2Vk7kjLW0060KU0cCpZKNmD2kqaQPOt5tM59OxON/xRRmYCvWnTu5r2GmiDLBY/zXY4iAFy1wNCJBIADduoTu3lkkwSVUyPVsJ+7Z/ptY/PDeWSdSNU4WS4b9DitrY2NSZncpk/GvFcA7OR5fjvM/mZBKs68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366254; c=relaxed/simple; bh=7I13wywrt99Sx2g/tHAlEPxkYo6PoehZ9YwsTffq1K4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Ljf4pUU+1ywEv1iYquKdpEodBWFBEoLQOPk29Fnrmrcogq2cSnj5YD6g0s12eQAHeIeeu7IlmKciyKy2BHDi98N4ECGpe0mt0m5OI1CK2eVm8fAhxOcqopmRq6USklsN34Ob1NzcHdF+dtajXUhsz9DpsGTLCEuodav7lRPpaZk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=tYwCCeD2; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tYwCCeD2" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 068741C0A; Thu, 24 Jul 2025 16:10:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366203; bh=7I13wywrt99Sx2g/tHAlEPxkYo6PoehZ9YwsTffq1K4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=tYwCCeD2RqUwRsIjzK/MrXphoDB/aUZK2i51QbchEsEofYFWAYY9BWRVDRwoQrb0f ZJ4puZMiIqlUpeGTo7a1ewbO8ezql4t78FRdbZiUK+tp6HuhmL0JMe377jIBJvRn4J FqHG72/aODRUtH9z3YgXHA4vGaeirty6Xf/rqx6E= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:19 +0200 Subject: [PATCH v2 12/27] media: v4l2-subdev: Introduce v4l2 subdev context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-12-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8677; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=7I13wywrt99Sx2g/tHAlEPxkYo6PoehZ9YwsTffq1K4=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7PHgrGE7vDI9TcLTP8iMB0VR5TMItrWF5wa lyRyEwBRYSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PI2OD/9XBflSxgvDX/sRzW+/cTbUgb+j2/OmOGJWBKOUskoqBrq3JTG3Kf0SemIUwimvXXUO+hP Pgl0X+K+rziXGIGT1/0dA/PGsGZNg6emZs7zrpe5KG2yVki1n2WoOUkaftqKDmG/ZEnft615X5K 3aJbfWBGJsLrnb8XYWf3uu4W5sfVwtvaeGVZuj6t2KB1O/joB5omnNvBJPKm22SLVAILRaDVokX jBzBtlCuRwXBVeLvaCkVttjCNiQgP0FVy81CN0xK7P3OCRsb2H4ecff6ebhwqHMhyjFbDKQsOXb O2NYMT5Sdh4FbRzoF5t6HlEhJlKbV9P09M2DbUXHGbF5i6tSJbI/kXh6IBAtZGc3AwhhEIj3KPc +0rMebILHqugZU7ElmzLzq/feayH3wxmWdqbsds8l3p+/eqvaaR8OT8O2u1gF3mFKtlc7oKpt+j SrkwMPmr57FtjLD909hYgR8y01o21uRO8HXxAL7HFOuGf6kHCVrklQ4pg8BLst/pg0lC6ac2WNB NM6Nnv+O7CKjyMyceiXuKcbHLvLrA/5sWBv7iIaBHWzOz5qkIog67Krz/CvSsOuepVmSBmudZWh 663+6udP5m34hl3sSGXzQJXRRQqC3wy5P8zvqavp5rgTqEif0PO1lS+nGgarfd+p6Se/aA48PA1 8TlhNvKNT4hiIJg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce a new type in v4l2 subdev that represents a v4l2 subdevice contex. It extends 'struct media_entity_context' and is intended to be extended by drivers that can store driver-specific information in their derived types. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 39 +++++++++++ include/media/v4l2-subdev.h | 126 ++++++++++++++++++++++++++++++= ++++ 2 files changed, 165 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-cor= e/v4l2-subdev.c index 4fd25fea3b58477056729665706ddbacc436379c..7307f57439499c8d5360c89f492= 944828ac23973 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1577,6 +1577,45 @@ bool v4l2_subdev_has_pad_interdep(struct media_entit= y *entity, } EXPORT_SYMBOL_GPL(v4l2_subdev_has_pad_interdep); =20 +struct v4l2_subdev_context * +v4l2_subdev_context_get(struct media_device_context *mdev_context, + struct v4l2_subdev *sd) +{ + struct media_entity *entity =3D &sd->entity; + struct media_entity_context *ctx =3D + media_device_get_entity_context(mdev_context, entity); + + if (!ctx) + return NULL; + + return container_of(ctx, struct v4l2_subdev_context, base); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_context_get); + +void v4l2_subdev_context_put(struct v4l2_subdev_context *ctx) +{ + if (!ctx) + return; + + media_entity_context_put(&ctx->base); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_context_put); + +int v4l2_subdev_init_context(struct v4l2_subdev *sd, + struct v4l2_subdev_context *context) +{ + media_entity_init_context(&sd->entity, &context->base); + + return 0; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_init_context); + +void v4l2_subdev_cleanup_context(struct v4l2_subdev_context *context) +{ + media_entity_cleanup_context(&context->base); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_cleanup_context); + struct v4l2_subdev_state * __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name, struct lock_class_key *lock_key) diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 5dcf4065708f32e7d3b5da003771810d5f7973b8..9d257b859acafb11cfe6976e906= e7baabd0206f6 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -757,6 +757,78 @@ struct v4l2_subdev_state { struct v4l2_subdev_stream_configs stream_configs; }; =20 +/** + * struct v4l2_subdev_context - The v4l2 subdevice context + * @base: The media entity context base class member + * @state: The subdevice state associated with this context + * + * This structure represents an isolated execution context of a subdevice. + * This type 'derives' the base 'struct media_entity_context' type which + * implements refcounting on our behalf and allows instances of this type = to be + * linked in the media_device_context contexts list. + * + * The subdevice context stores the subdev state in a per-file handle cont= ext, + * userspace is allowed to multiplex the usage of a subdevice devnode by o= pening + * it multiple times and by associating it with a media device context. Th= is + * operation is called 'bounding' and is performed using the + * VIDIOC_SUBDEV_BIND_CONTEXT ioctl. + * + * A subdevice context is created and stored in the v4l2_fh file handle + * associated with an open file descriptor when a subdevice is 'bound' to a + * media device context. The 'bounding' operation realizes a permanent + * association valid until the subdevice context is released. + * + * A subdevice can be bound to the same media device context once only. + * Trying to bind the same subdevice to the same media device context a + * second time, without releasing the already established context by closi= ng the + * bound file descriptor first, will result in an error. + * + * To create a subdevice context userspace shall use the + * VIDIOC_SUBDEV_BIND_CONTEXT ioctl that creates the subdevice context and + * uniquely associates it with a media device file descriptor. + * + * Once a subdevice file descriptor has been bound to a media device conte= xt, + * all the operations performed on the subdevice file descriptor will be + * directed on the just created subdevice context. This means, in example,= that + * the subdevice state and configuration is isolated from the ones associa= ted + * with a different file descriptor obtained by opening again the same sub= device + * devnode but bound to a different media device context. + * + * Drivers that implement multiplexing support have to provide a valid + * implementation of the context-related operations in the media entity + * operations. + * + * Drivers are allowed to sub-class the v4l2_subdevice_context structure by + * defining a driver-specific type which embeds a struct v4l2_subdevice_co= ntext + * instance as first member, and allocate the driver-specific structure si= ze in + * their implementation of the `alloc_context` operation. + * + * Subdevice contexts are ref-counted by embedding an instance of 'struct + * media_entity_context' and are freed once all the references to it are + * released. + * + * A subdevice context ref-count is increased when: + * - The context is created by bounding a video device to a media device c= ontext + * - The media pipeline it is part of starts streaming + * A subdevice context ref-count is decreased when: + * - The associated file handle is closed + * - The media pipeline it is part of stops streaming + * + * The ref-count is increased by a call to v4l2_subdev_context_get() and is + * reponsibility of the caller to decrease the reference count with a call= to + * v4l2_subdev_context_put(). + */ +struct v4l2_subdev_context { + struct media_entity_context base; + /* + * TODO: active_state should most likely be changed from a pointer to an + * embedded field. For the time being it's kept as a pointer to more + * easily catch uses of active_state in the cases where the driver + * doesn't support it. + */ + struct v4l2_subdev_state *state; +}; + /** * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations * @@ -1152,6 +1224,7 @@ struct v4l2_subdev_fh { struct module *owner; #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) struct v4l2_subdev_state *state; + struct v4l2_subdev_context *context; u64 client_caps; #endif }; @@ -1285,6 +1358,59 @@ int v4l2_subdev_link_validate(struct media_link *lin= k); bool v4l2_subdev_has_pad_interdep(struct media_entity *entity, unsigned int pad0, unsigned int pad1); =20 +/** + * v4l2_subdev_context_get - Helper to get a v4l2 subdev context from a + * media device context + * + * @mdev_context: The media device context + * @sd: The V4L2 subdevice the context refers to + * + * Helper function that wraps media_device_get_entity_context() and returns + * the v4l2 subdevice context associated with a subdevice in a media device + * context. + * + * The reference count of the returned v4l2 subdevice context is increased. + * Callers of this function are required to decrease the reference count of + * the context reference with a call to v4l2_subdev_context_put(). + */ +struct v4l2_subdev_context * +v4l2_subdev_context_get(struct media_device_context *mdev_context, + struct v4l2_subdev *sd); + +/** + * v4l2_subdev_context_put - Helper to decrease a v4l2 subdevice context + * reference count + * + * @ctx: The v4l2 subdevice context to put + */ +void v4l2_subdev_context_put(struct v4l2_subdev_context *ctx); + +/** + * v4l2_subdev_init_context - Initialize the v4l2 subdevice context + * + * @sd: The subdevice the context belongs to + * @ctx: The context to initialize + * + * Initialize the v4l2 subdevice context. The intended callers of this fun= ction + * are driver-specific implementations of the media_entity_ops.alloc_conte= xt() + * function that allocates their driver specific types that derive from + * struct v4l2_subdev_context. + */ +int v4l2_subdev_init_context(struct v4l2_subdev *sd, + struct v4l2_subdev_context *ctx); + +/** + * v4l2_subdev_cleanup_context - Cleanup the v4l2 subdevice context + * + * @ctx: The context to cleanup. + * + * Cleanup the v4l2 subdevice context. The intended callers of this functi= on are + * driver specific implementation of the media_entity_ops.destroy_context() + * function before releasing the memory previously allocated by + * media_entity_ops.alloc_context(). + */ +void v4l2_subdev_cleanup_context(struct v4l2_subdev_context *ctx); + /** * __v4l2_subdev_state_alloc - allocate v4l2_subdev_state * --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8669D2E7F02; Thu, 24 Jul 2025 14:10:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366254; cv=none; b=DVJF7vQkTC4hLNYn0o1W432+jhEaVyAlIKaeGT0fVpnl5YRsyNYdrPF0euYR4XDYRToRtt/PspaHma4SJCLbKAGROf9yu3EUR3lcIicj9HzCKeDT8iACEQvav2hkxmGm1hPUVPe+3r0IIx2D5JbnP8SvBv3STMifq8KLO0jdTmk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366254; c=relaxed/simple; bh=V4Bd2SEp0ALBPlJWG8EsVuj29YkOY/732hyVjUTvCTo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Eum+SeWxgXP2aWGDrRBwJdxQwWo1oosJQzbk2trrbRVVyyIgi6U01kpCm/eVwAnofnSz12LIYaUL4S3OA17naihEhspBdBgzzujYMr2HdMo4+gzib6kVRouQK/J5jIfpt/il9yKszw5N4KRTEXUF3sJ44v5Kk19Wto83k8m/fV4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=XRjXhEEn; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XRjXhEEn" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 39D35C64; Thu, 24 Jul 2025 16:10:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366204; bh=V4Bd2SEp0ALBPlJWG8EsVuj29YkOY/732hyVjUTvCTo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=XRjXhEEn13H6aY3oZsxLJY55/L9Tw65ogbxXhwI8kEvn9K4oZ1Re0eOFtrTRAbZH5 kTHHgh06nTrI2tB/Zh8lEqR+YGZj6NLUTes5ev8kZ34Qpa7Ukj5XUbQ6HAvMBdQnDF aBk57paYgPL80JZd2zKCgWkJTo5Vu0czYxv7Vx9c= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:20 +0200 Subject: [PATCH v2 13/27] media: v4l2-subdev: Introduce VIDIOC_SUBDEV_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-13-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4492; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=V4Bd2SEp0ALBPlJWG8EsVuj29YkOY/732hyVjUTvCTo=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7PdCfw+hprkRoPY/A7yYedTP/OgpDEkO/va XN9lVbNkHyJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PPgdD/4lyAybO0SI0vYiBbeWcj1BoSkTxEhslUbbePT7t3GYV30NNOF33+zC1FvxXCBrZVWheIv ExXotbLPGo+5ylE27Ds1z+iwCN8WitxhBphcGk/XU/9B/Q4unN8yWhl7spTmQWjsAydN+nvmIlm /ucxxvAiMCMk4atU0f2mOheMaZomAOp5aUbtihQ5tOhyUyzvQmADlhdz26CDjou9xO1eIzJQRu1 1ElveqRlXNyJEpQyQn/O1EGsLyC0ZorJMK2EQokUU/TBtJPUN5gcd2FYE31+7mGHSUrlbJwmJX4 g/zQenllabIdoXMY0R8FKmh9are6EsXij64oDTi0lFzcWJoePUnyqagTaPYFdM6x2RzD4UGmbl0 ho4N1l2MVqhCw4zIPu4lVWNRo5WThc1M+mrcK3ZMH2J2aJlckYz09UUxlLoM4NLS3j8AUQQnIeg KNnJmwvpJOeYsG8yCoy4idIYa06BbjizQRPw9OtVvgtyNrXWTKo2LTVeU6dEEKsxqxbNlMSNIqd URXJBoLhqi44hpsVUDun+2tPCY4LDd5pOBwvZhGti6pScMH7CtdR/4r2lFelG0XD+HysKdrQj+W ZzbipuLEVmacO1Y9BCxM3ZRfVCR84yDkoUTMT/+5iv+3/1QsY8W701xPrEdFTedp00QCvyJvMQA h4XBGzu5r5fOU2Q== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce a new ioctl in V4L2 subdev to allocate a subdevice context and associate it with a media device context. The ioctl handler calls the entity ops to let the driver allocate a new context and initialize the subdev state there contained. The new subdevice context is bound to the media device context associated with the file descriptor provided by userspace as the new ioctl argument. The newly allocated context is stored in the v4l2_subdev_fh handle that represent the open file handle on which the ioctl has been called. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 59 +++++++++++++++++++++++++++++++= ++++ include/uapi/linux/v4l2-subdev.h | 11 +++++++ 2 files changed, 70 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-cor= e/v4l2-subdev.c index 7307f57439499c8d5360c89f492944828ac23973..300f84317623dd082a4cd2caec9= 7057f972e82a3 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -607,6 +607,46 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct = v4l2_subdev_fh *subdev_fh, v4l2_subdev_get_unlocked_active_state(sd); } =20 +static int subdev_do_bind_context(struct v4l2_subdev *sd, + struct v4l2_subdev_context **context, + struct media_device_context *mdev_context) +{ + static struct lock_class_key key; + struct v4l2_subdev_state *state; + int ret; + + ret =3D sd->entity.ops->alloc_context(&sd->entity, + (struct media_entity_context **) + context); + if (ret) + return ret; + + state =3D __v4l2_subdev_state_alloc(sd, "context->state->lock", &key); + if (IS_ERR(state)) { + ret =3D PTR_ERR(state); + goto err_put_context; + } + (*context)->state =3D state; + + /* + * Bind the newly created video device context to the media device + * context identified by the file descriptor. + */ + ret =3D media_device_bind_context(mdev_context, + (struct media_entity_context *) + *context); + if (ret) + goto err_free_state; + + return 0; + +err_free_state: + __v4l2_subdev_state_free((*context)->state); +err_put_context: + v4l2_subdev_context_put(*context); + return ret; +} + static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, struct v4l2_subdev_state *state) { @@ -1089,6 +1129,25 @@ static long subdev_do_ioctl(struct file *file, unsig= ned int cmd, void *arg, return 0; } =20 + case VIDIOC_SUBDEV_BIND_CONTEXT: { + struct v4l2_subdev_bind_context *c =3D arg; + struct media_device_context *mdev_context; + int ret; + + if (!sd->entity.ops || !sd->entity.ops->alloc_context || + !sd->entity.ops->destroy_context) + return -ENOTTY; + + mdev_context =3D media_device_context_get_from_fd(c->context_fd); + if (!mdev_context) + return -EINVAL; + + ret =3D subdev_do_bind_context(sd, &subdev_fh->context, + mdev_context); + media_device_context_put(mdev_context); + return ret; + } + case VIDIOC_SUBDEV_G_CLIENT_CAP: { struct v4l2_subdev_client_capability *client_cap =3D arg; =20 diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-sub= dev.h index 2347e266cf7516b4073c1edd43b97a3ddddb183b..6184cc0153a9dd9fbfa6729e1b6= 127ba4a961395 100644 --- a/include/uapi/linux/v4l2-subdev.h +++ b/include/uapi/linux/v4l2-subdev.h @@ -243,6 +243,16 @@ struct v4l2_subdev_routing { __u32 reserved[11]; }; =20 +/** + * struct v4l2_subdev_bind_context - Subdev context information + * + * @context_fd: The file descriptor of the media_device instance the subde= vice + * has to be bound to + */ +struct v4l2_subdev_bind_context { + __u32 context_fd; +}; + /* * The client is aware of streams. Setting this flag enables the use of 's= tream' * fields (referring to the stream number) with various ioctls. If this is= not @@ -285,6 +295,7 @@ struct v4l2_subdev_client_capability { #define VIDIOC_SUBDEV_S_SELECTION _IOWR('V', 62, struct v4l2_subdev_selec= tion) #define VIDIOC_SUBDEV_G_ROUTING _IOWR('V', 38, struct v4l2_subdev_routin= g) #define VIDIOC_SUBDEV_S_ROUTING _IOWR('V', 39, struct v4l2_subdev_routin= g) +#define VIDIOC_SUBDEV_BIND_CONTEXT _IOWR('V', 50, struct v4l2_subdev_bind= _context) #define VIDIOC_SUBDEV_G_CLIENT_CAP _IOR('V', 101, struct v4l2_subdev_cli= ent_capability) #define VIDIOC_SUBDEV_S_CLIENT_CAP _IOWR('V', 102, struct v4l2_subdev_cl= ient_capability) =20 --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74DE92E8E04; Thu, 24 Jul 2025 14:10:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366257; cv=none; b=PvwIQsQyhoCKrmeG8Z1V2dGlXxM3FfSZ6ClfGEu53aOLOvGCwb+UVm7qheYDCCng6GzGwVsXlhEpnda5yeT2Cddhh4rs8aPxL1UB7jZ9x13isUCquN3kWMrUkvpR5L/aII+WvcD7Y64WqhKBy1t4px/YDKQQq08RZJF/2GAfYiE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366257; c=relaxed/simple; bh=u88FSqcsc0N2ZFvrgby11vKsM4ytR+rXokZ729cwq9o=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fxKqg9BO0bqEaIc8vdlu3NUcecUlV125/GJaYdk9aElHfGJmP907mr1vkpPtcHEBysJ32++3PK4J2lSJ8cshI5xd1rnyo1FqRF+IzZUhr34hiCd+OiNFdO7xK3puXdxpUhmq09oZOcDxMwanBsPVeS7/2vui0acqPQEjg1XFinM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=fagt0BE+; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fagt0BE+" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 667371A04; Thu, 24 Jul 2025 16:10:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366205; bh=u88FSqcsc0N2ZFvrgby11vKsM4ytR+rXokZ729cwq9o=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=fagt0BE+diV9+AbBlekTOJdiPNc9GafAg6df8urVTH54TtuiOuc+JsvT6NYdgjLXN bKddHDJe8MFsWKz+fFdcy+kZhBK/DoO/FgptKqeFRblUI4DR9zFzAkC9LNDR+A/o3A /gB+AseHNs27lfHnjM6zMkhQM1uHUeA54TfOS0JA= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:21 +0200 Subject: [PATCH v2 14/27] media: Documentation: Add VIDIOC_SUBDEV_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-14-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4626; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=u88FSqcsc0N2ZFvrgby11vKsM4ytR+rXokZ729cwq9o=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7PW7QjD8qMwTZWuNE7c3zeDKhIIFd1UylOr B9O82Ilv2eJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PE5eD/0buLR1LLpJcJn9kC4eWv+0KZur9zub9kI1BX9gWa1EIB8n+0lF9Z67+hSsSxR9dXZmgvW WtKuLlkcMG9/NmXE68PdkQwi1DlaJOcE4EurwMVNclTQ7SBwSIlP1RC6lA0d6PPHQKNE82foEg4 aolXSHns/6BrZSWgKLYXi8mx/KJpQuilfJPCwdGcR7HYhw01ypp14n1NaOtFIaa4EnWACNX4Gxv CssDizDXc0oHSjmLrSUoWWtawSlONG6cZkq9r76Uhf3EefgtFxKCG4+oAFKVO2rc6HQfA1nXDvn KSZNNnm1anSksyTAQm3VYh3YT5d0XaW+DcFJ0OIzJc9gbKJKBoRVoPRnvqrLNQOqjKCh5ERgtkP ztF2L1UcGkErdKlAKOZnejnomjVTk2XVaXy1QUtjQJD4caxkzTvMZ3LPufJtSkIBCU/BC3ENKi7 ZkzyZttc4ji5CMX4C11CobV6+OjCxdgCNVrwhQDVcIGQXRoAiD5TzYhiUZkMkIpWtKFnWJ02Zfn sQJpi/Axm/DAR0xgZfal7cuvjdMXGKhzAudiHvF5H+iPwX0QvEk67NQxzKL0yak2XyyQIcJVj9X rD6Izu5650bFcxtR5mssVGKJH4wUiA8PnpYhG7/MEqyxB0PSoxdL2duP98Cf4lT6P3j5RvpOI/6 wFcuenFV9OXNInQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Document the newly introduced VIDIOC_SUBDEV_BIND_CONTEXT ioctl. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/user-func.rst | 1 + .../media/v4l/vidioc-subdev-bind-context.rst | 81 ++++++++++++++++++= ++++ 2 files changed, 82 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Document= ation/userspace-api/media/v4l/user-func.rst index 0d9aff56ab653b2a4f6afe4828f88bc5637addf1..236847a1d6cb2a266bea30ecf75= 83979099b0343 100644 --- a/Documentation/userspace-api/media/v4l/user-func.rst +++ b/Documentation/userspace-api/media/v4l/user-func.rst @@ -66,6 +66,7 @@ Function Reference vidioc-remove-bufs vidioc-s-hw-freq-seek vidioc-streamon + vidioc-subdev-bind-context vidioc-subdev-enum-frame-interval vidioc-subdev-enum-frame-size vidioc-subdev-enum-mbus-code diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-bind-conte= xt.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-bind-context.r= st new file mode 100644 index 0000000000000000000000000000000000000000..5cba529a3cdcb63c7257f871d66= 7fa792c0ca382 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-bind-context.rst @@ -0,0 +1,81 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: V4L + +.. _vidioc_subdev_bind_context: + +******************************** +ioctl VIDIOC_SUBDEV_BIND_CONTEXT +******************************** + +Name +=3D=3D=3D=3D + +VIDIOC_SUBDEV_BIND_CONTEXT - Bind a subdevice file handle to a media device +context + +Synopsis +=3D=3D=3D=3D=3D=3D=3D=3D + +.. c:macro:: VIDIOC_SUBDEV_BIND_CONTEXT + +``int ioctl(int fd, VIDIOC_SUBDEV_BIND_CONTEXT, struct v4l2_subdev_bind_co= ntext *argp)`` + +Arguments +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`v4l2_subdev_bind_context`. + +Description +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Applications call the ``VIDIOC_SUBDEV_BIND_CONTEXT`` ioctl to bind a subde= vice +file handle to a media device context. Binding a subdevice file handle to= a +media device context creates an isolated execution context which allows to +multiplex the usage of a video device. This means, in practice, that the +subdevice configuration (format, sizes etc) applied on a file handle bound= to a +media device context won't be visible on file handles bound to a different= media +device context (or not bound at all). + +By opening a media device applications create a media device context to wh= ich +video devices and subdevices file handles can be bound to. The file descri= ptor +returned by a call to :c:func:`open()` on the media device identifies uniq= uely +the media device context. Application populates the ``context_fd`` field of +:c:type:`v4l2_subdev_bind_context` with the file descriptor of an open med= ia +device to identify the media context to which they want to bind a subdevice +to. + +Applications can open a subdevice node multiple times, and call +``VIDIOC_BIND_CONTEXT`` on each file handle returned by a successful call = to +:c:func:`open()` to isolate the operations performed on that file handle f= rom +any operation performed on other file handles bound to different contexts.= This +means, in example, that the subdevice format and sizes are isolated from t= he +ones associated with a file descriptor, obtained by opening the same subde= vice +but bound to a different media device context (or not bound at all). + +The bounding operation realizes a permanent association valid until the +subdevice context is released by closing the file handle. + +A subdevice file handle can be bound to the same media device context once +only. Trying to bind the same file handle to the same media device context= a +second time, without releasing the already established context by closing = the +bound file descriptor first, will result in an error. + +Bounding is an opt-in feature that applications are free to ignore. Any +operation directed to a non bound file handle will continue to work as it = used +to, and the video device configuration (formats, sizes etc) will be visible +across all the other non-bound file handles. + +Return Value +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes ` chapter. + +EINVAL + The media device context file handle ``context_fd`` is not valid or the + subdevice file handle is already bound to a context. --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 933022E8E08; Thu, 24 Jul 2025 14:10:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366257; cv=none; b=iBjVFk7J84Etuf8SWbw027CGFuNlOUBpKTvqsUGihJhmUH/AWt2w3+ovCBbS3+827qIZv8x76gmaX9xLn66XB0PI3qQs8X/HCQbVprO1mFQho/7be59FehqjPd/9RWgtdrIcE14MgWIORQLRun11jHCnQRITEmiAKQtjPAeLlJY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366257; c=relaxed/simple; bh=RMEJm/E/usNBixZcp+GBwmSg7HkpwQhhGNile6Mrpfg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Re8j0F4Ww3udwl+Ho65navQ2cpLolB/0FefaSKbmYuXVtsmXMksfuwEe5kKVbs25Tdtl8ie+lJ6N6L5DxgzUAWv8ye07RO8OmVCJQ8l4+yA6Piw+aSsSJhtuMNZUkcJ7X3evlTu++ajU+D9heKOdnZWGyjCxzpCaEdt/+YeI8K8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=KldGWVzh; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KldGWVzh" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8D0D9190D; Thu, 24 Jul 2025 16:10:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366206; bh=RMEJm/E/usNBixZcp+GBwmSg7HkpwQhhGNile6Mrpfg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=KldGWVzhzELrAs96NQ9xfnzg0umN6bK5CgorjcgIdyNgJhUW7vG/LvV1YCSH8duET LaSJGklP/fPcnZYJRaAGKJchQGM8T9xCbFXV9UVMUWrK/qAx5QUxzVQD7Fx5nB4zlk C9TCugCnoeIO8mzUypMusl+C/3a24dd+XMNFyI64= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:22 +0200 Subject: [PATCH v2 15/27] media: v4l2_subdev: Introduce default context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-15-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5859; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=RMEJm/E/usNBixZcp+GBwmSg7HkpwQhhGNile6Mrpfg=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7P1/MKU1Ktuh3wvx1qB/QU79/dRKxCfwG1p 2Oo0BAz+F6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+zwAKCRByNAaPFqFW PCc7D/460AOKgiuGxo8oHW7UIhLM+ZqXE8WCJjPH6HxSRZm0imC+qVc+LJG8YwdZ7InirAUxNJW C9Gzu7a9wEZOtDl2dgVtgsMTKBq4NCk83bEnIz1hdbIpiq8xsGMCpJFJW34QOzJHXmvi2bcFcZd kIYJVjQ9nXdDLdPO0Hj1jqpI4noQ7q4cLaIRe+SxRdQ6lkK0fqmjrquYuoTBd4BqWfwBXO8gme9 0EJXGZxMRTBtEOH+pfNYQV3n363wrs+smCkWubOKvSGmVffaSAntOqqT41zr9EiwJ9hBm3rZj1G k6vcGDe70r0hId4CKCXPBLOfT67qnphMJlOCQsFgZQyXo3wU5VYjFGG+P5F0XxA+1EMmMBwdX7d qlAXIx4YoeZKm+qKNZx1obWbWGSSQdu1CKZLnO96x00IiJ5CNr7fVi1Xr6MNvBLfl5MR3HRemmf 2Xy30ENpV+uSdzsbUtFjOMXYwi/f56RnqR+L0xBKcQT9QnvM13hy3FBzvn8v8cN9f970tCueSFg JKnnhPGSP3CBSwDIJtl2hrPTp7VUJNONCVq4g7jh2b7TA/3WnRp5cEWwjxeZrT2yVwirDFlUUyt bTbxvDcc7wCzG8v+yZwZYQOV4XnA2Lt4vig1vJz6trijNEIxW3CGTO2IR0v4Wgk4D/Kz02GVHO2 /niTwqVtIfEzVVA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Introduce a default context for v4l2 subdvice. Drivers ported to use multi-context support that used to work with a non-context aware userspace (which doesn't call VIDIOC_SUBDEV_BIND_CONTEXT) shall continue to work even if they are context aware. Provide a default context in the v4l2 subdev and bind it to the media device default context when the subdevice is fully registered by providing a v4l2_subdev_registered() function. Release the context when the subdevice gets unregistered by the core, providing a v4l2_subdev_unregistered() helper. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-device.c | 11 +++----- drivers/media/v4l2-core/v4l2-subdev.c | 50 +++++++++++++++++++++++++++++++= ++++ include/media/v4l2-subdev.h | 29 ++++++++++++++++++++ 3 files changed, 83 insertions(+), 7 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-device.c b/drivers/media/v4l2-cor= e/v4l2-device.c index 5e537454f5cd71b3c50a2a2864642f7d5548047b..bf3ebd77b7bb8b13c849a89d01e= 6d889e8a2e4fd 100644 --- a/drivers/media/v4l2-core/v4l2-device.c +++ b/drivers/media/v4l2-core/v4l2-device.c @@ -146,11 +146,9 @@ int __v4l2_device_register_subdev(struct v4l2_device *= v4l2_dev, } #endif =20 - if (sd->internal_ops && sd->internal_ops->registered) { - err =3D sd->internal_ops->registered(sd); - if (err) - goto error_unregister; - } + err =3D v4l2_subdev_registered(sd); + if (err) + goto error_unregister; =20 sd->owner =3D module; =20 @@ -274,8 +272,7 @@ void v4l2_device_unregister_subdev(struct v4l2_subdev *= sd) list_del(&sd->list); spin_unlock(&v4l2_dev->lock); =20 - if (sd->internal_ops && sd->internal_ops->unregistered) - sd->internal_ops->unregistered(sd); + v4l2_subdev_unregistered(sd); sd->v4l2_dev =3D NULL; =20 #if defined(CONFIG_MEDIA_CONTROLLER) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-cor= e/v4l2-subdev.c index 300f84317623dd082a4cd2caec97057f972e82a3..438f51980e5ac0f092ba6b0a979= a376133968ddf 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1275,6 +1275,56 @@ const struct v4l2_file_operations v4l2_subdev_fops = =3D { .poll =3D subdev_poll, }; =20 +#ifdef CONFIG_MEDIA_CONTROLLER +static int v4l2_subdev_register_default_context(struct v4l2_subdev *sd) +{ + struct media_device_context *mdev_context; + + /* If the driver does not support contexts, return here. */ + if (!sd->entity.ops || !sd->entity.ops->alloc_context || + !sd->entity.ops->destroy_context) + return 0; + + mdev_context =3D sd->entity.graph_obj.mdev->default_context; + return subdev_do_bind_context(sd, &sd->default_context, mdev_context); +} +#endif /* CONFIG_MEDIA_CONTROLLER */ + +int v4l2_subdev_registered(struct v4l2_subdev *sd) +{ + int ret; + +#ifdef CONFIG_MEDIA_CONTROLLER + ret =3D v4l2_subdev_register_default_context(sd); + if (ret) + return ret; +#endif /* CONFIG_MEDIA_CONTROLLER */ + + if (sd->internal_ops && sd->internal_ops->registered) { + ret =3D sd->internal_ops->registered(sd); + if (ret) + goto err_registered; + } + + return 0; + +err_registered: + if (sd->default_context) + v4l2_subdev_context_put(sd->default_context); + return ret; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_registered); + +void v4l2_subdev_unregistered(struct v4l2_subdev *sd) +{ + if (sd->default_context) + v4l2_subdev_context_put(sd->default_context); + + if (sd->internal_ops && sd->internal_ops->unregistered) + sd->internal_ops->unregistered(sd); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_unregistered); + #ifdef CONFIG_MEDIA_CONTROLLER =20 int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity, diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 9d257b859acafb11cfe6976e906e7baabd0206f6..1fa42a9f322be0be44fc9308744= f4f4ae0cf1606 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -1130,6 +1130,10 @@ struct v4l2_subdev_platform_data { * @active_state: Active state for the subdev (NULL for subdevs tracking t= he * state internally). Initialized by calling * v4l2_subdev_init_finalize(). + * @default_context: Default context for the subdev, allows to operate + * context-aware drivers with a context-unaware userspace. + * It is initialized when the subdev is registered in + * v4l2_subdev_registered(). * @enabled_pads: Bitmask of enabled pads used by v4l2_subdev_enable_strea= ms() * and v4l2_subdev_disable_streams() helper functions for * fallback cases. @@ -1182,6 +1186,7 @@ struct v4l2_subdev { * doesn't support it. */ struct v4l2_subdev_state *active_state; + struct v4l2_subdev_context *default_context; u64 enabled_pads; bool s_stream_enabled; }; @@ -1286,6 +1291,30 @@ static inline void *v4l2_get_subdev_hostdata(const s= truct v4l2_subdev *sd) return sd->host_priv; } =20 +/** + * v4l2_subdev_registered - Subdevice registered notification + * + * @sd: The subdevice that has been registered + * + * Notify that a subdevice has been registered by the core. This function = wraps + * a call to sd->internal_ops->registered (if available) and instantiates = the + * default v4l2 subdevice context. + * + * Returns 0 on success, a negative error code otherwise. + */ +int v4l2_subdev_registered(struct v4l2_subdev *sd); + +/** + * v4l2_subdev_unregistered - Subdevice unregistered notification + * + * @sd: The subdevice that has been unregistered + * + * Notify that a subdevice has been unregistered by the core. This function + * wraps a call to sd->internal_ops->unregistered (if available) and delet= es + * the default v4l2 subdevice context. + */ +void v4l2_subdev_unregistered(struct v4l2_subdev *sd); + #ifdef CONFIG_MEDIA_CONTROLLER =20 /** --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8DF82E9EBA; Thu, 24 Jul 2025 14:10:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366260; cv=none; b=eAym+bHrCetYIGzSTSSqX3vgxTG/PfIAVI67/E6PFrlikWEExHqgncUiNlWA7tNVbWwhqgQqAIgM1arQrHeSa+5waZozPyvoVybVKK3pp4PM3I0VZSADBnuKm8tSXJHINmVN3VEUuhpH2tQEZxEQaEA1Q0arjBtGmtwhTxCqHJU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366260; c=relaxed/simple; bh=LTlCiL6fmmVVuL8bKDmzSBOA/E25dmRY67EoUr9sHTM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tNcfgxW7q4dvz+SKlzfGHSvszwhchup3gqBQvTMt8IjKhfss2o8WIwGGC+/LIqUjYEkBi6WQzy8NxlFkNXCUxCD0Gw5jybomAm9UvJnLKkMz3bcXzxmm4d/sUq47UY98j6Rkl6xAEWAhIcQqq965VoJHRYiJOaJ1tkrKH/f1Bzg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=eflzKinz; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eflzKinz" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B44A2C79; Thu, 24 Jul 2025 16:10:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366208; bh=LTlCiL6fmmVVuL8bKDmzSBOA/E25dmRY67EoUr9sHTM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=eflzKinzcge3uNK6Yy71BC8PHhXpTWj2kEbEr9oWIREWTg6irW8FYgaC8MAM9wHa8 Fv2O5M4BMsi9kI2Hp9ykUzS5Fc91KKj0H34h6eBNQqJlh1diou9IcNRavRj7lmYEx4 Eb6gbVOBjx4V9dpNE6cHImYyeS7ISuURAZdD6+Ko= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:23 +0200 Subject: [PATCH v2 16/27] media: v4l2-subdev: Add subdev state accessor helpers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-16-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=10156; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=LTlCiL6fmmVVuL8bKDmzSBOA/E25dmRY67EoUr9sHTM=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7QjdzL4huWoPQWt9hdYUH28WTHaaCsqpjeU qna2kZzuD6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0AAKCRByNAaPFqFW PGOyD/0SmIsZq5YVWnS8yOYo+Js9CALdRiBwlS1FU+yS7Wah+l/YMy5QZ1c+pvSGbDEexB1IRLB Pke0D0N6TmmHG1aXW5ATpJoruhITQ18ZqTGAmguMPvSaJewWfi6nFQ1mt1fDPm4Vnk60cdW+xWI GCNb/jqBiJbjAnILhpEN2peDig8jSSM3WPtAeFCaD5y9Jr/oK4Tb7Kgs/itZdwvlZqwXHWJhURw QXtWM6PF42A+J3vM3zwcwvMQ4KSL07YfFQDyjP9ai8Xg7d2pLNKx5EvdeH1fqopHM6ZCDt4Cz81 U1RA9W8W/Lwq2d8ICw6yrn5GUfsh0ySeSlrejapD/96zV+1VW6jenjf04IN4V83N4CHWOJKGL8z RJpSloDiHE+8o9+rU3dwWA2AzoslILnPdWyoutRp00T3y32tHYy/yni9c/s+ASoT740dq7enqVl geoUDi+v8Qz7uRoh3MJHeW+FvfOtxO6tK+utrj9GpStNIuGcEVYUe7p4C+5KZHGODMPaNHq4ufc GIMHQCsdvvE7hm86iaV0nwI6PBD9d6E6vPjdW9Z169PSDrk88wNOEoiOoG9ku1G580l2KFnL08A C5wHL6XTGz+NEGuYQwJZC3EGtI0O+20m5RvdBiCFDEpTMvYD3YqnOng4gOY+lz3mCG24zbSfqS+ u4yXvcT7CyDuhtQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B The v4l2-subdev.c file offers three helpers to access the subdevice active state from a v4l2_subdev pointer: - v4l2_subdev_get_unlocked_active_state(sd) - v4l2_subdev_get_locked_active_state(sd) - v4l2_subdev_lock_and_get_active_state(sd) With the introduction of struct v4l2_subdev_context which contains a subdev_state as well, the actual "active" state is stored in three possible places: - A context associated with a v4l2_subdev_fh for context aware drivers operated by context aware userspace - The default subdevice context for context aware drivers operated by non-context aware userspace - The subdevice active state for non-context aware drivers Provide helpers similar in spirit the existing ones but accept as argument either a subdevice the context itself and retrieve the active state from the correct place. Helpers will be used in following patches for link validation. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 101 ++++++++++++++++++++++++++++++= +++ include/media/v4l2-subdev.h | 104 +++++++++++++++++++-----------= ---- 2 files changed, 159 insertions(+), 46 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-cor= e/v4l2-subdev.c index 438f51980e5ac0f092ba6b0a979a376133968ddf..7372f61127c871cec44a3d1900e= 2b8bef34632b9 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -20,6 +20,7 @@ #include #include =20 +#include #include #include #include @@ -189,6 +190,106 @@ static inline int check_pad(struct v4l2_subdev *sd, u= 32 pad) return 0; } =20 +/* subdev state accessor helpers */ + +/* + * Access the state from the subdevice. + * + * If the driver is context-aware use the state stored in the default cont= ext + * otherwise use the active state stored in the subdevice. + */ + +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_sd(struct v4l2_subdev *sd) +{ + if (!sd) + return NULL; + + if (sd->default_context) { + lockdep_assert_not_held(sd->default_context->state->lock); + + return sd->default_context->state; + } + + if (sd->active_state) + lockdep_assert_not_held(sd->active_state->lock); + return sd->active_state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_unlocked_active_state_from_sd); + +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_sd(struct v4l2_subdev *sd) +{ + if (!sd) + return NULL; + + if (sd->default_context) { + lockdep_assert_held(sd->default_context->state->lock); + + return sd->default_context->state; + } + + if (sd->active_state) + lockdep_assert_held(sd->active_state->lock); + return sd->active_state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_locked_active_state_from_sd); + +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_sd(struct v4l2_subdev *sd) +{ + if (!sd) + return NULL; + + if (sd->default_context) { + v4l2_subdev_lock_state(sd->default_context->state); + + return sd->default_context->state; + } + + if (sd->active_state) + v4l2_subdev_lock_state(sd->active_state); + return sd->active_state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_lock_and_get_active_state_from_sd); + +/* Access the subdevice state from a subdvice context. */ +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_ctx(struct v4l2_subdev_context = *ctx) +{ + if (!ctx) + return NULL; + + if (ctx->state) + lockdep_assert_not_held(ctx->state->lock); + return ctx->state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_unlocked_active_state_from_ctx); + +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_ctx(struct v4l2_subdev_context *c= tx) +{ + if (!ctx) + return NULL; + + if (ctx->state) + lockdep_assert_held(ctx->state->lock); + return ctx->state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_locked_active_state_from_ctx); + +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_ctx(struct v4l2_subdev_context = *ctx) +{ + if (!ctx) + return NULL; + + if (ctx->state) + v4l2_subdev_lock_state(ctx->state); + return ctx->state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_lock_and_get_active_state_from_ctx); + static int check_state(struct v4l2_subdev *sd, struct v4l2_subdev_state *s= tate, u32 which, u32 pad, u32 stream) { diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 1fa42a9f322be0be44fc9308744f4f4ae0cf1606..8087c0ae3bc0a0a95512b4b0ff5= 257522a104ca0 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -29,16 +29,17 @@ =20 #define V4L2_DEVICE_NOTIFY_EVENT _IOW('v', 2, struct v4l2_event) =20 -struct v4l2_device; +struct led_classdev; +struct media_device_context; +struct tuner_setup; struct v4l2_ctrl_handler; +struct v4l2_device; struct v4l2_event; struct v4l2_event_subscription; struct v4l2_fh; +struct v4l2_mbus_frame_desc; struct v4l2_subdev; struct v4l2_subdev_fh; -struct tuner_setup; -struct v4l2_mbus_frame_desc; -struct led_classdev; =20 /** * struct v4l2_decode_vbi_line - used to decode_vbi_line @@ -1968,64 +1969,75 @@ static inline void v4l2_subdev_unlock_states(struct= v4l2_subdev_state *state1, mutex_unlock(state2->lock); } =20 +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_sd(struct v4l2_subdev *sd); +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_sd(struct v4l2_subdev *sd); +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_sd(struct v4l2_subdev *sd); + +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_ctx(struct v4l2_subdev_context = *ctx); +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_ctx(struct v4l2_subdev_context *c= tx); +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_ctx(struct v4l2_subdev_context = *ctx); + /** - * v4l2_subdev_get_unlocked_active_state() - Checks that the active subdev= state - * is unlocked and returns it - * @sd: The subdevice + * v4l2_subdev_get_unlocked_active_state() - Checks that the subdev state = is + * unlocked and returns it + * @sdctx: The subdevice, or the subdevice context * - * Returns the active state for the subdevice, or NULL if the subdev does = not - * support active state. If the state is not NULL, calls - * lockdep_assert_not_held() to issue a warning if the state is locked. + * Returns the subdevice state, or NULL if it is not valid. If the state is + * not NULL, calls lockdep_assert_not_held() to issue a warning if the sta= te + * is locked. * - * This function is to be used e.g. when getting the active state for the = sole - * purpose of passing it forward, without accessing the state fields. + * This function is to be used e.g. when getting the state for the sole pu= rpose + * of passing it forward, without accessing the state fields. */ -static inline struct v4l2_subdev_state * -v4l2_subdev_get_unlocked_active_state(struct v4l2_subdev *sd) -{ - if (sd->active_state) - lockdep_assert_not_held(sd->active_state->lock); - return sd->active_state; -} +#define v4l2_subdev_get_unlocked_active_state(sdctx) \ + _Generic((sdctx), \ + struct v4l2_subdev *: \ + v4l2_subdev_get_unlocked_active_state_from_sd, \ + struct v4l2_subdev_context *: \ + v4l2_subdev_get_unlocked_active_state_from_ctx) \ + (sdctx) =20 /** - * v4l2_subdev_get_locked_active_state() - Checks that the active subdev s= tate - * is locked and returns it - * - * @sd: The subdevice + * v4l2_subdev_get_locked_active_state() - Checks that the subdev state is + * locked and returns it + * @sdctx: The subdevice, or the subdevice context * - * Returns the active state for the subdevice, or NULL if the subdev does = not - * support active state. If the state is not NULL, calls lockdep_assert_he= ld() - * to issue a warning if the state is not locked. + * Returns the subdevice state, or NULL is not valid. If the state is not = NULL, + * calls lockdep_assert_held() to issue a warning if the state is not lock= ed. * - * This function is to be used when the caller knows that the active state= is + * This function is to be used when the caller knows that the context stat= e is * already locked. */ -static inline struct v4l2_subdev_state * -v4l2_subdev_get_locked_active_state(struct v4l2_subdev *sd) -{ - if (sd->active_state) - lockdep_assert_held(sd->active_state->lock); - return sd->active_state; -} +#define v4l2_subdev_get_locked_active_state(sdctx) \ + _Generic((sdctx), \ + struct v4l2_subdev *: \ + v4l2_subdev_get_locked_active_state_from_sd, \ + struct v4l2_subdev_context *: \ + v4l2_subdev_get_locked_active_state_from_ctx) \ + (sdctx) =20 /** - * v4l2_subdev_lock_and_get_active_state() - Locks and returns the active = subdev - * state for the subdevice - * @sd: The subdevice + * v4l2_subdev_lock_and_get_active_state_from_ctx() - Locks and returns the + * subdevice state + * @sdctx: The subdevice, or the subdevice context * - * Returns the locked active state for the subdevice, or NULL if the subdev - * does not support active state. + * Returns the locked subdevice state, or NULL if it is not valid. * * The state must be unlocked with v4l2_subdev_unlock_state() after use. */ -static inline struct v4l2_subdev_state * -v4l2_subdev_lock_and_get_active_state(struct v4l2_subdev *sd) -{ - if (sd->active_state) - v4l2_subdev_lock_state(sd->active_state); - return sd->active_state; -} +#define v4l2_subdev_lock_and_get_active_state(sdctx) \ + _Generic((sdctx), \ + struct v4l2_subdev *: \ + v4l2_subdev_lock_and_get_active_state_from_sd, \ + struct v4l2_subdev_context *: \ + v4l2_subdev_get_locked_active_state_from_ctx) \ + (sdctx) =20 /** * v4l2_subdev_init - initializes the sub-device struct --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC67D2E9EBF; Thu, 24 Jul 2025 14:10:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366260; cv=none; b=Nfmsbx/pbQSqes6vvJCLOsuO8OEezb0MWZp2k7Iw6RbFf83Ttg/a8aXHrj1qQRrsWqhhd22HCeXOld+oBE2mH/8xjq2c4TOFlMMDeahSb9GKelMX2FdjxA9bnYlBiPdII8AfGIQtPVq/VbILx9pUer6aaMyb857YsRk0TKobTZ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366260; c=relaxed/simple; bh=Av+qDIxci6/rNYahjLOq/w3ykEkjt8C35QIlJVVOQlA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cqETz/ZkwIbCp+JZgXmk9XmJjvwhCGs/ppR4Bq8tfcFMmQm/UsPisNBgVXTy5ltwyI0CYiGoNAB4HmRefpnQA/BvCZAF2BRHh3Xk5lcQgyUsUjtwZbPQcq7H7ZDRQ0VRyvbWaKqvS46GpugGGoowE1M6ClfhSBey525nwuA2WH0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=ZW7lFNdA; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZW7lFNdA" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3280A1ACF; Thu, 24 Jul 2025 16:10:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366209; bh=Av+qDIxci6/rNYahjLOq/w3ykEkjt8C35QIlJVVOQlA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ZW7lFNdA65XXY3ybK3Q73IoMYzD/RWIj+d+DrncsIh+Dqeyw7iP+SuaKMH2EbE78N 9Z5KWcqk3Jm3yGkHfMiv8wGCc9pzzOhf+IvT4qWCrQdDvQcL8No6/mmTCC6ZHX/UlA 1+TEp4oAHeC4mGn52A4ocO9NCaZWkZbTrLNhXhgk= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:24 +0200 Subject: [PATCH v2 17/27] media: v4l2-subdev: Get state from context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-17-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1875; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Av+qDIxci6/rNYahjLOq/w3ykEkjt8C35QIlJVVOQlA=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7QE00gkL8XXmeox2W9uHAwgb+JjO4ss6i47 BwPdlK4VyqJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0AAKCRByNAaPFqFW PG3YEAC+9w342JKOCDxd2dM51MvvRVUhx/BFvFwL4CgpDaC7ZVi/oNGBmTGPo0X8SKnmhry1qvy u5C6hhd4a/ZMEmm+m6YmEMVMegZPpuZ0RloG1SLMBbq3ad33vSRnxn4FgUJS+VeuuAzA1Uy8aSc T19MV2LhqB5ruFUQbVX5U27vR5Bv1YJhmJPRaa+RXjR8+fUuCZzVKhc4CVk8O1m5yUhebZcKua+ e/67zxAvx93NTRKexaP3zIqKDE6N2h2lD4zwh2NGMu54zNcA6cL7oHmONzC/4qXqflG2u0eUrMT PR/jJbHUxudATsUcsmFOLQknJuaRz4od1CZQQzaoWRR95Zlso/gv+mv0TOlcT0VYrP/ANeUZGnH gs+isw5mjA6D/d8J6zW3xjpJddjZKlPNVL19+J6ujlkcLzSH9Nc46vVwlYchjZD0KANFdIXfHTu eR10UwhiRG03ouAYmL+Q52iDwVEhuqbpWc8FQE8/N1QhRRQTeUsXpn5RKUTQ2q587I3zoMN3Kiy Sa6O5+eANcaExxIqOmmmXHpgMGKEqEAMf5598moofsRxYfoNFofIcWlFtwpGl4tliS2BxhUsgPW J7h64tP9GYKN3vTVDq0mxmCNWft/+7Fn4a4+14zzGjC1r3XxcczqAn2K1xpSnJR98xYd/Xj1Zko mz5/wj+gouL6BVA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B The V4L2 subdev ioctl handler retrieves the correct subdev state inspecting the 'which' field of the ioctl argument. So far the subdev state either come from the file handle in case of V4L2_SUBDEV_FORMAT_TRY or from the active state stored in the subdev directly in case of V4L2_SUBDEV_FORMAT_ACTIVE. With the introduction of multi-contexts support, there will be a subdev state associated to each bound context. If we have a valid context, use the state from there in case of V4L2_SUBDEV_FORMAT_ACTIVE, and default to the subdev active state in case the context is not valid. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-cor= e/v4l2-subdev.c index 7372f61127c871cec44a3d1900e2b8bef34632b9..66c539d880127844893620d325a= 2b05ac4aa9e96 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -703,9 +703,20 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct = v4l2_subdev_fh *subdev_fh, break; } =20 - return which =3D=3D V4L2_SUBDEV_FORMAT_TRY ? - subdev_fh->state : - v4l2_subdev_get_unlocked_active_state(sd); + /* + * If which is FORMAT_TRY return the state stored in the file handle. + * If a context has been allocated because the subdev has been bound + * then return the state stored in the context. Otherwise default to the + * subdevice active state. + */ + + if (which =3D=3D V4L2_SUBDEV_FORMAT_TRY) + return subdev_fh->state; + + if (subdev_fh->context) + return v4l2_subdev_get_unlocked_active_state(subdev_fh->context); + + return v4l2_subdev_get_unlocked_active_state(sd); } =20 static int subdev_do_bind_context(struct v4l2_subdev *sd, --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 138812EA494; Thu, 24 Jul 2025 14:11:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366264; cv=none; b=Vs91ptGVOCcevfPD7z0nyd5++Kv5ubWd/1EJOzJc0LBa6rhO8I7LwAsFsCUKKSi+gFbN7tr7mIB6/+dMnR3+0tyL9PPoehIOqaxIkAbuam6WqXGQqYUETVRjpn8uRfhjt4tzRzpBANJlUYMkYFLSaUQZAroV/gETF4yLAeAXW+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366264; c=relaxed/simple; bh=cRd3wiIxy2b+WbPMrfjq8RLbsaBxYWFg+yvih1JD75Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Z1KKn8Xs+udYBvK4Yco/5sbw8jChBgj8MwuKvHSJzPUWSfX64PSDw/xrsJMyaX/hQFbkiiVi6U9SE9MihaTdnd5pO7JPwAFm4z9Mqxyk3kfFxZrnvw+TA2VWgCbjE5ktx9YFxJKAzq/ydTk8jBDN52/3q36fT+5+HW0LFIIdnV8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=WHNUdf7V; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WHNUdf7V" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9991E11E6; Thu, 24 Jul 2025 16:10:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366210; bh=cRd3wiIxy2b+WbPMrfjq8RLbsaBxYWFg+yvih1JD75Q=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=WHNUdf7V6TyInk8Hbrn5slQ5R27N2m1RKmK7K6eC6xcHFCy4sOym1JRmGoNp9t12F q4UO8yA9HlI83e2Sp7CSkdOtGIfktUc5mznHQDOWkrOy5c2ijrH2XITYPKwa4LJfqz 7K2dEAzQDtz8OKDHdrA21+8gO9f+eAnqHkmvEax4= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:25 +0200 Subject: [PATCH v2 18/27] media: media-entity: Support context in pipeline_start Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-18-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=17561; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=cRd3wiIxy2b+WbPMrfjq8RLbsaBxYWFg+yvih1JD75Q=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7QS2BR9nJ+xLBDuoGX5iktpW8UcGgeZ2bf6 1iB6v1XDaiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0AAKCRByNAaPFqFW PB8GD/4+VVfFKJX+xLOD79DsOM6cfe2hQKUSqi3hO4qHCh8nU+c0Y1soBnriouUUxQq9uBis1zr XE+LSvdqZJnk/n6noJ3ExuqjHtt6gKTXk2/9PNjqk75EvDKrXJsZ16zv4tYnunsfSj180+kdUaq VoB8que+xji3zl+WzEEyXgiCnsyaFJZymr1u1H/ZvfeXMeFIpHrW8MEgNvcn5lW/5LpKvP5f1H0 xiNEty+wS23y2vrRCD8w3N4/arnQEGUBYP/3/MguFBxRdBm8zCSv6ao0ZuZNCkPHNfv2Dfnt7TW js0U1SpxvGxdOiqVQSPzSFXDxn2HN8QFG9S7bfxTGNXPTXegkh8/HoLyDsiSZ0EAbDz89re6lJf AX+cSxAcbbDqoLTs4a91DoY0l4FxHs3f5LOQa3QkUUUHdWZQkxImcYWaPHayzfo/FKLfVRLSEXX UtWZni67U9sRepn4BDXW6oqcJOVe8Mjy5V0bX59hISEk4r7tx8kIVSq5X2udyUZTVMTIwE2A6kn 58dzZ/hTPsH42mODlbNkadWeEtjrUbpaKq2yA5dtVr8a2+L4k11gXDlzkSW5VpiQZoGGWGKpKSa wpxYFHQnMla7S8QOHpPz8/IoUwDP1B2vsC+To61AS73AybcodeJE16HAS3bfjdj5W54DLOJocbY PPHFzddX4LczEZA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Add helpers to propagate an execution context when starting the media pipeline. Starting a pipeline with an execution context implies that when creating the streaming pipeline all the entities part of the pipeline shall have a valid context associated with the media context in use. When creating the media pipeline increase the device context use count to guarantee that during the streaming phase drivers will always operate with valid contexts. Add 'overrides' for all the video_device_pipeline_start() and media_pipeline_start() variants to support contexts. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 150 ++++++++++++++++++++++++++++++++-= ---- drivers/media/v4l2-core/v4l2-dev.c | 44 +++++++++++ include/media/media-entity.h | 56 +++++++++++++- include/media/v4l2-dev.h | 57 ++++++++++++++ 4 files changed, 287 insertions(+), 20 deletions(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 7bc276c725f974539ea06e3882d004b81be1de68..f421d6d74630bb96400d39d805c= 5db5d3d1ff913 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -768,8 +768,50 @@ static int media_pipeline_populate(struct media_pipeli= ne *pipe, return ret; } =20 -__must_check int __media_pipeline_start(struct media_pad *origin, - struct media_pipeline *pipe) +static int +media_pipeline_validate_context(struct media_device_context *mdev_context, + struct media_entity *entity, + struct media_pipeline_pad *ppad) +{ + struct media_entity_context *context; + + if (!mdev_context) + return 0; + + /* + * It's not mandatory for all entities in the pipeline to support + * contexts. + */ + if (!entity->ops || !entity->ops->alloc_context || + !entity->ops->destroy_context) + return 0; + + /* + * But if they do they should be bound to the same media device context + * as all other entities. + * + * media_device_get_entity_context increases the ref-counting of the + * context. Store a reference in the ppad for later decreasing the + * reference count when the pipeline is stopped. + * + * Fail validation if no context is associated with this media context + * and be loud about that as userspace should be informed it has to + * bind all entities of the pipeline in the same context. + */ + context =3D media_device_get_entity_context(mdev_context, entity); + if (WARN_ON(IS_ERR(context))) + return -EPIPE; + + media_entity_context_get(context); + ppad->context =3D context; + + return 0; +} + +__must_check int +__media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *mdev_context) { struct media_device *mdev =3D origin->graph_obj.mdev; struct media_pipeline_pad *err_ppad; @@ -829,7 +871,15 @@ __must_check int __media_pipeline_start(struct media_p= ad *origin, } =20 /* - * 2. Validate all active links whose sink is the current pad. + * 2. If we have a media context, ensure the entity has a device + * context associated with it. + */ + ret =3D media_pipeline_validate_context(mdev_context, entity, ppad); + if (ret) + goto error; + + /* + * 3. Validate all active links whose sink is the current pad. * Validation of the source pads is performed in the context of * the connected sink pad to avoid duplicating checks. */ @@ -875,7 +925,7 @@ __must_check int __media_pipeline_start(struct media_pa= d *origin, } =20 /* - * 3. If the pad has the MEDIA_PAD_FL_MUST_CONNECT flag set, + * 4. If the pad has the MEDIA_PAD_FL_MUST_CONNECT flag set, * ensure that it has either no link or an enabled link. */ if ((pad->flags & MEDIA_PAD_FL_MUST_CONNECT) && @@ -905,6 +955,9 @@ __must_check int __media_pipeline_start(struct media_pa= d *origin, if (err_ppad =3D=3D ppad) break; =20 + if (err_ppad->context) + media_entity_context_put(err_ppad->context); + err_ppad->pad->pipe =3D NULL; } =20 @@ -912,19 +965,35 @@ __must_check int __media_pipeline_start(struct media_= pad *origin, =20 return ret; } +EXPORT_SYMBOL_GPL(__media_pipeline_start_context); + +__must_check int __media_pipeline_start(struct media_pad *origin, + struct media_pipeline *pipe) +{ + return __media_pipeline_start_context(origin, pipe, NULL); +} EXPORT_SYMBOL_GPL(__media_pipeline_start); =20 -__must_check int media_pipeline_start(struct media_pad *origin, - struct media_pipeline *pipe) +__must_check int +media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *context) { struct media_device *mdev =3D origin->graph_obj.mdev; int ret; =20 mutex_lock(&mdev->graph_mutex); - ret =3D __media_pipeline_start(origin, pipe); + ret =3D __media_pipeline_start_context(origin, pipe, context); mutex_unlock(&mdev->graph_mutex); return ret; } +EXPORT_SYMBOL_GPL(media_pipeline_start_context); + +__must_check int media_pipeline_start(struct media_pad *origin, + struct media_pipeline *pipe) +{ + return media_pipeline_start_context(origin, pipe, NULL); +} EXPORT_SYMBOL_GPL(media_pipeline_start); =20 void __media_pipeline_stop(struct media_pad *pad) @@ -942,8 +1011,11 @@ void __media_pipeline_stop(struct media_pad *pad) if (--pipe->start_count) return; =20 - list_for_each_entry(ppad, &pipe->pads, list) + list_for_each_entry(ppad, &pipe->pads, list) { + if (ppad->context) + media_entity_context_put(ppad->context); ppad->pad->pipe =3D NULL; + } =20 media_pipeline_cleanup(pipe); =20 @@ -962,14 +1034,13 @@ void media_pipeline_stop(struct media_pad *pad) } EXPORT_SYMBOL_GPL(media_pipeline_stop); =20 -__must_check int media_pipeline_alloc_start(struct media_pad *pad) +static struct media_pipeline * +media_pipeline_alloc(struct media_pad *pad) { struct media_device *mdev =3D pad->graph_obj.mdev; - struct media_pipeline *new_pipe =3D NULL; struct media_pipeline *pipe; - int ret; =20 - mutex_lock(&mdev->graph_mutex); + lockdep_assert_held(&mdev->graph_mutex); =20 /* * Is the pad already part of a pipeline? If not, we need to allocate @@ -977,19 +1048,33 @@ __must_check int media_pipeline_alloc_start(struct m= edia_pad *pad) */ pipe =3D media_pad_pipeline(pad); if (!pipe) { - new_pipe =3D kzalloc(sizeof(*new_pipe), GFP_KERNEL); - if (!new_pipe) { - ret =3D -ENOMEM; - goto out; - } + pipe =3D kzalloc(sizeof(*pipe), GFP_KERNEL); + if (!pipe) + return ERR_PTR(-ENOMEM); =20 - pipe =3D new_pipe; pipe->allocated =3D true; } =20 + return pipe; +} + +__must_check int media_pipeline_alloc_start(struct media_pad *pad) +{ + struct media_device *mdev =3D pad->graph_obj.mdev; + struct media_pipeline *pipe; + int ret; + + mutex_lock(&mdev->graph_mutex); + + pipe =3D media_pipeline_alloc(pad); + if (IS_ERR(pipe)) { + ret =3D PTR_ERR(pipe); + goto out; + } + ret =3D __media_pipeline_start(pad, pipe); if (ret) - kfree(new_pipe); + kfree(pipe); =20 out: mutex_unlock(&mdev->graph_mutex); @@ -998,6 +1083,33 @@ __must_check int media_pipeline_alloc_start(struct me= dia_pad *pad) } EXPORT_SYMBOL_GPL(media_pipeline_alloc_start); =20 +__must_check +int media_pipeline_alloc_start_context(struct media_pad *pad, + struct media_device_context *mdev_context) +{ + struct media_device *mdev =3D pad->graph_obj.mdev; + struct media_pipeline *pipe; + int ret; + + mutex_lock(&mdev->graph_mutex); + + pipe =3D media_pipeline_alloc(pad); + if (IS_ERR(pipe)) { + ret =3D PTR_ERR(pipe); + goto out; + } + + ret =3D __media_pipeline_start_context(pad, pipe, mdev_context); + if (ret) + kfree(pipe); + +out: + mutex_unlock(&mdev->graph_mutex); + + return ret; +} +EXPORT_SYMBOL_GPL(media_pipeline_alloc_start_context); + struct media_pad * __media_pipeline_pad_iter_next(struct media_pipeline *pipe, struct media_pipeline_pad_iter *iter, diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v= 4l2-dev.c index fafd4209e2cab320a6e164d33e9a0f73784d22db..eacaa38cffe8847b66acdeebf23= 30a072b3be85f 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1221,6 +1221,36 @@ __must_check int __video_device_pipeline_start(struc= t video_device *vdev, } EXPORT_SYMBOL_GPL(__video_device_pipeline_start); =20 +__must_check int +video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe) +{ + struct video_device *vdev =3D context->vdev; + struct media_entity *entity =3D &vdev->entity; + + if (entity->num_pads !=3D 1) + return -ENODEV; + + return media_pipeline_start_context(&entity->pads[0], pipe, + context->base.mdev_context); +} +EXPORT_SYMBOL(video_device_context_pipeline_start); + +__must_check int +__video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe) +{ + struct video_device *vdev =3D context->vdev; + struct media_entity *entity =3D &vdev->entity; + + if (entity->num_pads !=3D 1) + return -ENODEV; + + return __media_pipeline_start_context(&entity->pads[0], pipe, + context->base.mdev_context); +} +EXPORT_SYMBOL(__video_device_context_pipeline_start); + void video_device_pipeline_stop(struct video_device *vdev) { struct media_entity *entity =3D &vdev->entity; @@ -1254,6 +1284,20 @@ __must_check int video_device_pipeline_alloc_start(s= truct video_device *vdev) } EXPORT_SYMBOL_GPL(video_device_pipeline_alloc_start); =20 +__must_check int +video_device_context_pipeline_alloc_start(struct video_device_context *con= text) +{ + struct video_device *vdev =3D context->vdev; + struct media_entity *entity =3D &vdev->entity; + + if (entity->num_pads !=3D 1) + return -ENODEV; + + return media_pipeline_alloc_start_context(&entity->pads[0], + context->base.mdev_context); +} +EXPORT_SYMBOL_GPL(video_device_context_pipeline_alloc_start); + struct media_pipeline *video_device_pipeline(struct video_device *vdev) { struct media_entity *entity =3D &vdev->entity; diff --git a/include/media/media-entity.h b/include/media/media-entity.h index 32298fe8a18c6ee3c1dbcff9ef869548904417a7..b60c311ab390beb6931fe8f2bbe= 8939e11cda452 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -119,16 +119,20 @@ struct media_pipeline { * @list: Entry in the media_pad pads list * @pipe: The media_pipeline that the pad is part of * @pad: The media pad + * @context: Reference to a video device or subdevice context * * This structure associate a pad with a media pipeline. Instances of * media_pipeline_pad are created by media_pipeline_start() when it builds= the * pipeline, and stored in the &media_pad.pads list. media_pipeline_stop() - * removes the entries from the list and deletes them. + * removes the entries from the list and deletes them. The context field is + * populated only if a valid context has been associated with the pad. */ +struct media_entity_context; struct media_pipeline_pad { struct list_head list; struct media_pipeline *pipe; struct media_pad *pad; + struct media_entity_context *context; }; =20 /** @@ -1212,6 +1216,39 @@ __must_check int media_pipeline_start(struct media_p= ad *origin, __must_check int __media_pipeline_start(struct media_pad *origin, struct media_pipeline *pipe); =20 +/** + * media_pipeline_start_context - Mark a pipeline as streaming + * @origin: Starting pad + * @pipe: Media pipeline to be assigned to all pads in the pipeline. + * @context: The media device context the pipeline belongs to + * + * Mark all pads connected to a given pad through enabled links, either + * directly or indirectly, as streaming. The given pipeline object is assi= gned + * to every pad in the pipeline and stored in the media_pad pipe field. + * + * Calls to this function can be nested, in which case the same number of + * media_pipeline_stop() calls will be required to stop streaming. The + * pipeline pointer must be identical for all nested calls to + * media_pipeline_start(). + */ +__must_check int +media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *context); + +/** + * __media_pipeline_start_context - Mark a pipeline as streaming + * @origin: Starting pad + * @pipe: Media pipeline to be assigned to all pads in the pipeline. + * @context: The media device context the pipeline belongs to + * + * ..note:: This is the non-locking version of media_pipeline_start_contex= t() + */ +__must_check int +__media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *context); + /** * media_pipeline_stop - Mark a pipeline as not streaming * @pad: Starting pad @@ -1318,6 +1355,23 @@ __media_pipeline_entity_iter_next(struct media_pipel= ine *pipe, */ __must_check int media_pipeline_alloc_start(struct media_pad *pad); =20 +/** + * media_pipeline_alloc_start_context - Mark a pipeline as streaming + * @pad: Starting pad + * @context: The media device context the pipeline belongs to + * + * media_pipeline_alloc_start_context() is similar to + * media_pipeline_start_context() but instead of working on a given pipeli= ne the + * function will use an existing pipeline if the pad is already part of a + * pipeline, or allocate a new pipeline. + * + * Calls to media_pipeline_alloc_start_context() must be matched with + * media_pipeline_stop(). + */ +__must_check int +media_pipeline_alloc_start_context(struct media_pad *pad, + struct media_device_context *context); + /** * media_devnode_create() - creates and initializes a device node interface * diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index bab4b13b109362bec84d8d16440b6ea895206b60..93095df692e3628d4be003d2584= 3fa744d6b20a4 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -602,6 +602,48 @@ __must_check int video_device_pipeline_start(struct vi= deo_device *vdev, __must_check int __video_device_pipeline_start(struct video_device *vdev, struct media_pipeline *pipe); =20 +/** + * video_device_context_pipeline_start - Mark a pipeline as streaming star= ting + * from a video device context + * @context: The video device context that starts the streaming + * @pipe: Media pipeline to be assigned to all entities in the pipeline. + * + * ..note:: This is the multi-context version of video_device_pipeline_sta= rt() + * Documentation of this function only describes specific aspects= of + * this version. Refer to the video_device_pipeline_start() + * documentation for a complete reference. + * + * Validate that all entities connected to a video device through enabled = links + * by ensuring that a context associated with the same media device context + * exists for them. Increase the reference counting of each of the context= s part + * of the pipeline to guarantee their lifetime is maintained as long as the + * pipeline is streaming. + * + * Context validation and refcounting of all entities that are part of a + * streaming pipeline ensures that device drivers can safely access device + * contexts in a media device context during streaming. References to cont= exts + * retried by a call to media_device_get_entity_context(), are guaranteed = to be + * valid as long as the pipeline is streaming. Likewise, the media device + * context that contains the device contexts is guaranteed to be valid as = long + * as the pipeline is streaming. + */ +__must_check int +video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe); + +/** + * __video_device_context_pipeline_start - Mark a pipeline as streaming st= arting + * from a video device context + * @context: The video device context that starts the streaming + * @pipe: Media pipeline to be assigned to all entities in the pipeline. + * + * ..note:: This is the non-locking version of + * __video_device_context_pipeline_start() + */ +__must_check int +__video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe); + /** * video_device_pipeline_stop - Mark a pipeline as not streaming * @vdev: Starting video device @@ -646,6 +688,21 @@ void __video_device_pipeline_stop(struct video_device = *vdev); */ __must_check int video_device_pipeline_alloc_start(struct video_device *vd= ev); =20 +/** + * video_device_context_pipeline_alloc_start - Mark a pipeline as streaming + * @context: The video device context that starts the streaming + * + * video_device_context_pipeline_alloc_start() is similar to + * video_device_context_pipeline_start() but instead of working on a given + * pipeline the function will use an existing pipeline if the video device= is + * already part of a pipeline, or allocate a new pipeline. + * + * Calls to video_device_context_pipeline_alloc_start() must be matched wi= th + * video_device_pipeline_stop(). + */ +__must_check int +video_device_context_pipeline_alloc_start(struct video_device_context *con= text); + /** * video_device_pipeline - Get the media pipeline a video device is part of * @vdev: The video device --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2E3E2EA493; Thu, 24 Jul 2025 14:11:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366263; cv=none; b=mzJl3ICt+ZSkyxXsGpldxTkmpf/sWJbELGJ8ktaTzg92KDwcRKToHU84O4b3IL6/1TFR3hylJJH6QYGmYViLwToo0CZEtG13uRAHIZS9ByLk4TqA4fiHeUj/5ShvWvP71Mkb2/ccfONlSOPYzEg9Cy21PlHCamJWSf7PXJ9L3VQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366263; c=relaxed/simple; bh=Vmfy9P6HgOvkJ1o2D9ktCyX+PB0ofnZcx1ytCaFVfCs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QXmdbh8EnfVHbH2Jt1mCajNHZExnxK+s6V8Q+2cMqUvXArxHLxhswa9ZfIjCRZ6wA07HsQccU8RwgRnYZByTciB+AxtArpkxuPg68fMqVFt6idtOQrRHw8r9L65vNb2EVK7oHG3duNBsK21kfkoShs1/KIkvYhcFIk6l+SzOvHw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=e5FWXca7; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="e5FWXca7" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CBD5F1A37; Thu, 24 Jul 2025 16:10:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366211; bh=Vmfy9P6HgOvkJ1o2D9ktCyX+PB0ofnZcx1ytCaFVfCs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=e5FWXca7LlbJwCYMKZqouJ8vJW5NxXfPpEgJZqAPDd0bOIoz4Ic3Fn0iVaBckzNQT R8w0DSh6jiiDF8uz82213695+XnlVBOWmojdIYYLYFMhxqIoyZXITz1yGWtCjivgaM XXGqqpkPngpRtajtlfud5inEW5oSSc0d8FTVaIzs= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:26 +0200 Subject: [PATCH v2 19/27] media: mc-entity: Add link_validate_context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-19-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3666; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Vmfy9P6HgOvkJ1o2D9ktCyX+PB0ofnZcx1ytCaFVfCs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7QICvUpwUUHAHWbWnmWUel95FZ9qcwgzNxS 0oqDL1ZzTiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0AAKCRByNAaPFqFW PHzjD/4xp+EwWa2WUaG5sW/TrEB0qMmWUkUDgbPEZc1cnQIHrW7C1xs8etc89pD+u/M+Fp0Lt18 k1WIMI5r4IBbeFG0XAG3I2AZlGevhedWPrS4/CE6v+b8N0mIslNcTH6jKlIw+i+Zx1y74JDADNw /9dsneu6kjWuYpbI7KbzYEe64DJ50I6t93u5F8zVPGxlPL1M8uPMikcnHGclhIjOI7F3Tl6AXAR CldHHZALbqUsiRPbggI5q0QZxljQLeNl362AZzC1SL6p2WX5NQ/TTxfrx4YE/hlk4WyrwjbbG7g WYCWMMLg6A4i/u1XR4/wcdIJNl9yXrQ4k16UrzJGhq1t2wF+5+TtMsYjo1fDgoogqsGHOUPU1I5 NR7PsDoHb8cd4WFjHlCZNoG+Ph0gKXlqTRMkbFgJbbWrMYuxNFtx0qYCtVfryKBwJcwpq6AbZp/ SVGCJlfnyq1WqK0g3yQcbC+NwrjFKolBaayYC1R0RXTOlYi3WeWDUy4G8NVOSSg/05pSY6t9OW4 1qC5IxvCIajRpSYQk8TuuXxBPmPQB8g2OQ2Yk1LT4WMh9Uk7MHPaMn1W7E0mHb2ohSsniAZv+Hd CT8ei06YXGtiGhnshIcDM+bkDwcEdlpQAw+lNm2SSkcpKrXBCCMd0qiFJfobjspBAqQp2Fn4iE3 sLKOnx+KopjACjg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Add an 'override' of the link_validate() media entity operation that accepts a media_device_context as second argument to allow entities to validate the link state in the provided media_device context. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 23 +++++++++++++++++++---- include/media/media-entity.h | 6 ++++++ 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index f421d6d74630bb96400d39d805c5db5d3d1ff913..675ceaede0d10a2420b8ea6a89e= 5963dcfde5ffe 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -884,6 +884,8 @@ __media_pipeline_start_context(struct media_pad *origin, * the connected sink pad to avoid duplicating checks. */ for_each_media_entity_data_link(entity, link) { + const struct media_entity_operations *ops; + /* Skip links unrelated to the current pad. */ if (link->sink !=3D pad && link->source !=3D pad) continue; @@ -902,13 +904,23 @@ __media_pipeline_start_context(struct media_pad *orig= in, if (link->sink !=3D pad) continue; =20 - if (!entity->ops || !entity->ops->link_validate) + ops =3D entity->ops; + if (!ops || (!ops->link_validate && + !ops->link_validate_context)) continue; =20 - ret =3D entity->ops->link_validate(link); + if (mdev_context && ops->link_validate_context) + ret =3D ops->link_validate_context(link, + mdev_context); + else + ret =3D entity->ops->link_validate(link); + if (ret) { dev_dbg(mdev->dev, - "Link '%s':%u -> '%s':%u failed validation: %d\n", + "%sink '%s':%u -> '%s':%u failed validation: %d\n", + (mdev_context && + ops->link_validate_context) ? + "Context l" : "L", link->source->entity->name, link->source->index, link->sink->entity->name, @@ -917,7 +929,10 @@ __media_pipeline_start_context(struct media_pad *origi= n, } =20 dev_dbg(mdev->dev, - "Link '%s':%u -> '%s':%u is valid\n", + "%sink '%s':%u -> '%s':%u is valid\n", + (mdev_context && + ops->link_validate_context) ? + "Context l" : "L", link->source->entity->name, link->source->index, link->sink->entity->name, diff --git a/include/media/media-entity.h b/include/media/media-entity.h index b60c311ab390beb6931fe8f2bbe8939e11cda452..b053a0baee4031a464edf506d3d= 131bacb810f81 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -295,6 +295,10 @@ struct media_entity_context { * @link_validate: Return whether a link is valid from the entity point of * view. The media_pipeline_start() function * validates all links by calling this operation. Optional. + * @link_validate_context: Return whether a link is valid from the entity + * context point of view. The + * media_pipeline_start_context() function validates all + * links calling this operation. Optional. * @has_pad_interdep: Return whether two pads of the entity are * interdependent. If two pads are interdependent they are * part of the same pipeline and enabling one of the pads @@ -327,6 +331,8 @@ struct media_entity_operations { const struct media_pad *local, const struct media_pad *remote, u32 flags); int (*link_validate)(struct media_link *link); + int (*link_validate_context)(struct media_link *link, + struct media_device_context *mdev_context); bool (*has_pad_interdep)(struct media_entity *entity, unsigned int pad0, unsigned int pad1); int (*alloc_context)(struct media_entity *entity, --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA67C2EAB96; Thu, 24 Jul 2025 14:11:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366267; cv=none; b=e3EDhUaDQ/WqEuOyKiZwQRzDyAVsjyDSKvsI9dvGFlov7q2tphA/myCpj2VE/i6OMzFQP283vqOfummOM8R9IIJTBIioEsBAndVQ4s5yAFAqY4Hi5O8gzCp2QtUYbGmIXWw3bU8JfxwFbu5QdWf1HR9yLxvFWqyuqSVtpjpb6x8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366267; c=relaxed/simple; bh=Rakpl27ZFfuZPNzLUJwkywWxLHQibWoreQq3wqBUYHs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fZIavhHAbRO3oZLQMrJHxrD5mWt4xUd56LM/QbPR8aJFdLFug4NNqDh5+tAx83xvyirh6MrQX94bTCWaq5lZ5jt5ulxeCeJP17jRa4HxL38nxWb9oxEjucEgbcijhVmky+G19h4G6mMgucNTA8TkCQuIvxlUXNaQdT3QmBigeRs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=rkxFX7sA; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rkxFX7sA" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 033AF1E5F; Thu, 24 Jul 2025 16:10:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366212; bh=Rakpl27ZFfuZPNzLUJwkywWxLHQibWoreQq3wqBUYHs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=rkxFX7sAQCEKaPxSPE2/4a3i2tkB56Hv+1ujwS7RD9Hz2U1UnNsTe3uPaUWJH7GW3 HLSoIKsGP9BTmJMgra5HH9J+JPKG67vKyeLc9yKfvl8biarE5pljl5KzxoMafsKv4V lyYdm/h4N8Jn+soiQlInnqXT3NrO4mqIbySApqQ0= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:27 +0200 Subject: [PATCH v2 20/27] media: v4l2-subdev: Validate media links with context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-20-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=13600; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Rakpl27ZFfuZPNzLUJwkywWxLHQibWoreQq3wqBUYHs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7Q/wJzxHEeiOyXCg/K0SW47Nh8MR3vvMnBO cFG4qPLsvKJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0AAKCRByNAaPFqFW PPuHD/4vVZqoTvFMhF/X/zg35PTtWILmEfRXKwssYAm/KpZIwVz1sLTkBaU9dzzT1jkTlyhz5no QshxJi3xBYeh1ts2t6caLdg0HtEQDkCFCxiubXzZMAIlaCQz8lWdLDR+MT4YOBqc9axvci+beKw D9AErFlXR9yeVQ12NlhZdNxokMV8K6hZnfuVaDLfNwKdfHZtb4m6DWbIkPVkZdhXEj42Q3/6MhH eUFRq9vpy/5CZ2ttyfekkfaQIKioyDmNND82ORl/14sWqHPIjCsLtLo/E78jaPOiW3Qzs62dEzZ V00+PnMfBa1WT1vE3QCTr4/D+aHqCydDzLbv9CUdeSbZHugpTsYFfyxZGgKVJt4TsGVxeUz2rWl Q1765PIuZ9HRAB2+iDUec6vLC8jjfgseNvsLP6vcmtRrHgVDKkPGIZe5dBh4SOM/h5Vy3rtzQqR 7hIzN5p9aboB6k06VHXck1SdYaRclSL6eBTZ8TQ0nZgfrKtrPGhxIJcJRouqua2b+DtBTGfc5X2 OEQNVa1P4qGwMjW/1m+G+/qrr69QGNPdnBGyW//MyTLRKN5weSbDGrZ7XDiDIsr51cswU+7qoFi pnc4qJWgiFNALqvzW1mx+31sU3eEwjTOrNAQ6Ey81QvMtenPikAwXDOYmACc9HfTJsTCRbbKWo4 e419l8FUQCiX2Yw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B The v4l2-subdev.c file provides an helper for subdevs to implement the media entity .link_validate() operation which can be used by subdevice drivers. Provide an 'overload' of the v4l2_subdev_link_validate() function that supports contexts to be used by context-aware subdev drivers to implement .link_validate_context(). When v4l2_subdev_link_validate() is used to validate a subdev-to-subdev link, propagate the media_device_context to all the call chain and introduce (and use) helpers to get the subdevice state from either the context or the subdev in case no context is available. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 148 +++++++++++++++++++++++++++---= ---- include/media/v4l2-subdev.h | 34 +++++++- 2 files changed, 151 insertions(+), 31 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-cor= e/v4l2-subdev.c index 66c539d880127844893620d325a2b05ac4aa9e96..59f42a3a9755a77ea74442605db= bc2af1b67a0ea 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1460,6 +1460,71 @@ int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_e= ntity *entity, } EXPORT_SYMBOL_GPL(v4l2_subdev_get_fwnode_pad_1_to_1); =20 +/* + * Retrieve the subdevice state from the media device context or, + * if there is no context, use the active state from the subdevice. + * + * These three functions wraps the usual subdev state helpers: + * + * - get_unlocked + * - get_locked + * - lock_and_get + */ + +static struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_state_from_mdev_ctx(struct v4l2_subdev *sd, + struct media_device_context *mdev_ctx) +{ + struct v4l2_subdev_context *ctx =3D NULL; + + if (mdev_ctx) { + ctx =3D v4l2_subdev_context_get(mdev_ctx, sd); + if (WARN_ON(!ctx)) + return NULL; + } + + if (ctx) + return v4l2_subdev_get_unlocked_active_state(ctx); + + return v4l2_subdev_get_unlocked_active_state(sd); +} + +static struct v4l2_subdev_state * +v4l2_subdev_get_locked_state_from_mdev_ctx(struct v4l2_subdev *sd, + struct media_device_context *mdev_ctx) +{ + struct v4l2_subdev_context *ctx =3D NULL; + + if (mdev_ctx) { + ctx =3D v4l2_subdev_context_get(mdev_ctx, sd); + if (WARN_ON(!ctx)) + return NULL; + } + + if (ctx) + return v4l2_subdev_get_locked_active_state(ctx); + + return v4l2_subdev_get_locked_active_state(sd); +} + +static struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_state_from_mdev_ctx(struct v4l2_subdev *sd, + struct media_device_context *mdev_ctx) +{ + struct v4l2_subdev_context *ctx =3D NULL; + + if (mdev_ctx) { + ctx =3D v4l2_subdev_context_get(mdev_ctx, sd); + if (WARN_ON(!ctx)) + return NULL; + } + + if (ctx) + return v4l2_subdev_lock_and_get_active_state(ctx); + + return v4l2_subdev_lock_and_get_active_state(sd); +} + int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd, struct media_link *link, struct v4l2_subdev_format *source_fmt, @@ -1518,8 +1583,9 @@ int v4l2_subdev_link_validate_default(struct v4l2_sub= dev *sd, EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default); =20 static int -v4l2_subdev_link_validate_get_format(struct media_pad *pad, u32 stream, - struct v4l2_subdev_format *fmt, +v4l2_subdev_link_validate_get_format(struct media_pad *pad, + struct media_device_context *mdev_context, + u32 stream, struct v4l2_subdev_format *fmt, bool states_locked) { struct v4l2_subdev_state *state; @@ -1533,9 +1599,11 @@ v4l2_subdev_link_validate_get_format(struct media_pa= d *pad, u32 stream, fmt->stream =3D stream; =20 if (states_locked) - state =3D v4l2_subdev_get_locked_active_state(sd); + state =3D v4l2_subdev_get_locked_state_from_mdev_ctx(sd, + mdev_context); else - state =3D v4l2_subdev_lock_and_get_active_state(sd); + state =3D v4l2_subdev_lock_and_get_state_from_mdev_ctx(sd, + mdev_context); =20 ret =3D v4l2_subdev_call(sd, pad, get_fmt, state, fmt); =20 @@ -1548,6 +1616,7 @@ v4l2_subdev_link_validate_get_format(struct media_pad= *pad, u32 stream, #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) =20 static void __v4l2_link_validate_get_streams(struct media_pad *pad, + struct media_device_context *mdev_context, u64 *streams_mask, bool states_locked) { @@ -1560,10 +1629,11 @@ static void __v4l2_link_validate_get_streams(struct= media_pad *pad, *streams_mask =3D 0; =20 if (states_locked) - state =3D v4l2_subdev_get_locked_active_state(subdev); + state =3D v4l2_subdev_get_locked_state_from_mdev_ctx(subdev, + mdev_context); else - state =3D v4l2_subdev_lock_and_get_active_state(subdev); - + state =3D v4l2_subdev_lock_and_get_state_from_mdev_ctx(subdev, + mdev_context); if (WARN_ON(!state)) return; =20 @@ -1592,6 +1662,7 @@ static void __v4l2_link_validate_get_streams(struct m= edia_pad *pad, #endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ =20 static void v4l2_link_validate_get_streams(struct media_pad *pad, + struct media_device_context *mdev_context, u64 *streams_mask, bool states_locked) { @@ -1604,14 +1675,17 @@ static void v4l2_link_validate_get_streams(struct m= edia_pad *pad, } =20 #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) - __v4l2_link_validate_get_streams(pad, streams_mask, states_locked); + __v4l2_link_validate_get_streams(pad, mdev_context, streams_mask, + states_locked); #else /* This shouldn't happen */ *streams_mask =3D 0; #endif } =20 -static int v4l2_subdev_link_validate_locked(struct media_link *link, bool = states_locked) +static int v4l2_subdev_link_validate_locked(struct media_link *link, + struct media_device_context *mdev_context, + bool states_locked) { struct v4l2_subdev *sink_subdev =3D media_entity_to_v4l2_subdev(link->sink->entity); @@ -1626,8 +1700,10 @@ static int v4l2_subdev_link_validate_locked(struct m= edia_link *link, bool states link->source->entity->name, link->source->index, link->sink->entity->name, link->sink->index); =20 - v4l2_link_validate_get_streams(link->source, &source_streams_mask, states= _locked); - v4l2_link_validate_get_streams(link->sink, &sink_streams_mask, states_loc= ked); + v4l2_link_validate_get_streams(link->source, mdev_context, + &source_streams_mask, states_locked); + v4l2_link_validate_get_streams(link->sink, mdev_context, + &sink_streams_mask, states_locked); =20 /* * It is ok to have more source streams than sink streams as extra @@ -1654,7 +1730,8 @@ static int v4l2_subdev_link_validate_locked(struct me= dia_link *link, bool states link->source->entity->name, link->source->index, stream, link->sink->entity->name, link->sink->index, stream); =20 - ret =3D v4l2_subdev_link_validate_get_format(link->source, stream, + ret =3D v4l2_subdev_link_validate_get_format(link->source, + mdev_context, stream, &source_fmt, states_locked); if (ret < 0) { dev_dbg(dev, @@ -1664,7 +1741,8 @@ static int v4l2_subdev_link_validate_locked(struct me= dia_link *link, bool states continue; } =20 - ret =3D v4l2_subdev_link_validate_get_format(link->sink, stream, + ret =3D v4l2_subdev_link_validate_get_format(link->sink, + mdev_context, stream, &sink_fmt, states_locked); if (ret < 0) { dev_dbg(dev, @@ -1693,7 +1771,8 @@ static int v4l2_subdev_link_validate_locked(struct me= dia_link *link, bool states return 0; } =20 -int v4l2_subdev_link_validate(struct media_link *link) +int __v4l2_subdev_link_validate(struct media_link *link, + struct media_device_context *mdev_context) { struct v4l2_subdev *source_sd, *sink_sd; struct v4l2_subdev_state *source_state, *sink_state; @@ -1716,28 +1795,35 @@ int v4l2_subdev_link_validate(struct media_link *li= nk) if (is_media_entity_v4l2_video_device(link->source->entity)) { struct media_entity *source =3D link->source->entity; =20 - if (!source->ops || !source->ops->link_validate) { + if (!source->ops || + (mdev_context && !source->ops->link_validate_context) || + (!mdev_context && !source->ops->link_validate)) { /* - * Many existing drivers do not implement the required - * .link_validate() operation for their video devices. - * Print a warning to get the drivers fixed, and return - * 0 to avoid breaking userspace. This should - * eventually be turned into a WARN_ON() when all - * drivers will have been fixed. + * Many existing drivers do not implement the correct + * .link_validate() or .link_validate_context() + * operations for their video devices. Print a warning + * to get the drivers fixed, and return 0 to avoid + * breaking userspace. This should eventually be turned + * into a WARN_ON() when all drivers will have been + * fixed. */ - pr_warn_once("video device '%s' does not implement .link_validate(), dr= iver bug!\n", + pr_warn_once("video device '%s' does not implement the correct .link_va= lidate operation: driver bug!\n", source->name); return 0; } =20 /* * Avoid infinite loops in case a video device incorrectly uses - * this helper function as its .link_validate() handler. + * this helper function as its .link_validate[_context]() + * handler. */ - if (WARN_ON(source->ops->link_validate =3D=3D v4l2_subdev_link_validate)) + if (WARN_ON(source->ops->link_validate =3D=3D v4l2_subdev_link_validate = || + source->ops->link_validate_context =3D=3D v4l2_subdev_link_validate= _context)) return -EINVAL; =20 - return source->ops->link_validate(link); + return (mdev_context && source->ops->link_validate_context) ? + source->ops->link_validate_context(link, mdev_context) : + source->ops->link_validate(link); } =20 /* @@ -1750,22 +1836,24 @@ int v4l2_subdev_link_validate(struct media_link *li= nk) sink_sd =3D media_entity_to_v4l2_subdev(link->sink->entity); source_sd =3D media_entity_to_v4l2_subdev(link->source->entity); =20 - sink_state =3D v4l2_subdev_get_unlocked_active_state(sink_sd); - source_state =3D v4l2_subdev_get_unlocked_active_state(source_sd); - + sink_state =3D v4l2_subdev_get_unlocked_state_from_mdev_ctx(sink_sd, + mdev_context); + source_state =3D v4l2_subdev_get_unlocked_state_from_mdev_ctx(source_sd, + mdev_context); states_locked =3D sink_state && source_state; =20 if (states_locked) v4l2_subdev_lock_states(sink_state, source_state); =20 - ret =3D v4l2_subdev_link_validate_locked(link, states_locked); + ret =3D v4l2_subdev_link_validate_locked(link, mdev_context, states_locke= d); =20 if (states_locked) v4l2_subdev_unlock_states(sink_state, source_state); =20 return ret; + } -EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate); +EXPORT_SYMBOL_GPL(__v4l2_subdev_link_validate); =20 bool v4l2_subdev_has_pad_interdep(struct media_entity *entity, unsigned int pad0, unsigned int pad1) diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 8087c0ae3bc0a0a95512b4b0ff5257522a104ca0..16b6b265e711cf2293ce2478ef9= 0a622beb869e5 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -1223,6 +1223,7 @@ struct v4l2_subdev { * @vfh: pointer to &struct v4l2_fh * @state: pointer to &struct v4l2_subdev_state * @owner: module pointer to the owner of this file handle + * @context: pointer to subdevice context associated with the file handle * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*`` */ struct v4l2_subdev_fh { @@ -1351,6 +1352,9 @@ int v4l2_subdev_link_validate_default(struct v4l2_sub= dev *sd, struct v4l2_subdev_format *source_fmt, struct v4l2_subdev_format *sink_fmt); =20 +int __v4l2_subdev_link_validate(struct media_link *link, + struct media_device_context *mdev_context); + /** * v4l2_subdev_link_validate - validates a media link * @@ -1368,7 +1372,35 @@ int v4l2_subdev_link_validate_default(struct v4l2_su= bdev *sd, * the video devices also implement their &media_entity_ops.link_validate * operation. */ -int v4l2_subdev_link_validate(struct media_link *link); +static inline int v4l2_subdev_link_validate(struct media_link *link) +{ + return __v4l2_subdev_link_validate(link, NULL); +} + +/** + * v4l2_subdev_link_validate_context - validates a media link in a media c= ontext + * + * @link: pointer to &struct media_link + * @mdev_context: the media device context + * + * This function calls the subdev's link_validate_context ops to validate + * if a media link is valid for streaming in a media device context. It al= so + * internally calls v4l2_subdev_link_validate_default() to ensure that wid= th, + * height and the media bus pixel code are equal on both source and sink o= f the + * link. + * + * The function can be used as a drop-in &media_entity_ops.link_validate_c= ontext + * implementation for v4l2_subdev instances. It supports all links between + * subdevs, as well as links between subdevs and video devices, provided t= hat + * the video devices also implement their + * &media_entity_ops.link_validate_context operation. + */ +static inline int +v4l2_subdev_link_validate_context(struct media_link *link, + struct media_device_context *mdev_context) +{ + return __v4l2_subdev_link_validate(link, mdev_context); +} =20 /** * v4l2_subdev_has_pad_interdep - MC has_pad_interdep implementation for s= ubdevs --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E33B22EACEF; Thu, 24 Jul 2025 14:11:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366267; cv=none; b=FHVHTX3F0U0u+tRq7UUxlJIyVMUOiHCNLA1HEXKKjEnMUPUP3hKwMV3OMavY/A1B36b2v6bapmxSqFG85KEhTWSw/CHx75REX2e2ujm+QZ5HgbgsRtwArsRBJVpvCUjzVsu4Sos+4f/vrkq6pmwbnFRSGfgGuCsUb3pmhHRTl5I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366267; c=relaxed/simple; bh=V0WqgKr/zEfw+asoSaJEdqWVxbY1tSCJAuE38B6yIuk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Y50/mZ2/6M8qFiKYhfeAw1747/TSN/Kh6jIzJ8XfFYoLXSfhM6stowtbPoi4y0SQBQT+W/rneg5rRzGVeEjwQAQsq3IszX+59H3AKU6PTqZP08zF7dvXFtcUVWGkCRbzf9DcnzGIOGSRKWafD0iJd3hlRd085FzDl/7xv6wEvjg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=tDF8ihNF; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tDF8ihNF" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 23FFB19E7; Thu, 24 Jul 2025 16:10:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366214; bh=V0WqgKr/zEfw+asoSaJEdqWVxbY1tSCJAuE38B6yIuk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=tDF8ihNFzRturUk5ExkYHvqM//6DNG0tnFhewVa9QkTR95TLsa8g0W1Cde/D66EyB 82+ld9IZSD0BGF2qppIjZoja8jY3k+IIiPz2ck02E2kFR2dRMaf2R1hiJyCaMGYbRK BVr/46+dwE58ZvUeHbCDWa7tfCEuA4sMYhIKi128= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:28 +0200 Subject: [PATCH v2 21/27] media: uapi: Add 'flags' to media_device_info Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-21-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5051; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=V0WqgKr/zEfw+asoSaJEdqWVxbY1tSCJAuE38B6yIuk=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7Q0UTep6lSZUTnL+csNpJBbJvZqx7bNQXb+ a9RSvGGif2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0AAKCRByNAaPFqFW PEJED/9eZYnxIRsTYgasIOWfgqMLm8sHX8coaxImLs+Vu0fI8Gb3H0dNKMAvYwBME4TmDRF49wQ 659iKLTU4vmTOaqi443lCYObLjhUNpCim9PWkd1GFZKn+lCyyJU77aP+RfSSPjCkjb8x1mvi4AF uc3QfKiN4EJ8634LMwHxLPfW4aFmSLQhCTmDyLhgJ3+MQkSu+LSOG0fqVAYC3XemKLzvKR2HgvV lEUxihLaqSs8Ygc+99rWOLK7Dssj9LH7Yjfz2h5FhbiXX2HfZP3SV4hBq0fJh29PxOaO5t4CWij j+0Iba4iMDem78CH1a/wmYNTMai73ZpiFc+3xR0pc8/Cg0atvJmSimSYysbp3AVoFkCTsGid0FN 5rbPx3lMkWE2+DBzwoI9ndIgOgvp3Y/z1FnEqB5rmPF2jNi8M1ZYp8PIsLW/k19lOaFLEynpiG5 nNMA7a0QcetRQ1+u/OAv1rLYWsght7oBZ1hByN95Igsbbm9TNqDZz5djXAmNh9U/JPsNWN7lc/5 wkDkc2cUZQ0apUV3zaONtXv7Nk2WRiGbRXf8dD2SE3RNK7R5eAcIHq9xVpB9Y8zsPei6UYavo2P hxtnXap39xuYInNuA75LxQKeMbx6RGVcVWwhjLJWk2Fy0FIT+/cltAlfpDo2m/is0qHpyfY85TM 4fKe2htFb2ZL4oA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B The 'struct media_device_info' type is used to return information on the media device using the MEDIA_IOC_DEVICE_INFO ioctl. Add a 'flags' field to the structure by using a 32-bit integers from the reserved space to report the media device flags to userspace. Introduce a new flag MEDIA_DEVICE_FL_CONTEXT that allows drivers to report support for multi-context operations. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/mediactl/media-ioc-device-info.rst | 6 +++++- Documentation/userspace-api/media/mediactl/media-types.rst | 13 +++++++++= ++++ drivers/media/mc/mc-device.c | 1 + include/media/media-device.h | 7 +++++++ include/uapi/linux/media.h | 6 +++++- 5 files changed, 31 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/mediactl/media-ioc-device-in= fo.rst b/Documentation/userspace-api/media/mediactl/media-ioc-device-info.r= st index d56ee6669ab9eb17a77f3714ecd5e3ef9ad65358..f2ff7a2b860cb848683a31d5731= a06c782c70316 100644 --- a/Documentation/userspace-api/media/mediactl/media-ioc-device-info.rst +++ b/Documentation/userspace-api/media/mediactl/media-ioc-device-info.rst @@ -87,7 +87,11 @@ ioctl never fails. this identifies a particular driver. =20 * - __u32 - - ``reserved``\ [31] + - ``flags`` + - Media device flags, see :ref:`media-device-flag` for more detail= s. + + * - __u32 + - ``reserved``\ [30] - Reserved for future extensions. Drivers and applications must set this array to zero. =20 diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/D= ocumentation/userspace-api/media/mediactl/media-types.rst index 6332e8395263b04935758d42a17adfe690fcd042..40e943af37c08339af928ff89e9= 37740a48893a8 100644 --- a/Documentation/userspace-api/media/mediactl/media-types.rst +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -235,6 +235,19 @@ Types and flags used to represent the media graph elem= ents =20 .. tabularcolumns:: |p{5.5cm}|p{12.0cm}| =20 +.. _media-device-flag: +.. _MEDIA-DEVICE-FL-CONTEXT: + +.. flat-table:: Media device flags + :header-rows: 0 + :stub-columns: 0 + + * - ``MEDIA_DEVICE_FL_CONTEXT`` + - The media device supports multi-context operations + + +.. tabularcolumns:: |p{5.5cm}|p{12.0cm}| + .. _media-entity-flag: .. _MEDIA-ENT-FL-DEFAULT: .. _MEDIA-ENT-FL-CONNECTOR: diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index e1f34f884fee2e8c3750f9c1e85142ff2f6b7bf0..a5cdc679a4ef11c7ec35dd69ac9= 987eb5718069f 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -121,6 +121,7 @@ static long media_device_get_info(struct media_device *= dev, void *arg) info->media_version =3D LINUX_VERSION_CODE; info->driver_version =3D info->media_version; info->hw_revision =3D dev->hw_revision; + info->flags =3D dev->flags; =20 return 0; } diff --git a/include/media/media-device.h b/include/media/media-device.h index b3cc6793a8b5eff4c26e57b01e1a62ab71e8195b..e53334fc21fa353cb3e0e7948b0= 8739855dd07ca 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -157,6 +157,7 @@ struct media_device_ops { * @serial: Device serial number (optional) * @bus_info: Unique and stable device location identifier * @hw_revision: Hardware device revision + * @flags: Media device flags * @topology_version: Monotonic counter for storing the version of the gra= ph * topology. Should be incremented each time the topology changes. * @id: Unique ID used on the last registered graph object @@ -221,6 +222,11 @@ struct media_device_ops { * necessary to run @enable_source and @disable_source handlers. * Callers should hold graph_mutex to access and call @enable_source * and @disable_source handlers. + * + * The @flags field reports the media device capabilities: + * + * %MEDIA_DEVICE_FL_CONTEXT + * indicates that the media device supports multi-context operations. */ struct media_device { /* dev->driver_data points to this struct. */ @@ -232,6 +238,7 @@ struct media_device { char serial[40]; char bus_info[32]; u32 hw_revision; + u32 flags; =20 u64 topology_version; =20 diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index 1c80b1d6bbaf36b76aaf1363dec50751469d8fed..f711bdda595718092a294336d4d= 364ba8296e08d 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -23,6 +23,9 @@ #include #include =20 +/* Media device flags. */ +#define MEDIA_DEVICE_FL_CONTEXT 0x00000001 + struct media_device_info { char driver[16]; char model[32]; @@ -31,7 +34,8 @@ struct media_device_info { __u32 media_version; __u32 hw_revision; __u32 driver_version; - __u32 reserved[31]; + __u32 flags; + __u32 reserved[30]; }; =20 /* --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A517F2EB5BF; Thu, 24 Jul 2025 14:11:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366270; cv=none; b=VuQinvCS+PpIg0UlR3Oqlb9tVDZBwG1XS6y7y1rumGh7R8rJRMt/YfXGBBvms0KGAQ8L0auCG8i9A7ePudjqjHPMPHFwcumbTeLJbxDmSPHpY1ZMMxu/tx4NMElwFsFCi10QRYTD54dnuDrxeH3dSAvsc+alRLtjjtHQeKGY9Ms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366270; c=relaxed/simple; bh=NRkJxIVd0ZlILnv8udnGakJfoaHUIwP9DNE9boCy8vE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fHeutNOegI9GFX2MQYenjh+6jlLDe6Nup2RaxJrqhQv+MygjMIOTz/O/AaYDHDQG0n83pYHhiFP5hkkalEx/e3DolZE/ihBAw04DioUdHIOIb8tQ5UR57BZGMvv/4QcGQVJPR7S6ICzeMvpAVS4lu5w0RPn3VFvrnRZIWrrXfWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=UlL2edDM; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UlL2edDM" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4DB6E1AE0; Thu, 24 Jul 2025 16:10:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366215; bh=NRkJxIVd0ZlILnv8udnGakJfoaHUIwP9DNE9boCy8vE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=UlL2edDMMY3vE+u6qV+WJ5BxDywdBIyB7RAh+Z1A2BSwPLSD/Tjbt5wH7jSgmXo2S pGBRsikEUIjG95tu0BC/fX+ieONc23GevfVFtkRlG6l/cs2+R/12EmFf/5GGga2ti2 Yh0gHaKR3Na2WZ19pYJNQxwVJ5K8BZ+kgX9479Zs= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:29 +0200 Subject: [PATCH DNI v2 22/27] media: pisp_be: Start and stop the media pipeline Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-22-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1293; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=NRkJxIVd0ZlILnv8udnGakJfoaHUIwP9DNE9boCy8vE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7QEP+68dq8KkWsJ+v1ViUzSitaYErWzTwC/ sRx00/o4rWJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0AAKCRByNAaPFqFW PGUtD/sGh0joplnq6k8QTbkDJBZ7jg6en6tZ4O2bziq8gjtTEVqGjo6UzEq3/UweR5JgoZBb1/k Tx1QTAG0XQoByC1DXrmtvmywS+XInxq5IYbgYMAUXOU5uVhYsuewLsp4Agy4zYV3j4Xe/jHC0XZ YgLcevIjOuTFEFWEt0/aD/zvlF3gKqC24dX0fnHwxvU0MQbB8rdZCvgQFUdjrTtdeqD57bU8Uso H5XbNZpI5rxaNYKtt2OxPTTEfPENl7CxH0djPgtmT+hAutEd7i5KuTQbaaN0FjTV2RpIx3/Qfah VZOcfW2Qpeg6LVCBY5p1HjwzVDsxGeTwsKI7ZtIZvmlp2uffKF7QpQ1h4lJ/yRn8icuiVgoQaBi SkxqeU2qvUzeXb/kYEjpS01FX5pq4tMTeSJB2Tgd4yREA1S/+RQcQG9nFvWVdlMw+0nQuSttOMC qJNKCcMa87drjlDepLc9wMtSC5PzlGwJ8p1kBibetqLF2ZlxMQGBsZ+4KYBBUTKkwGhAz0LbXPf m3xRQ80J+9pkklAc0OfOHXwXjDPZ+ju9eFD24deqbVNLQJ3xj+OXMdWWCh7p7q1eY1waEBdNvmw zxSmDb3WIy2ThJL9y/r+52Usz+LasLT4o+qHPz+6yrtPHjZZLmm7fZ7sp8NCupM3s1wxJnt/zTK 4NxoG8FwDE/wjcA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Call video_device_pipeline_alloc_start() and video_device_pipeline_stop() at streaming start/stop time for all video nodes. Signed-off-by: Jacopo Mondi --- drivers/media/platform/raspberrypi/pisp_be/pisp_be.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers= /media/platform/raspberrypi/pisp_be/pisp_be.c index b30891718d8df9c48ce1b83ad9fcafb201105625..423cb21298309c2ba51214b129f= bf6e875370c98 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -878,6 +878,10 @@ static int pispbe_node_start_streaming(struct vb2_queu= e *q, unsigned int count) if (ret < 0) goto err_return_buffers; =20 + ret =3D video_device_pipeline_alloc_start(&node->vfd); + if (ret) + goto err_return_buffers; + scoped_guard(spinlock_irq, &pispbe->hw_lock) { node->pispbe->streaming_map |=3D BIT(node->id); node->pispbe->sequence =3D 0; @@ -933,6 +937,8 @@ static void pispbe_node_stop_streaming(struct vb2_queue= *q) =20 vb2_wait_for_all_buffers(&node->queue); =20 + video_device_pipeline_stop(&node->vfd); + spin_lock_irq(&pispbe->hw_lock); pispbe->streaming_map &=3D ~BIT(node->id); =20 --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49E702EB5CA; Thu, 24 Jul 2025 14:11:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366270; cv=none; b=gtxA1fvxGYVG47DEJ9mSljaRqVsrmwiKYaqkU9/HAFF/F1uw5GfV5PPdYqoxqR5WwtYMhN2uQwl1dgl8xNcob3Z35HI79o+FCjiWOYVx23KmxS+fcHn1E/mg3DN/IaNXQ2DVxwhWXFgqe3vJ5IKa1Mi3yRZBacvdCkB+ID/3Fa4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366270; c=relaxed/simple; bh=9aZhUEi8nzk0nQ1lg/OYG7ZJZJnqA3OQHgxh4Ti+sSs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=P+s2Yz0xuQPr9bVtDTWpdiG/ASNWrANvkjU32rpPxmC6/ycLu2bYNysCzEWsnqD3oyBIgSSYvPBN+nxGnPt3M8rSfd9dii1t7zxIAkxdRVn+zFA2MBDN6McHhOfPeSE8SKxEg22WF2JDWrlcVGndglNIn96Vf98/Au6kqyHbTcI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=VRnt5rjt; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VRnt5rjt" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7131A1D4F; Thu, 24 Jul 2025 16:10:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366216; bh=9aZhUEi8nzk0nQ1lg/OYG7ZJZJnqA3OQHgxh4Ti+sSs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=VRnt5rjtqSYNmPTP6tyBO9oGp/Borz7nHI+YpvDhvjT4GLS3/8WaSftJ2uUDR4J1I t/c2y5ZkTBolKfu8KtGPnsKpJUefza5a/IEMqhL9lXtWRs/75yTgbkRGaq0doWW9Q6 /OClRfTTNWRe05cZ0f8t0rzsaugKE1ngi2M48MHs= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:30 +0200 Subject: [PATCH DNI v2 23/27] media: pisp_be: Add support for subdev state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-23-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2271; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=9aZhUEi8nzk0nQ1lg/OYG7ZJZJnqA3OQHgxh4Ti+sSs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7RY0e4xaDYYLJ3qoVCAQxIeUre2V57aqbY6 Ls6WAj3FueJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0QAKCRByNAaPFqFW PF4+EACO24TyFJ/23FdbJxiNhAcLd20SEuDeKlrG+PosKaIqqq1hcGdVXYWhUNWcCGOC8F4/Oqr PCxSqmosJcV4j55zUC5d7cEQaM0P4JiKYchzFJXlFAwxEBUhL3sSjwZrx81jFKP74993ZfdqGoI 9dsxHGiZ/0x+9fegBWuz6uMBY8dbuj4egcNInF6lW9CCtGv0/iqQznT9At45k0Ao08m/5BIki/+ 5Qvc+f0n49QjTw/oI1LyWUtsJJCXwowwHF8nTxke9QEhx5M5Q23aXMzbj71vpkLUxTmaBL1C5b5 FCtYXCrIvMjtyoMM70ACunHxbg02N5Mm3nREVE1SlMJgMa8VWFmradb+f58+PGIm1/YdIxCVpbc XbBOKfmAztgAWQIwkXhJ3QyOf8P3Ms7CTMNqOHSzkcalh1VvwsUdEsAiFbW9mic8chNQSAGb+t5 ZvGO9uDao0xbxkoCB9mmbVdwDjjHoa0uzTSdGvrSCPaS6WYC8wDWcPpRXXW47W8HfkABA6+HDsV 1y8RoiJaQEP14+RJZ2UwzTNzZPic96dgzB0ZhpGs7BU5FDw6Sf7/ChtKvFXJDZlVvlyqCAojdun nw5BJkda5UQ5i6n4e+BJmVVUsm3JmNzP3d4KkhaldiggZrwjkDEkowYqsdt15JZvDPUsYYTZHk2 YisWCLbWiAtUaNQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Add support for subdev state in the ISP subdevice by providing an init_state() callback and by calling init_finalize() before registering the subdev. Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 41 ++++++++++++++++++= ++++ 1 file changed, 41 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers= /media/platform/raspberrypi/pisp_be/pisp_be.c index 423cb21298309c2ba51214b129fbf6e875370c98..5aec4f8979053500c870e71ce71= 71bbd1cac9606 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1496,12 +1496,49 @@ static const struct v4l2_subdev_ops pispbe_sd_ops = =3D { .pad =3D &pispbe_pad_ops, }; =20 +static int pispbe_init_state(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state) +{ + struct v4l2_mbus_framefmt *fmt; + + for (unsigned int i =3D 0; i < PISPBE_NUM_NODES; i++) { + fmt =3D v4l2_subdev_state_get_format(state, i); + + switch (i) { + case MAIN_INPUT_NODE: + fallthrough; + case OUTPUT0_NODE: + fallthrough; + case OUTPUT1_NODE: + fmt->width =3D 1920; + fmt->height =3D 1080; + fmt->code =3D MEDIA_BUS_FMT_FIXED; + break; + case CONFIG_NODE: + fmt->width =3D sizeof(struct pisp_be_tiles_config); + fmt->height =3D 1; + fmt->code =3D MEDIA_BUS_FMT_METADATA_FIXED; + break; + default: + /* No need to configure other nodes. */ + continue; + } + } + + return 0; +} + +static const struct v4l2_subdev_internal_ops pispbe_subdev_internal_ops = =3D { + .init_state =3D pispbe_init_state, +}; + static int pispbe_init_subdev(struct pispbe_dev *pispbe) { struct v4l2_subdev *sd =3D &pispbe->sd; int ret; =20 v4l2_subdev_init(sd, &pispbe_sd_ops); + sd->internal_ops =3D &pispbe_subdev_internal_ops; sd->entity.function =3D MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER; sd->owner =3D THIS_MODULE; sd->dev =3D pispbe->dev; @@ -1517,6 +1554,10 @@ static int pispbe_init_subdev(struct pispbe_dev *pis= pbe) if (ret) goto error; =20 + ret =3D v4l2_subdev_init_finalize(sd); + if (ret) + goto error; + ret =3D v4l2_device_register_subdev(&pispbe->v4l2_dev, sd); if (ret) goto error; --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8D9B2EACEF; Thu, 24 Jul 2025 14:11:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366273; cv=none; b=N5/aorm6lQZQXsm5XhuKfLbp9epboIMd1WXdLoJMTNF5+54TobNFleu3rGut2e5+h+L5jbAa4Lw7IRBQPw/CYr/ctLj2IJTT7M5cigxOCKClh3sdSLFLvlbrQYeoazB7xD+fsAcVutrbDDbmii1eLYXF3BM1ULwY5ex6vQuCy8I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366273; c=relaxed/simple; bh=YyXmr6Wtr5A4ih2ag5qo99kLDMbBWLONqHu1vyrmcVI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dXSDgGEZcMoZ+Wz5lZht9hWjOgEwx0Tn88daEgooljtkc6W+SYguR9xuT0r4n/4y1p0bywe/F4oXPKHz6LWmocDgindKjRAVgviRBbFddazMMue+FrjaeqomXjSy2dqOrA6/nwi8Lgji/DE5JDbFBnKA2EJcFXdfm61CUNt0Ty8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=h+ht9XLw; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="h+ht9XLw" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 94EBAC64; Thu, 24 Jul 2025 16:10:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366217; bh=YyXmr6Wtr5A4ih2ag5qo99kLDMbBWLONqHu1vyrmcVI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=h+ht9XLwnR7UUmgO5C1AnL/fmYRCmYCmDKsEl4ajfFSsNUFW/vKCksTmj00tKoXNZ XmVRYnAutmBWI+AAIXuys8VnL05rd7n5Zu1h0LZsZCEMmhXwh5gByar/62zTFgV+bu 4Ci1jWtb3K76SoOaZ3gXHz0MghHEvgo7Uk+hdolI= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:31 +0200 Subject: [PATCH DNI v2 24/27] media: pisp_be: Implement set/get_pad_fmt Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-24-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1831; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=YyXmr6Wtr5A4ih2ag5qo99kLDMbBWLONqHu1vyrmcVI=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7R60gOltUMYV82hQEHCOM7Z9Qxe0nMqQmWx fEVl6pPqgGJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0QAKCRByNAaPFqFW PHkCD/9qdD97ai0c7MCMnH3YjlFvnHU4m+d7AKDsxdcHUkaQC3fNm0Vf5s7ThTfX8ag8sil/2SR p1LfNxyGmzquhg+qI1Nl26LHi4DdSSXrxSk+/O13c0pKG6cIvoWit0DtnW4eGmX+ju7Q9t09fN8 V22Eheu5sYjw0Cyi+C5PQU1VQGt3iow9/VXWVwPwuaQNgD16CvIVjkVLNiQZ1DXHMoafj+Plru1 sIlc+fimyMZphtvb8CfrAvnKCCyY+nfKx02gNqgo3xN/cO6sOmbzjSI45F6gjBRSwcwPga+oub5 93Az+XiI8+BJEs7XMy/yBLmBTvyXgX882Ge61o+49nuTnunSnpj7NqSg5vRhQTAKXjK3TvkkBOb FHOeo8HlR5V2dsXuR+ueC4q8u1RVNb/7f0JBWUNMdcybKpt392/G8++fRnYIgw44g2o36V3VnxG D25Ty2DGKhxyxJ/T/EetSfMEUhyT23/a0jVjkMggNb6/EkE9WiT42WZkywHDVRAR+NvYLLmM7KD sCaDuCzNPhUubYTXxyO25NaALJ/oZyAE4vVYpv6I/82FRO0E6RqeoqQImOpF+EKSxjdssqF+RVI tgg+l012YM4Qeo5fxrQiY8JdcW7Ly50ejzQ9hstqd+MfSfSHHeExGy9QY9VoThzdMMrDAyIw5ij DNh9e56xdOx2BAQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Implement operation handlers for the set and get pad_fmt subdev pad operations. Format is only modifiable on the image nodes (of both output and capture types). Only sizes can be modified. Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 29 ++++++++++++++++++= ++++ 1 file changed, 29 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers= /media/platform/raspberrypi/pisp_be/pisp_be.c index 5aec4f8979053500c870e71ce7171bbd1cac9606..22e440f387b5e5560b2cc80a8b3= bf6064dc12d7c 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1488,7 +1488,36 @@ static int pispbe_init_node(struct pispbe_dev *pispb= e, unsigned int id) return ret; } =20 +static int pispbe_subdev_set_pad_fmt(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, + struct v4l2_subdev_format *format) +{ + struct v4l2_mbus_framefmt *fmt =3D &format->format; + + /* Only allow setting sizes on the image input and output pads. */ + + switch (format->pad) { + case TDN_INPUT_NODE: + case STITCH_INPUT_NODE: + case TDN_OUTPUT_NODE: + case STITCH_OUTPUT_NODE: + return -EINVAL; + } + + fmt->width =3D clamp(fmt->width, PISP_BACK_END_MIN_TILE_WIDTH, + PISP_BACK_END_MAX_TILE_WIDTH); + fmt->height =3D clamp(fmt->height, PISP_BACK_END_MIN_TILE_HEIGHT, + PISP_BACK_END_MAX_TILE_HEIGHT); + fmt->code =3D MEDIA_BUS_FMT_FIXED; + + *v4l2_subdev_state_get_format(state, format->pad) =3D *fmt; + + return 0; +} + static const struct v4l2_subdev_pad_ops pispbe_pad_ops =3D { + .set_fmt =3D pispbe_subdev_set_pad_fmt, + .get_fmt =3D v4l2_subdev_get_fmt, .link_validate =3D v4l2_subdev_link_validate_default, }; =20 --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 845032ECD00; Thu, 24 Jul 2025 14:11:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366274; cv=none; b=EI5+AtrshOzhoMU+ZUpGbsgsoHUT4wwbWqzd8SsRDdZxTxX1VCP8VBzUohceqyAnACuvNLEDdqFFfs0TPgzn73DeSPSUpbX6bafvdcISveIZLG1vZJQ8+qyAsvaNJl4D5l5tDtpQ1i6dsY8L/q5LvMljQ6ficcR8w6aUYUtCnIE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366274; c=relaxed/simple; bh=Ee9J4mCFwsXOb6mRepzI/4oK3qWzitZE/F4KVxjoUTk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Gf1YT5xLEaditx84OryF4OlDPOPWAacbmUl57op68L+aP+4nHq5/xscgjPQ+9DaBf0niW+V0xq828S1tgzjZZMPPq8kQnqJbVLMFv2svyeNNTRMOlpW5EmmL8cxnQ1SUHtp40/+apYcoflmU3jqMYWfAlDGizWJl2Nz1aEz2reM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=pzHa1tjZ; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pzHa1tjZ" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CF2DA1C0A; Thu, 24 Jul 2025 16:10:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366218; bh=Ee9J4mCFwsXOb6mRepzI/4oK3qWzitZE/F4KVxjoUTk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pzHa1tjZMGgYMU078QicAtgwWMuYcA9cnDrSFTuykjJHaoEKfgnLhDSXu5z0QhEpQ jY2bsX5Mv18P89zp/y6H9tdEG/sgVCEI0ob5Ee92O8SBobDukmlGqNtICsnABIJuKu 5FNMJ4n9+HP2YVdteavsS1Rc9eQhjvmWcvzDGzxY= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:32 +0200 Subject: [PATCH DNI v2 25/27] media: pisp_be: Implement link validation Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-25-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3786; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Ee9J4mCFwsXOb6mRepzI/4oK3qWzitZE/F4KVxjoUTk=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7RDaGW0mIc3AfkJWN26XnOzBGJjJeUiPFQd JH2LcAA4PWJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0QAKCRByNAaPFqFW PG+CD/4mldTV0hLxTaCPPXBaV6zXOSZrS3SQeXmb5Q5F6gOH8g+iZNHcl1hnJbPLgT6JzC0yHiz 0fzTFmKzPy+AYJY64XeKBKK2UEgcYmPYRAt3GXtd1voVIepfSnhCEPBJ0j7qJoD9034kwsZNzRL Jyr//C+HnZJuWF3UHFXyQ5IupxB6fEAiOr+PgRB/b1WfaPMAY27JT1FTaJaDZkGR7cKb9m9ia44 HUYWIWdP8t55v0R6Cry0RY7hJ+7YTj3kkeKbPimvKx6Cxaix5x7YvJT158VS5N1LSDUpqbt6+A7 YWS6TveLGCrPaETMcQt0FEfIwwkF6mjoi+ceZkw4iyFe5NTQrK7tc8vNfHCpGyReLlLaFD8iFgh REgXFd/j7PMj3C1/Irhgl1W4a6zabgaeZn9VDmBVwridn6YL0lVqjv3lFov53waE5LG6nGEkRYU /56Bfb2ae4kbt0UeQXutV0qanklfaV1EOMn3CtrB4flUTj8h3V5/uN7PST8sjwfVF4ixIUuIGOL 7InbN3r3cbniY64r2sgzC+S/xJGMXVZjEEgz0pv70yOFq6j55sJzh+JYeacaD5qu+VvnSrIW2x/ RFN3obsvJug96z1KROEyDfSla9RgVWmmBBEdPCIQgigpviGBqVGb+4dTsCrlcZigEhulkBvk58X 3hUMPlEKlUcYDgg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Implement link validation to validate that the sizes of the format on the video device matches the size programmed on the ISP subdevice connected pad. Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 67 ++++++++++++++++++= ++++ 1 file changed, 67 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers= /media/platform/raspberrypi/pisp_be/pisp_be.c index 22e440f387b5e5560b2cc80a8b3bf6064dc12d7c..2a8c09a9c70952c9f99e542271e= 994d62392c617 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1362,6 +1362,67 @@ static const struct v4l2_ioctl_ops pispbe_node_ioctl= _ops =3D { .vidioc_streamoff =3D vb2_ioctl_streamoff, }; =20 +static int pispbe_link_validate(struct media_link *link) +{ + const struct v4l2_mbus_framefmt *sd_fmt; + struct v4l2_pix_format_mplane *pix_mp; + struct v4l2_subdev_state *state; + struct media_entity *vdev_ent; + struct media_entity *sd_ent; + struct pispbe_node *node; + struct v4l2_subdev *sd; + + if (is_media_entity_v4l2_video_device(link->source->entity)) { + vdev_ent =3D link->source->entity; + sd_ent =3D link->sink->entity; + } else { + sd_ent =3D link->source->entity; + vdev_ent =3D link->sink->entity; + } + + node =3D container_of(media_entity_to_video_device(vdev_ent), + struct pispbe_node, vfd); + switch (node->id) { + case TDN_INPUT_NODE: + fallthrough; + case STITCH_INPUT_NODE: + fallthrough; + case TDN_OUTPUT_NODE: + fallthrough; + case STITCH_OUTPUT_NODE: + fallthrough; + case CONFIG_NODE: + /* Skip validation for these nodes. */ + return 0; + } + pix_mp =3D &node->format.fmt.pix_mp; + + sd =3D media_entity_to_v4l2_subdev(sd_ent); + state =3D v4l2_subdev_get_unlocked_active_state(sd); + sd_fmt =3D v4l2_subdev_state_get_format(state, node->id); + + /* Only check for sizes. */ + if (pix_mp->width !=3D sd_fmt->width) { + dev_dbg(node->pispbe->dev, + "%s: width does not match (vdev %u, sd %u)\n", + __func__, pix_mp->width, sd_fmt->width); + return -EPIPE; + } + + if (pix_mp->height !=3D sd_fmt->height) { + dev_dbg(node->pispbe->dev, + "%s: height does not match (vdev %u, sd %u)\n", + __func__, pix_mp->height, sd_fmt->height); + return -EPIPE; + } + + return 0; +} + +static const struct media_entity_operations pispbe_node_entity_ops =3D { + .link_validate =3D pispbe_link_validate, +}; + static const struct video_device pispbe_videodev =3D { .name =3D PISPBE_NAME, .vfl_dir =3D VFL_DIR_M2M, /* gets overwritten */ @@ -1445,6 +1506,7 @@ static int pispbe_init_node(struct pispbe_dev *pispbe= , unsigned int id) vdev->device_caps =3D V4L2_CAP_STREAMING | node_desc[id].caps; =20 node->pad.flags =3D output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK; + entity->ops =3D &pispbe_node_entity_ops; ret =3D media_entity_pads_init(entity, 1, &node->pad); if (ret) { dev_err(pispbe->dev, @@ -1561,6 +1623,10 @@ static const struct v4l2_subdev_internal_ops pispbe_= subdev_internal_ops =3D { .init_state =3D pispbe_init_state, }; =20 +static const struct media_entity_operations pispbe_subdev_entity_ops =3D { + .link_validate =3D v4l2_subdev_link_validate, +}; + static int pispbe_init_subdev(struct pispbe_dev *pispbe) { struct v4l2_subdev *sd =3D &pispbe->sd; @@ -1569,6 +1635,7 @@ static int pispbe_init_subdev(struct pispbe_dev *pisp= be) v4l2_subdev_init(sd, &pispbe_sd_ops); sd->internal_ops =3D &pispbe_subdev_internal_ops; sd->entity.function =3D MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER; + sd->entity.ops =3D &pispbe_subdev_entity_ops; sd->owner =3D THIS_MODULE; sd->dev =3D pispbe->dev; strscpy(sd->name, PISPBE_NAME, sizeof(sd->name)); --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA4C62E1738; Thu, 24 Jul 2025 14:11:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366276; cv=none; b=HJOwu08G9vFzT4J8pct3uXnfWBXs7kEOK5SPM60A5q0AIoO+08psj5YX/cnBUwSje3xlgO2Oz+gIwf9PqOVGaLPf69wOaKUKzYHHYhz6ZmXRQfCVaJRQOwN+014KHjpzUwxwz9G8SGVLuXkN1gZmrDHOw2JsBc9IuirivqrhpWw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366276; c=relaxed/simple; bh=veD5bCafCFdh1GJ2SEAikts/NXjEHZdIvuKFJ7cMJ+I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=K/OtJFTa63uYu7htwU6zo0n65ND/RdbtlN8CJf4w74YTX+hzdmZRvp8PhD3Zp0pJRrVyHCaqFIA/59I5/TWu4561vLEWNwIC33QmLzVOg3NoniMrXoaIKiQhxMlRjgoFwfuuS8QEU5TW0DxWfdyAiIdKzktLIT1IGzYNVFYnX7I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=qpJsR4Eq; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qpJsR4Eq" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 01E38190D; Thu, 24 Jul 2025 16:10:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366219; bh=veD5bCafCFdh1GJ2SEAikts/NXjEHZdIvuKFJ7cMJ+I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qpJsR4Eq1CYBfrwX4+EEF4wZye0yZPR0NiKQyRDv6d2Npq65JgUZg+MGO9BW/505M Qpw/rKtLIDtxIUvveNkpoNnfsLmbnOh5Wcwrs67ARdRkmSxnYpOOFT52H+nh7+226l m/65qY24S9anJ0ztB3E6f4pREDr6O3jGGsPDm7ZM= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:33 +0200 Subject: [PATCH DNI v2 26/27] media: pisp_be: Register devnode to userspace Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-26-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1240; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=veD5bCafCFdh1GJ2SEAikts/NXjEHZdIvuKFJ7cMJ+I=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7RK2B7rlcZ6phF6WyDjWYLWReVmALIB47qD sLD5LQyjAGJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0QAKCRByNAaPFqFW PCgJEACHE03JmXOSbqh81a4WUrMGcAq/5CVVYdSIVk1PaAkQ21odSIEGu7iNLoHQ1b5toGTNfLt kFnAY94A6HhJNosfjvGUgKaaN24nc9NczLrJnbyqMRZ+rJAE7DM3qg2iWUsdGPBgutPJEJbT4c7 rjfALg5/+0RLq3DbD5cGLyppvcmTA7l3HQLLT1WWNhWDXWy/wdptQqD6Y3EszYKBxtYKkvWrlf3 BtuOZ3Bg26O0Yvrq7cpxaIhmTc/EpC6EPhMk1PkQV9yOMG13Hckt3zwPtJ5u3TFqMACeKnhQUNG vvWbyhkBFN9CKUrIJFr5OPcPWN20IQMalkw7R+ofp0fygP9O8U0WZzikXlCOARghMffTXFUuLFL SEx/6vDRyQagfEZf9JHHAnfBtIO0BxwYToOzMspMaXev4f/ao6xEAVsr1Aonp3CGojVNobVe53+ jy0fYVDxncPEOyw7TGMJbho4FG/tvi7HLJFkIuixU0pu1PY2rZ1+dvQiMc5xpwdAi/rQZeC53/t dqnSBc4YA7CMiNf1FafTk0vbqEuQU7RrKT7hCC+YyPJPbwPsMPDtRwMo7RyXZUg/OELfs8NBVYC lt+iWcig19nDTsYCW3q0ZhtwkuR5L2RGmzaPhUDHU5UiPtDBTduTnlRX1GQE7DRnPE/r3kGBH6l QaXOUYMWgWhyeDw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B To set formats on the subdev, we need to have the devnode registered to userspace. Signed-off-by: Jacopo Mondi --- drivers/media/platform/raspberrypi/pisp_be/pisp_be.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers= /media/platform/raspberrypi/pisp_be/pisp_be.c index 2a8c09a9c70952c9f99e542271e994d62392c617..df3cdd81843376abf98bb184cde= 74d4d66b0ecfe 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1635,6 +1635,7 @@ static int pispbe_init_subdev(struct pispbe_dev *pisp= be) v4l2_subdev_init(sd, &pispbe_sd_ops); sd->internal_ops =3D &pispbe_subdev_internal_ops; sd->entity.function =3D MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER; + sd->flags =3D V4L2_SUBDEV_FL_HAS_DEVNODE; sd->entity.ops =3D &pispbe_subdev_entity_ops; sd->owner =3D THIS_MODULE; sd->dev =3D pispbe->dev; @@ -1658,7 +1659,7 @@ static int pispbe_init_subdev(struct pispbe_dev *pisp= be) if (ret) goto error; =20 - return 0; + return v4l2_device_register_subdev_nodes(&pispbe->v4l2_dev); =20 error: media_entity_cleanup(&sd->entity); --=20 2.49.0 From nobody Mon Oct 6 04:57:10 2025 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70AB92ECEA1; Thu, 24 Jul 2025 14:11:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366278; cv=none; b=IbAkOwR8Cw2c4ICm9+Ck3c0YMjwIfmx/ZPjuVQgR32Qu5abBH9Q8LizInAUVjKWS7Au5jyl4CI1/YMsB9K45Oa0iWHwtrLnv9OIfv/DgGKWnJgyP0p/fa0ItGN0VyKqCjxuNUlKVrrnJJV3rOK2WkRuwqEJBJ3yvynBO6WiT398= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753366278; c=relaxed/simple; bh=5wBv8PH1Xcf7VhSHcn5ChTvkKVL5+HspXWeE12Zm/Jc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ejCwNC05WQ2qXhHXoBbkUsDPZ7ch5KPAemuqoOCKLeqdsAxCZmp4Mcz5gArNPg9ZFUTDtMETH2Xud11+AEMOuLwhdGmxoOQgH7Y+7SbnFiaR7fm9MPjB51gEJKx9Uh4Tej/gQUtfvS7nE+PHNno+C28KMEzPZIMPPY2vd7L4PT0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=dK5a2K1Q; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dK5a2K1Q" Received: from [192.168.1.76] (unknown [IPv6:2001:b07:6462:5de2:520d:d7a3:63ca:99e8]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 249721A04; Thu, 24 Jul 2025 16:10:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753366221; bh=5wBv8PH1Xcf7VhSHcn5ChTvkKVL5+HspXWeE12Zm/Jc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dK5a2K1QUbxhe4wSr5i/3IIGy34AqbmzHrqa6LmYs/VND0eQiXwMhEq4NOeyBD5d3 r47lmn5J0zUyGoAoYKPfA63ZH/9toCB1/CRz9gbtWggoa4G4QFnLje3QBlIyebewKL aGDrGOMMZBT5I9vj99e6slRD00rv1imYcqIf3hHc= From: Jacopo Mondi Date: Thu, 24 Jul 2025 16:10:34 +0200 Subject: [PATCH DNI v2 27/27] media: pisp_be: Add support for multi-context Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-multicontext-mainline-2025-v2-27-c9b316773486@ideasonboard.com> References: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> In-Reply-To: <20250724-multicontext-mainline-2025-v2-0-c9b316773486@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=45518; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=5wBv8PH1Xcf7VhSHcn5ChTvkKVL5+HspXWeE12Zm/Jc=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBogj7R+2Po1qQZwBmMotXD3r6ZZHDmKnGJZ7yN0 ZY0ldofW7+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaII+0QAKCRByNAaPFqFW PA+6D/9KYWguLGw/Q3bl46tneyKGcbS0CwJMprJlfHviI739/G26exWBTMAy+K9VV3PgEE9FA/P kSJa0QyH91lpMD9gOFpFiXyInsUKfoAZND6pmcbMQPZNRDVITvNdpeQsZhb+vD9SjfLIod12RgE 5N0U1wEvw0Q2siHwbKLsi57mhWbSiadm1i8Ax8EnaS67YIyWMmima0U3ChD7QR7ketAWQ634GL9 SFQSx1AtjzEYAsPhXXNH2eQk+tn2659ylg6z0/k0Bsa3ahi3MZSt/SHjAaAthGw3EA7UsZswGqw iIlYyVE0rvXYpcHEtYF/LdulQ1Vtc5W/ebwxuw/tWjLZ7vDj/W6yUFCsDgyHDbp/BLay+D//xr4 k0bIt4LrEwWW8tfX7EeI7pZNmFYEoY+fQSRKmiBZYS33MOQ8bZojOq/PZhzt2LiSdvVnuU5+e2z 3p5kQcnDJrgh25tckyT+8ZovMq9ai0u9G9mAwpdJGXe0miaQQi5HRNt0a1SMJX442RDtrJ8F9JU E9XawCuNceqNgrx1vgfrxErRen+IzYw2hZeJnlJkVb+FxL5A86VGtEybkztQLTfOPXA4dn1oLtZ Y/tY205T6uXPDCbr8Ein+RPsc5zFG44bVZ3zfTGjkUCeJOQqO6Nn8GORo7rVAHC8eiUXyt27mel Jl8kVmq438RbC2g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B Add support for context multiplexing to the PiSP BackEnd driver. Move information that used to be per-node to the video context and global information to the media context. Implement media and video context allocation and release operations to allocate and initialize a context and initialize the video queue there contained. Remove the per-node video device format and the buffer queues and port the driver to use the ones in the video device context. The operations that used to work with a pispbe_node now operates on a pispbe_context. - v4l2 ioctl ops: receive a file pointer, retrieve the associated context from the open file handle - vb2 ops: receive a queue, retrieve the context from the queue - internal driver ops: given a media device context retrieve the video context from the other video devices when assembling a job Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 653 +++++++++++++++--= ---- 1 file changed, 475 insertions(+), 178 deletions(-) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers= /media/platform/raspberrypi/pisp_be/pisp_be.c index df3cdd81843376abf98bb184cde74d4d66b0ecfe..8f45d6e3437f820f99dc9ed2b30= 38077eb5632e4 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -146,6 +147,29 @@ static const struct pispbe_node_description node_desc[= PISPBE_NUM_NODES] =3D { ((node)->buf_type =3D=3D V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) || \ ((node)->buf_type =3D=3D V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) =20 +/* -----------------------------------------------------------------------= ----- + * Media device context + */ + +struct pispbe_media_context { + struct media_device_context mdev_context; + u32 streaming_map; + unsigned int sequence; + dma_addr_t config_dma_addr; + struct pisp_be_tiles_config *config; +}; + +static struct pispbe_media_context * +pispbe_media_context(struct media_device_context *mdev_context) +{ + return container_of(mdev_context, struct pispbe_media_context, + mdev_context); +} + +/* -----------------------------------------------------------------------= ----- + * Video device context + */ + /* * Structure to describe a single node /dev/video which represents a si= ngle * input or output queue to the PiSP Back End device. @@ -161,14 +185,96 @@ struct pispbe_node { struct pispbe_dev *pispbe; /* Video device lock */ struct mutex node_lock; - /* vb2_queue lock */ - struct mutex queue_lock; - struct list_head ready_queue; - struct vb2_queue queue; +}; + +static struct pispbe_node *pispbe_node_from_vdev(struct video_device *vdev) +{ + return container_of(vdev, struct pispbe_node, vfd); +} + +/* Structure to describe a single execution context for a node. */ + +struct pispbe_node_context { + struct video_device_context vdev_context; + struct v4l2_format format; + struct list_head ready_queue; const struct pisp_be_format *pisp_format; + struct pispbe_dev *pispbe; + struct pispbe_node *node; +}; + +static struct pispbe_node_context * +pispbe_node_context(struct video_device_context *ctx) +{ + return container_of(ctx, struct pispbe_node_context, vdev_context); +} + +static struct pispbe_node_context * +pispbe_node_context_from_entity(struct media_entity_context *ctx) +{ + struct video_device_context *vdev_context =3D + container_of(ctx, struct video_device_context, base); + + return pispbe_node_context(vdev_context); +} + +static struct pispbe_node_context * +pispbe_node_context_from_queue(struct vb2_queue *queue) +{ + return pispbe_node_context(video_device_context_from_queue(queue)); +} + +static struct pispbe_node_context * +pispbe_node_context_from_file(struct file *file, struct video_device *vfd) +{ + return pispbe_node_context(video_device_context_from_file(file, vfd)); +} + +static struct pispbe_node_context * +pispbe_get_dev_context(struct pispbe_media_context *pispbe_mdev_context, + struct video_device *vdev) +{ + struct video_device_context *ctx =3D + video_device_context_get(&pispbe_mdev_context->mdev_context, + vdev); + + return ctx ? pispbe_node_context(ctx) : NULL; +} + +static void pispbe_put_dev_context(struct pispbe_node_context *ctx) +{ + video_device_context_put(&ctx->vdev_context); +} + +static struct pispbe_media_context * +pispbe_media_context_from_dev(struct pispbe_node_context *context) +{ + return pispbe_media_context(context->vdev_context.base.mdev_context); +} + +/* -----------------------------------------------------------------------= ----- + * ISP subdevice context + */ +struct pispbe_subdev_context { + struct v4l2_subdev_context sd_context; }; =20 +static struct pispbe_subdev_context * +pispbe_subdev_context(struct v4l2_subdev_context *ctx) +{ + return container_of(ctx, struct pispbe_subdev_context, sd_context); +} + +static struct pispbe_subdev_context * +pispbe_subdev_context_from_entity(struct media_entity_context *ctx) +{ + struct v4l2_subdev_context *sd_context =3D + container_of(ctx, struct v4l2_subdev_context, base); + + return pispbe_subdev_context(sd_context); +} + /* For logging only, use the entity name with "pispbe" and separator remov= ed */ #define NODE_NAME(node) \ (node_desc[(node)->id].ent_name + sizeof(PISPBE_NAME)) @@ -181,6 +287,7 @@ struct pispbe_job { * then captures, then metadata last. */ struct pispbe_buffer *buf[PISPBE_NUM_NODES]; + struct pispbe_media_context *context; }; =20 struct pispbe_hw_enables { @@ -196,6 +303,7 @@ struct pispbe_job_descriptor { struct pisp_be_tiles_config *config; struct pispbe_hw_enables hw_enables; dma_addr_t tiles; + struct pispbe_media_context *context; }; =20 /* @@ -205,7 +313,6 @@ struct pispbe_job_descriptor { struct pispbe_dev { struct device *dev; struct pispbe_dev *pispbe; - struct pisp_be_tiles_config *config; void __iomem *be_reg_base; struct clk *clk; struct v4l2_device v4l2_dev; @@ -213,17 +320,15 @@ struct pispbe_dev { struct media_device mdev; struct media_pad pad[PISPBE_NUM_NODES]; /* output pads first */ struct pispbe_node node[PISPBE_NUM_NODES]; - dma_addr_t config_dma_addr; - unsigned int sequence; - u32 streaming_map; struct pispbe_job queued_job, running_job; - /* protects "hw_busy" flag, streaming_map and job_queue */ + /* protects "hw_busy" flag and job_queue */ spinlock_t hw_lock; bool hw_busy; /* non-zero if a job is queued or is being started */ struct list_head job_queue; int irq; u32 hw_version; u8 done, started; + struct media_pipeline pipe; }; =20 static u32 pispbe_rd(struct pispbe_dev *pispbe, unsigned int offset) @@ -307,30 +412,31 @@ struct pispbe_buffer { }; =20 static int pispbe_get_planes_addr(dma_addr_t addr[3], struct pispbe_buffer= *buf, - struct pispbe_node *node) + struct pispbe_node_context *context) { - unsigned int num_planes =3D node->format.fmt.pix_mp.num_planes; + struct v4l2_format *format =3D &context->format; + unsigned int num_planes =3D format->fmt.pix_mp.num_planes; unsigned int plane_factor =3D 0; unsigned int size; unsigned int p; =20 - if (!buf || !node->pisp_format) + if (!buf || !context->pisp_format) return 0; =20 /* * Determine the base plane size. This will not be the same - * as node->format.fmt.pix_mp.plane_fmt[0].sizeimage for a single + * as format->fmt.pix_mp.plane_fmt[0].sizeimage for a single * plane buffer in an mplane format. */ - size =3D node->format.fmt.pix_mp.plane_fmt[0].bytesperline * - node->format.fmt.pix_mp.height; + size =3D format->fmt.pix_mp.plane_fmt[0].bytesperline * + format->fmt.pix_mp.height; =20 for (p =3D 0; p < num_planes && p < PISPBE_MAX_PLANES; p++) { addr[p] =3D vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, p); - plane_factor +=3D node->pisp_format->plane_factor[p]; + plane_factor +=3D context->pisp_format->plane_factor[p]; } =20 - for (; p < PISPBE_MAX_PLANES && node->pisp_format->plane_factor[p]; p++) { + for (; p < PISPBE_MAX_PLANES && context->pisp_format->plane_factor[p]; p+= +) { /* * Calculate the address offset of this plane as needed * by the hardware. This is specifically for non-mplane @@ -338,7 +444,7 @@ static int pispbe_get_planes_addr(dma_addr_t addr[3], s= truct pispbe_buffer *buf, * for the V4L2_PIX_FMT_YUV420 format. */ addr[p] =3D addr[0] + ((size * plane_factor) >> 3); - plane_factor +=3D node->pisp_format->plane_factor[p]; + plane_factor +=3D context->pisp_format->plane_factor[p]; } =20 return num_planes; @@ -354,11 +460,13 @@ static dma_addr_t pispbe_get_addr(struct pispbe_buffe= r *buf) =20 static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, struct pispbe_job_descriptor *job, + struct pispbe_media_context *mdev_context, struct pispbe_buffer *buf[PISPBE_NUM_NODES]) { struct pispbe_hw_enables *hw_en =3D &job->hw_enables; struct pisp_be_tiles_config *config =3D job->config; dma_addr_t *addrs =3D job->hw_dma_addrs; + struct pispbe_node_context *ctx; int ret; =20 /* Take a copy of the "enable" bitmaps so we can modify them. */ @@ -369,8 +477,10 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pisp= be, * Main input first. There are 3 address pointers, corresponding to up * to 3 planes. */ - ret =3D pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], - &pispbe->node[MAIN_INPUT_NODE]); + ctx =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[MAIN_INPUT_NODE].vfd); + ret =3D pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], ctx); + pispbe_put_dev_context(ctx); if (ret <=3D 0) { /* Shouldn't happen, we have validated an input is available. */ dev_warn(pispbe->dev, "ISP-BE missing input\n"); @@ -426,9 +536,11 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pisp= be, =20 /* Main image output channels. */ for (unsigned int i =3D 0; i < PISP_BACK_END_NUM_OUTPUTS; i++) { + ctx =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + i].vfd); ret =3D pispbe_get_planes_addr(addrs + 7 + 3 * i, - buf[OUTPUT0_NODE + i], - &pispbe->node[OUTPUT0_NODE + i]); + buf[OUTPUT0_NODE + i], ctx); + pispbe_put_dev_context(ctx); if (ret <=3D 0) hw_en->rgb_enables &=3D ~(PISP_BE_RGB_ENABLE_OUTPUT0 << i); } @@ -449,37 +561,48 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pis= pbe, * * Returns 0 if a job has been successfully prepared, < 0 otherwise. */ -static int pispbe_prepare_job(struct pispbe_dev *pispbe) +static int pispbe_prepare_job(struct pispbe_dev *pispbe, + struct pispbe_node_context *context) { + struct pispbe_media_context *mdev_context =3D + pispbe_media_context_from_dev(context); struct pispbe_job_descriptor __free(kfree) *job =3D NULL; struct pispbe_buffer *buf[PISPBE_NUM_NODES] =3D {}; + struct pispbe_node_context *ctx; unsigned int streaming_map; unsigned int config_index; - struct pispbe_node *node; =20 lockdep_assert_irqs_enabled(); =20 scoped_guard(spinlock_irq, &pispbe->hw_lock) { static const u32 mask =3D BIT(CONFIG_NODE) | BIT(MAIN_INPUT_NODE); =20 - if ((pispbe->streaming_map & mask) !=3D mask) + if ((mdev_context->streaming_map & mask) !=3D mask) return -ENODEV; =20 /* * Take a copy of streaming_map: nodes activated after this * point are ignored when preparing this job. */ - streaming_map =3D pispbe->streaming_map; + streaming_map =3D mdev_context->streaming_map; } =20 job =3D kzalloc(sizeof(*job), GFP_KERNEL); if (!job) return -ENOMEM; =20 - node =3D &pispbe->node[CONFIG_NODE]; - buf[CONFIG_NODE] =3D list_first_entry_or_null(&node->ready_queue, + job->context =3D mdev_context; + + ctx =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[CONFIG_NODE].vfd); + if (!ctx) + return -ENODEV; + + buf[CONFIG_NODE] =3D list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (!buf[CONFIG_NODE]) return -ENODEV; =20 @@ -487,8 +610,8 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) job->buffers[CONFIG_NODE] =3D buf[CONFIG_NODE]; =20 config_index =3D buf[CONFIG_NODE]->vb.vb2_buf.index; - job->config =3D &pispbe->config[config_index]; - job->tiles =3D pispbe->config_dma_addr + + job->config =3D &mdev_context->config[config_index]; + job->tiles =3D mdev_context->config_dma_addr + config_index * sizeof(struct pisp_be_tiles_config) + offsetof(struct pisp_be_tiles_config, tiles); =20 @@ -529,12 +652,17 @@ static int pispbe_prepare_job(struct pispbe_dev *pisp= be) ignore_buffers =3D true; } =20 - node =3D &pispbe->node[i]; - /* Pull a buffer from each V4L2 queue to form the queued job */ - buf[i] =3D list_first_entry_or_null(&node->ready_queue, + ctx =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx && !ignore_buffers) + goto err_return_buffers; + + buf[i] =3D list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (buf[i]) { list_del(&buf[i]->ready_list); job->buffers[i] =3D buf[i]; @@ -545,7 +673,7 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) } =20 /* Convert buffers to DMA addresses for the hardware */ - pispbe_xlate_addrs(pispbe, job, buf); + pispbe_xlate_addrs(pispbe, job, mdev_context, buf); =20 scoped_guard(spinlock_irq, &pispbe->hw_lock) { list_add_tail(&job->queue, &pispbe->job_queue); @@ -558,19 +686,25 @@ static int pispbe_prepare_job(struct pispbe_dev *pisp= be) =20 err_return_buffers: for (unsigned int i =3D 0; i < PISPBE_NUM_NODES; i++) { - struct pispbe_node *n =3D &pispbe->node[i]; - if (!buf[i]) continue; =20 /* Return the buffer to the ready_list queue */ - list_add(&buf[i]->ready_list, &n->ready_queue); + ctx =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx) + continue; + + list_add(&buf[i]->ready_list, &ctx->ready_queue); + pispbe_put_dev_context(ctx); } =20 return -ENODEV; } =20 -static void pispbe_schedule(struct pispbe_dev *pispbe, bool clear_hw_busy) +static void pispbe_schedule(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, + bool clear_hw_busy) { struct pispbe_job_descriptor *job; =20 @@ -591,9 +725,11 @@ static void pispbe_schedule(struct pispbe_dev *pispbe,= bool clear_hw_busy) =20 for (unsigned int i =3D 0; i < PISPBE_NUM_NODES; i++) pispbe->queued_job.buf[i] =3D job->buffers[i]; - pispbe->queued_job.valid =3D true; =20 pispbe->hw_busy =3D true; + + pispbe->queued_job.context =3D job->context; + pispbe->queued_job.valid =3D true; } =20 /* @@ -615,13 +751,13 @@ static void pispbe_isr_jobdone(struct pispbe_dev *pis= pbe, for (unsigned int i =3D 0; i < PISPBE_NUM_NODES; i++) { if (buf[i]) { buf[i]->vb.vb2_buf.timestamp =3D ts; - buf[i]->vb.sequence =3D pispbe->sequence; + buf[i]->vb.sequence =3D job->context->sequence; vb2_buffer_done(&buf[i]->vb.vb2_buf, VB2_BUF_STATE_DONE); } } =20 - pispbe->sequence++; + job->context->sequence++; } =20 static irqreturn_t pispbe_isr(int irq, void *dev) @@ -674,18 +810,20 @@ static irqreturn_t pispbe_isr(int irq, void *dev) } =20 /* check if there's more to do before going to sleep */ - pispbe_schedule(pispbe, can_queue_another); + pispbe_schedule(pispbe, NULL, can_queue_another); =20 return IRQ_HANDLED; } =20 static int pisp_be_validate_config(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, struct pisp_be_tiles_config *config) { u32 bayer_enables =3D config->config.global.bayer_enables; u32 rgb_enables =3D config->config.global.rgb_enables; + struct pispbe_node_context *context; + struct v4l2_pix_format_mplane *mp; struct device *dev =3D pispbe->dev; - struct v4l2_format *fmt; unsigned int bpl, size; =20 if (!(bayer_enables & PISP_BE_BAYER_ENABLE_INPUT) =3D=3D @@ -702,36 +840,50 @@ static int pisp_be_validate_config(struct pispbe_dev = *pispbe, } =20 /* Ensure output config strides and buffer sizes match the V4L2 formats. = */ - fmt =3D &pispbe->node[TDN_OUTPUT_NODE].format; + context =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[TDN_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp =3D &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_TDN_OUTPUT) { bpl =3D config->config.tdn_output_format.stride; size =3D bpl * config->config.tdn_output_format.height; =20 - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on tdn_output\n", __func__); return -EINVAL; } =20 - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on tdn_output\n", __func__); return -EINVAL; } } =20 - fmt =3D &pispbe->node[STITCH_OUTPUT_NODE].format; + context =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[STITCH_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp =3D &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_STITCH_OUTPUT) { bpl =3D config->config.stitch_output_format.stride; size =3D bpl * config->config.stitch_output_format.height; =20 - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on stitch_output\n", __func__); return -EINVAL; } =20 - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on stitch_output\n", __func__); return -EINVAL; @@ -746,8 +898,15 @@ static int pisp_be_validate_config(struct pispbe_dev *= pispbe, PISP_IMAGE_FORMAT_WALLPAPER_ROLL) continue; /* TODO: Size checks for wallpaper formats */ =20 - fmt =3D &pispbe->node[OUTPUT0_NODE + j].format; - for (unsigned int i =3D 0; i < fmt->fmt.pix_mp.num_planes; i++) { + context =3D pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + j].vfd); + if (!context) + return -EINVAL; + + mp =3D &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + + for (unsigned int i =3D 0; i < mp->num_planes; i++) { bpl =3D !i ? config->config.output_format[j].image.stride : config->config.output_format[j].image.stride2; size =3D bpl * config->config.output_format[j].image.height; @@ -756,13 +915,15 @@ static int pisp_be_validate_config(struct pispbe_dev = *pispbe, PISP_IMAGE_FORMAT_SAMPLING_420) size >>=3D 1; =20 - if (fmt->fmt.pix_mp.plane_fmt[i].bytesperline < bpl) { + if (mp->plane_fmt[i].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on output %d\n", __func__, j); + dev_dbg(dev, "Expected %u, got %u\n", + bpl, mp->plane_fmt[i].bytesperline); return -EINVAL; } =20 - if (fmt->fmt.pix_mp.plane_fmt[i].sizeimage < size) { + if (mp->plane_fmt[i].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on output\n", __func__); return -EINVAL; @@ -777,10 +938,12 @@ static int pispbe_node_queue_setup(struct vb2_queue *= q, unsigned int *nbuffers, unsigned int *nplanes, unsigned int sizes[], struct device *alloc_devs[]) { + struct pispbe_node_context *context =3D pispbe_node_context_from_queue(q); + struct v4l2_format *format =3D &context->format; struct pispbe_node *node =3D vb2_get_drv_priv(q); struct pispbe_dev *pispbe =3D node->pispbe; unsigned int num_planes =3D NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; =20 if (*nplanes) { if (*nplanes !=3D num_planes) @@ -788,8 +951,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q,= unsigned int *nbuffers, =20 for (unsigned int i =3D 0; i < *nplanes; i++) { unsigned int size =3D NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; =20 if (sizes[i] < size) return -EINVAL; @@ -801,8 +964,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q,= unsigned int *nbuffers, *nplanes =3D num_planes; for (unsigned int i =3D 0; i < *nplanes; i++) { unsigned int size =3D NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; sizes[i] =3D size; } =20 @@ -815,15 +978,20 @@ static int pispbe_node_queue_setup(struct vb2_queue *= q, unsigned int *nbuffers, =20 static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) { - struct pispbe_node *node =3D vb2_get_drv_priv(vb->vb2_queue); + struct vb2_queue *q =3D vb->vb2_queue; + struct pispbe_node_context *context =3D pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context =3D + pispbe_media_context_from_dev(context); + struct v4l2_format *format =3D &context->format; + struct pispbe_node *node =3D context->node; struct pispbe_dev *pispbe =3D node->pispbe; unsigned int num_planes =3D NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; =20 for (unsigned int i =3D 0; i < num_planes; i++) { unsigned long size =3D NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; =20 if (vb2_plane_size(vb, i) < size) { dev_dbg(pispbe->dev, @@ -836,12 +1004,12 @@ static int pispbe_node_buffer_prepare(struct vb2_buf= fer *vb) } =20 if (node->id =3D=3D CONFIG_NODE) { - void *dst =3D &node->pispbe->config[vb->index]; + void *dst =3D &mdev_context->config[vb->index]; void *src =3D vb2_plane_vaddr(vb, 0); =20 memcpy(dst, src, sizeof(struct pisp_be_tiles_config)); =20 - return pisp_be_validate_config(pispbe, dst); + return pisp_be_validate_config(pispbe, mdev_context, dst); } =20 return 0; @@ -849,27 +1017,34 @@ static int pispbe_node_buffer_prepare(struct vb2_buf= fer *vb) =20 static void pispbe_node_buffer_queue(struct vb2_buffer *buf) { - struct vb2_v4l2_buffer *vbuf =3D - container_of(buf, struct vb2_v4l2_buffer, vb2_buf); - struct pispbe_buffer *buffer =3D - container_of(vbuf, struct pispbe_buffer, vb); - struct pispbe_node *node =3D vb2_get_drv_priv(buf->vb2_queue); + struct vb2_v4l2_buffer *vbuf =3D container_of(buf, struct vb2_v4l2_buffer, + vb2_buf); + struct pispbe_buffer *buffer =3D container_of(vbuf, struct pispbe_buffer, + vb); + struct vb2_queue *q =3D buf->vb2_queue; + struct pispbe_node_context *context =3D pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context =3D + pispbe_media_context_from_dev(context); + struct pispbe_node *node =3D context->node; struct pispbe_dev *pispbe =3D node->pispbe; =20 dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); - list_add_tail(&buffer->ready_list, &node->ready_queue); + list_add_tail(&buffer->ready_list, &context->ready_queue); =20 /* * Every time we add a buffer, check if there's now some work for the hw * to do. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); } =20 static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int c= ount) { - struct pispbe_node *node =3D vb2_get_drv_priv(q); + struct pispbe_node_context *context =3D pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context =3D + pispbe_media_context_from_dev(context); + struct pispbe_node *node =3D context->node; struct pispbe_dev *pispbe =3D node->pispbe; struct pispbe_buffer *buf, *tmp; int ret; @@ -878,28 +1053,28 @@ static int pispbe_node_start_streaming(struct vb2_qu= eue *q, unsigned int count) if (ret < 0) goto err_return_buffers; =20 - ret =3D video_device_pipeline_alloc_start(&node->vfd); + ret =3D video_device_context_pipeline_alloc_start(&context->vdev_context); if (ret) goto err_return_buffers; =20 scoped_guard(spinlock_irq, &pispbe->hw_lock) { - node->pispbe->streaming_map |=3D BIT(node->id); - node->pispbe->sequence =3D 0; + mdev_context->streaming_map |=3D BIT(node->id); + mdev_context->sequence =3D 0; } =20 dev_dbg(pispbe->dev, "%s: for node %s (count %u)\n", __func__, NODE_NAME(node), count); dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - node->pispbe->streaming_map); + mdev_context->streaming_map); =20 /* Maybe we're ready to run. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); =20 return 0; =20 err_return_buffers: - list_for_each_entry_safe(buf, tmp, &node->ready_queue, ready_list) { + list_for_each_entry_safe(buf, tmp, &context->ready_queue, ready_list) { list_del(&buf->ready_list); vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); } @@ -909,7 +1084,10 @@ static int pispbe_node_start_streaming(struct vb2_que= ue *q, unsigned int count) =20 static void pispbe_node_stop_streaming(struct vb2_queue *q) { - struct pispbe_node *node =3D vb2_get_drv_priv(q); + struct pispbe_node_context *context =3D pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context =3D + pispbe_media_context_from_dev(context); + struct pispbe_node *node =3D context->node; struct pispbe_dev *pispbe =3D node->pispbe; struct pispbe_job_descriptor *job, *temp; struct pispbe_buffer *buf; @@ -926,7 +1104,7 @@ static void pispbe_node_stop_streaming(struct vb2_queu= e *q) */ dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); do { - buf =3D list_first_entry_or_null(&node->ready_queue, + buf =3D list_first_entry_or_null(&context->ready_queue, struct pispbe_buffer, ready_list); if (buf) { @@ -935,14 +1113,14 @@ static void pispbe_node_stop_streaming(struct vb2_qu= eue *q) } } while (buf); =20 - vb2_wait_for_all_buffers(&node->queue); + vb2_wait_for_all_buffers(&context->vdev_context.queue); =20 video_device_pipeline_stop(&node->vfd); =20 spin_lock_irq(&pispbe->hw_lock); - pispbe->streaming_map &=3D ~BIT(node->id); + mdev_context->streaming_map &=3D ~BIT(node->id); =20 - if (pispbe->streaming_map =3D=3D 0) { + if (mdev_context->streaming_map =3D=3D 0) { /* * If all nodes have stopped streaming release all jobs * without holding the lock. @@ -960,7 +1138,7 @@ static void pispbe_node_stop_streaming(struct vb2_queu= e *q) pm_runtime_put_autosuspend(pispbe->dev); =20 dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - pispbe->streaming_map); + mdev_context->streaming_map); } =20 static const struct vb2_ops pispbe_node_queue_ops =3D { @@ -1001,6 +1179,8 @@ static int pispbe_node_g_fmt_vid_cap(struct file *fil= e, void *priv, { struct pispbe_node *node =3D video_drvdata(file); struct pispbe_dev *pispbe =3D node->pispbe; + struct pispbe_node_context *context =3D + pispbe_node_context_from_file(file, &node->vfd); =20 if (!NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -1009,7 +1189,7 @@ static int pispbe_node_g_fmt_vid_cap(struct file *fil= e, void *priv, return -EINVAL; } =20 - *f =3D node->format; + *f =3D context->format; dev_dbg(pispbe->dev, "Get capture format for node %s\n", NODE_NAME(node)); =20 @@ -1021,6 +1201,8 @@ static int pispbe_node_g_fmt_vid_out(struct file *fil= e, void *priv, { struct pispbe_node *node =3D video_drvdata(file); struct pispbe_dev *pispbe =3D node->pispbe; + struct pispbe_node_context *context =3D + pispbe_node_context_from_file(file, &node->vfd); =20 if (NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -1029,7 +1211,7 @@ static int pispbe_node_g_fmt_vid_out(struct file *fil= e, void *priv, return -EINVAL; } =20 - *f =3D node->format; + *f =3D context->format; dev_dbg(pispbe->dev, "Get output format for node %s\n", NODE_NAME(node)); =20 @@ -1041,6 +1223,8 @@ static int pispbe_node_g_fmt_meta_out(struct file *fi= le, void *priv, { struct pispbe_node *node =3D video_drvdata(file); struct pispbe_dev *pispbe =3D node->pispbe; + struct pispbe_node_context *context =3D + pispbe_node_context_from_file(file, &node->vfd); =20 if (!NODE_IS_META(node) || NODE_IS_CAPTURE(node)) { dev_dbg(pispbe->dev, @@ -1049,7 +1233,7 @@ static int pispbe_node_g_fmt_meta_out(struct file *fi= le, void *priv, return -EINVAL; } =20 - *f =3D node->format; + *f =3D context->format; dev_dbg(pispbe->dev, "Get output format for meta node %s\n", NODE_NAME(node)); =20 @@ -1219,17 +1403,19 @@ static int pispbe_node_s_fmt_vid_cap(struct file *f= ile, void *priv, { struct pispbe_node *node =3D video_drvdata(file); struct pispbe_dev *pispbe =3D node->pispbe; + struct pispbe_node_context *context =3D + pispbe_node_context_from_file(file, &node->vfd); int ret; =20 ret =3D pispbe_node_try_fmt_vid_cap(file, priv, f); if (ret < 0) return ret; =20 - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; =20 - node->format =3D *f; - node->pisp_format =3D pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format =3D *f; + context->pisp_format =3D pispbe_find_fmt(f->fmt.pix_mp.pixelformat); =20 dev_dbg(pispbe->dev, "Set capture format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1242,17 +1428,19 @@ static int pispbe_node_s_fmt_vid_out(struct file *f= ile, void *priv, { struct pispbe_node *node =3D video_drvdata(file); struct pispbe_dev *pispbe =3D node->pispbe; + struct pispbe_node_context *context =3D + pispbe_node_context_from_file(file, &node->vfd); int ret; =20 ret =3D pispbe_node_try_fmt_vid_out(file, priv, f); if (ret < 0) return ret; =20 - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; =20 - node->format =3D *f; - node->pisp_format =3D pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format =3D *f; + context->pisp_format =3D pispbe_find_fmt(f->fmt.pix_mp.pixelformat); =20 dev_dbg(pispbe->dev, "Set output format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1265,17 +1453,19 @@ static int pispbe_node_s_fmt_meta_out(struct file *= file, void *priv, { struct pispbe_node *node =3D video_drvdata(file); struct pispbe_dev *pispbe =3D node->pispbe; + struct pispbe_node_context *context =3D + pispbe_node_context_from_file(file, &node->vfd); int ret; =20 ret =3D pispbe_node_try_fmt_meta_out(file, priv, f); if (ret < 0) return ret; =20 - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; =20 - node->format =3D *f; - node->pisp_format =3D &meta_out_supported_formats[0]; + context->format =3D *f; + context->pisp_format =3D &meta_out_supported_formats[0]; =20 dev_dbg(pispbe->dev, "Set output format for meta node %s to %p4cc\n", NODE_NAME(node), &f->fmt.meta.dataformat); @@ -1287,8 +1477,10 @@ static int pispbe_node_enum_fmt(struct file *file, v= oid *priv, struct v4l2_fmtdesc *f) { struct pispbe_node *node =3D video_drvdata(file); + struct pispbe_node_context *context =3D + pispbe_node_context_from_file(file, &node->vfd); =20 - if (f->type !=3D node->queue.type) + if (f->type !=3D context->vdev_context.queue.type) return -EINVAL; =20 if (NODE_IS_META(node)) { @@ -1362,13 +1554,17 @@ static const struct v4l2_ioctl_ops pispbe_node_ioct= l_ops =3D { .vidioc_streamoff =3D vb2_ioctl_streamoff, }; =20 -static int pispbe_link_validate(struct media_link *link) +static int pispbe_link_validate(struct media_link *link, + struct media_device_context *mdev_context) { + struct pispbe_subdev_context *isp_context; + struct pispbe_node_context *node_context; const struct v4l2_mbus_framefmt *sd_fmt; struct v4l2_pix_format_mplane *pix_mp; struct v4l2_subdev_state *state; struct media_entity *vdev_ent; struct media_entity *sd_ent; + struct video_device *vdev; struct pispbe_node *node; struct v4l2_subdev *sd; =20 @@ -1380,8 +1576,12 @@ static int pispbe_link_validate(struct media_link *l= ink) vdev_ent =3D link->sink->entity; } =20 - node =3D container_of(media_entity_to_video_device(vdev_ent), - struct pispbe_node, vfd); + vdev =3D media_entity_to_video_device(vdev_ent); + node_context =3D pispbe_node_context(video_device_context_get(mdev_contex= t, + vdev)); + + node =3D node_context->node; + switch (node->id) { case TDN_INPUT_NODE: fallthrough; @@ -1395,10 +1595,13 @@ static int pispbe_link_validate(struct media_link *= link) /* Skip validation for these nodes. */ return 0; } - pix_mp =3D &node->format.fmt.pix_mp; + pix_mp =3D &node_context->format.fmt.pix_mp; =20 sd =3D media_entity_to_v4l2_subdev(sd_ent); - state =3D v4l2_subdev_get_unlocked_active_state(sd); + isp_context =3D pispbe_subdev_context(v4l2_subdev_context_get(mdev_contex= t, + sd)); + + state =3D v4l2_subdev_get_unlocked_active_state(&isp_context->sd_context); sd_fmt =3D v4l2_subdev_state_get_format(state, node->id); =20 /* Only check for sizes. */ @@ -1419,28 +1622,16 @@ static int pispbe_link_validate(struct media_link *= link) return 0; } =20 -static const struct media_entity_operations pispbe_node_entity_ops =3D { - .link_validate =3D pispbe_link_validate, -}; - -static const struct video_device pispbe_videodev =3D { - .name =3D PISPBE_NAME, - .vfl_dir =3D VFL_DIR_M2M, /* gets overwritten */ - .fops =3D &pispbe_fops, - .ioctl_ops =3D &pispbe_node_ioctl_ops, - .minor =3D -1, - .release =3D video_device_release_empty, -}; - -static void pispbe_node_def_fmt(struct pispbe_node *node) +static void pispbe_node_def_fmt(struct pispbe_node *node, + struct pispbe_node_context *context) { + struct v4l2_format *format =3D &context->format; + if (NODE_IS_META(node) && NODE_IS_OUTPUT(node)) { /* Config node */ - struct v4l2_format *f =3D &node->format; - - f->fmt.meta.dataformat =3D V4L2_META_FMT_RPI_BE_CFG; - f->fmt.meta.buffersize =3D sizeof(struct pisp_be_tiles_config); - f->type =3D node->buf_type; + format->fmt.meta.dataformat =3D V4L2_META_FMT_RPI_BE_CFG; + format->fmt.meta.buffersize =3D sizeof(struct pisp_be_tiles_config); + format->type =3D node->buf_type; } else { struct v4l2_format f =3D { .fmt.pix_mp.pixelformat =3D V4L2_PIX_FMT_YUV420, @@ -1449,36 +1640,39 @@ static void pispbe_node_def_fmt(struct pispbe_node = *node) .type =3D node->buf_type, }; pispbe_try_format(&f, node); - node->format =3D f; + *format =3D f; } =20 - node->pisp_format =3D pispbe_find_fmt(node->format.fmt.pix_mp.pixelformat= ); + context->pisp_format =3D pispbe_find_fmt(format->fmt.pix_mp.pixelformat); } =20 -/* - * Initialise a struct pispbe_node and register it as /dev/video - * to represent one of the PiSP Back End's input or output streams. - */ -static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +static int pispbe_alloc_node_context(struct media_entity *entity, + struct media_entity_context **ctx) { - bool output =3D NODE_DESC_IS_OUTPUT(&node_desc[id]); - struct pispbe_node *node =3D &pispbe->node[id]; - struct media_entity *entity =3D &node->vfd.entity; - struct video_device *vdev =3D &node->vfd; - struct vb2_queue *q =3D &node->queue; + struct video_device *vdev =3D container_of(entity, struct video_device, + entity); + struct pispbe_node *node =3D pispbe_node_from_vdev(vdev); + struct pispbe_dev *pispbe =3D node->pispbe; + struct pispbe_node_context *context; + struct vb2_queue *q; int ret; =20 - node->id =3D id; - node->pispbe =3D pispbe; - node->buf_type =3D node_desc[id].buf_type; + *ctx =3D kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context =3D (struct pispbe_node_context *)*ctx; =20 - mutex_init(&node->node_lock); - mutex_init(&node->queue_lock); - INIT_LIST_HEAD(&node->ready_queue); + video_device_init_context(vdev, &context->vdev_context); + + context->pispbe =3D pispbe; + context->node =3D node; =20 - node->format.type =3D node->buf_type; - pispbe_node_def_fmt(node); + INIT_LIST_HEAD(&context->ready_queue); =20 + context->format.type =3D node->buf_type; + pispbe_node_def_fmt(node, context); + + q =3D &context->vdev_context.queue; q->type =3D node->buf_type; q->io_modes =3D VB2_MMAP | VB2_DMABUF; q->mem_ops =3D &vb2_dma_contig_memops; @@ -1488,21 +1682,70 @@ static int pispbe_init_node(struct pispbe_dev *pisp= be, unsigned int id) q->timestamp_flags =3D V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; q->dev =3D pispbe->dev; /* get V4L2 to handle node->queue locking */ - q->lock =3D &node->queue_lock; + q->lock =3D &context->vdev_context.queue_lock; =20 ret =3D vb2_queue_init(q); if (ret < 0) { dev_err(pispbe->dev, "vb2_queue_init failed\n"); - goto err_mutex_destroy; + goto err_cleanup; } =20 + return 0; + +err_cleanup: + kfree(*ctx); + return ret; +} + +static void pispbe_free_node_context(struct media_entity_context *ctx) +{ + struct pispbe_node_context *context =3D + pispbe_node_context_from_entity(ctx); + + list_del_init(&context->ready_queue); + video_device_cleanup_context(&context->vdev_context); + kfree(context); +} + +static const struct media_entity_operations pispbe_node_entity_ops =3D { + .alloc_context =3D pispbe_alloc_node_context, + .destroy_context =3D pispbe_free_node_context, + .link_validate_context =3D pispbe_link_validate, +}; + +static const struct video_device pispbe_videodev =3D { + .name =3D PISPBE_NAME, + .vfl_dir =3D VFL_DIR_M2M, /* gets overwritten */ + .fops =3D &pispbe_fops, + .ioctl_ops =3D &pispbe_node_ioctl_ops, + .minor =3D -1, + .release =3D video_device_release_empty, +}; + +/* + * Initialise a struct pispbe_node and register it as /dev/video + * to represent one of the PiSP Back End's input or output streams. + */ +static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +{ + bool output =3D NODE_DESC_IS_OUTPUT(&node_desc[id]); + struct pispbe_node *node =3D &pispbe->node[id]; + struct media_entity *entity =3D &node->vfd.entity; + struct video_device *vdev =3D &node->vfd; + int ret; + + node->id =3D id; + node->pispbe =3D pispbe; + node->buf_type =3D node_desc[id].buf_type; + + mutex_init(&node->node_lock); + *vdev =3D pispbe_videodev; /* default initialization */ strscpy(vdev->name, node_desc[id].ent_name, sizeof(vdev->name)); vdev->v4l2_dev =3D &pispbe->v4l2_dev; vdev->vfl_dir =3D output ? VFL_DIR_TX : VFL_DIR_RX; /* get V4L2 to serialise our ioctls */ vdev->lock =3D &node->node_lock; - vdev->queue =3D &node->queue; vdev->device_caps =3D V4L2_CAP_STREAMING | node_desc[id].caps; =20 node->pad.flags =3D output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK; @@ -1512,15 +1755,16 @@ static int pispbe_init_node(struct pispbe_dev *pisp= be, unsigned int id) dev_err(pispbe->dev, "Failed to register media pads for %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } =20 ret =3D video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) { dev_err(pispbe->dev, "Failed to register video %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } video_set_drvdata(vdev, node); =20 @@ -1542,14 +1786,15 @@ static int pispbe_init_node(struct pispbe_dev *pisp= be, unsigned int id) =20 err_unregister_video_dev: video_unregister_device(&node->vfd); -err_unregister_queue: - vb2_queue_release(&node->queue); err_mutex_destroy: mutex_destroy(&node->node_lock); - mutex_destroy(&node->queue_lock); return ret; } =20 +/* -----------------------------------------------------------------------= ----- + * ISP subdevice operations + */ + static int pispbe_subdev_set_pad_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state, struct v4l2_subdev_format *format) @@ -1623,8 +1868,83 @@ static const struct v4l2_subdev_internal_ops pispbe_= subdev_internal_ops =3D { .init_state =3D pispbe_init_state, }; =20 +static int pispbe_alloc_subdev_context(struct media_entity *entity, + struct media_entity_context **ctx) +{ + struct v4l2_subdev *sd =3D container_of(entity, struct v4l2_subdev, + entity); + struct pispbe_subdev_context *context; + + *ctx =3D kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context =3D (struct pispbe_subdev_context *)*ctx; + + v4l2_subdev_init_context(sd, &context->sd_context); + + return 0; +} + +static void pispbe_free_subdev_context(struct media_entity_context *ctx) +{ + struct pispbe_subdev_context *context =3D + pispbe_subdev_context_from_entity(ctx); + + v4l2_subdev_cleanup_context(&context->sd_context); + kfree(context); +} + static const struct media_entity_operations pispbe_subdev_entity_ops =3D { - .link_validate =3D v4l2_subdev_link_validate, + .alloc_context =3D pispbe_alloc_subdev_context, + .destroy_context =3D pispbe_free_subdev_context, + .link_validate_context =3D v4l2_subdev_link_validate_context, +}; + +static int pispbe_media_alloc_context(struct media_device *mdev, + struct media_device_context **ctx) +{ + struct pispbe_media_context *context; + + *ctx =3D kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context =3D (struct pispbe_media_context *)*ctx; + + media_device_init_context(mdev, &context->mdev_context); + + context->streaming_map =3D 0; + + context->config =3D + dma_alloc_coherent(mdev->dev, + sizeof(struct pisp_be_tiles_config) * + PISP_BE_NUM_CONFIG_BUFFERS, + &context->config_dma_addr, GFP_KERNEL); + if (!context->config) { + dev_err(mdev->dev, + "Unable to allocate cached config buffers.\n"); + return -ENOMEM; + } + + return 0; +} +static void pispbe_media_destroy_context(struct media_device_context *ctx) +{ + struct pispbe_media_context *context =3D + (struct pispbe_media_context *)ctx; + + if (context->config) + dma_free_coherent(ctx->mdev->dev, + sizeof(struct pisp_be_tiles_config) * + PISP_BE_NUM_CONFIG_BUFFERS, + context->config, context->config_dma_addr); + + media_device_cleanup_context(&context->mdev_context); + kfree(context); +} + +static const struct media_device_ops pispbe_media_device_ops =3D { + .alloc_context =3D pispbe_media_alloc_context, + .destroy_context =3D pispbe_media_destroy_context, }; =20 static int pispbe_init_subdev(struct pispbe_dev *pispbe) @@ -1676,7 +1996,9 @@ static int pispbe_init_devices(struct pispbe_dev *pis= pbe) /* Register v4l2_device and media_device */ mdev =3D &pispbe->mdev; mdev->hw_revision =3D pispbe->hw_version; + mdev->flags =3D MEDIA_DEVICE_FL_CONTEXT; mdev->dev =3D pispbe->dev; + mdev->ops =3D &pispbe_media_device_ops; strscpy(mdev->model, PISPBE_NAME, sizeof(mdev->model)); media_device_init(mdev); =20 @@ -1704,26 +2026,11 @@ static int pispbe_init_devices(struct pispbe_dev *p= ispbe) if (ret) goto err_unregister_nodes; =20 - pispbe->config =3D - dma_alloc_coherent(pispbe->dev, - sizeof(struct pisp_be_tiles_config) * - PISP_BE_NUM_CONFIG_BUFFERS, - &pispbe->config_dma_addr, GFP_KERNEL); - if (!pispbe->config) { - dev_err(pispbe->dev, "Unable to allocate cached config buffers.\n"); - ret =3D -ENOMEM; - goto err_unregister_mdev; - } - return 0; =20 -err_unregister_mdev: - media_device_unregister(mdev); err_unregister_nodes: - while (num_regist-- > 0) { + while (num_regist-- > 0) video_unregister_device(&pispbe->node[num_regist].vfd); - vb2_queue_release(&pispbe->node[num_regist].queue); - } v4l2_device_unregister_subdev(&pispbe->sd); media_entity_cleanup(&pispbe->sd.entity); err_unregister_v4l2: @@ -1735,14 +2042,6 @@ static int pispbe_init_devices(struct pispbe_dev *pi= spbe) =20 static void pispbe_destroy_devices(struct pispbe_dev *pispbe) { - if (pispbe->config) { - dma_free_coherent(pispbe->dev, - sizeof(struct pisp_be_tiles_config) * - PISP_BE_NUM_CONFIG_BUFFERS, - pispbe->config, - pispbe->config_dma_addr); - } - dev_dbg(pispbe->dev, "Unregister from media controller\n"); =20 v4l2_device_unregister_subdev(&pispbe->sd); @@ -1751,9 +2050,7 @@ static void pispbe_destroy_devices(struct pispbe_dev = *pispbe) =20 for (int i =3D PISPBE_NUM_NODES - 1; i >=3D 0; i--) { video_unregister_device(&pispbe->node[i].vfd); - vb2_queue_release(&pispbe->node[i].queue); mutex_destroy(&pispbe->node[i].node_lock); - mutex_destroy(&pispbe->node[i].queue_lock); } =20 media_device_cleanup(&pispbe->mdev); --=20 2.49.0