From nobody Sat Nov 15 14:52:42 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=9elements.com ARC-Seal: i=1; a=rsa-sha256; t=1752007728; cv=none; d=zohomail.com; s=zohoarc; b=nI5o9ZNxo+z/+jKfKaxmINfxANHz3vzXq+K5YXg4VyZs3VxDF64AxsdFa5WchXKDeG0q9SUDYzbF8SY+JE49xjzOpjlfc83tQH7dMMrSdSvvvDBeNxq3Ol3zAz3aDhEYeuxnKAaBSfb0P1b0ydgqC0Gxec/6VNq7/hT5JwF2fDQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752007728; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3P98FaovY9iwvLZaff233B7gZMlMWlgSqALv1uUW1KM=; b=f3ViPdcMrtfZs1wHCj3BM9cql2BSmsSVq0Xs82pGvjR78Rij99xl8sf9wER11/QKphZMqOKOIz+0H9Y6KWwYQxzEMDyP1uKcgYiFFtiatTOl01T+biGPqeRl0eMICBc4ks4hnwP9hmJmxkXNUIbSD7idmlMEU0I1Uo40a/eqFps= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1752007727801715.5309189659517; Tue, 8 Jul 2025 13:48:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uZFDT-00029R-2P; Tue, 08 Jul 2025 16:47:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uZDnI-0004Sx-9f for qemu-devel@nongnu.org; Tue, 08 Jul 2025 15:16:06 -0400 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uZDn4-00078h-K6 for qemu-devel@nongnu.org; Tue, 08 Jul 2025 15:15:52 -0400 Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-3a6f2c6715fso4235047f8f.1 for ; Tue, 08 Jul 2025 12:15:39 -0700 (PDT) Received: from cbox.sec.9e.network (p200300f75f10f341000000000000002d.dip0.t-ipconnect.de. [2003:f7:5f10:f341::2d]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454cd3d2749sm25445735e9.25.2025.07.08.07.58.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 07:58:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=9elements.com; s=google; t=1752002139; x=1752606939; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3P98FaovY9iwvLZaff233B7gZMlMWlgSqALv1uUW1KM=; b=J5bQQeR5sUIHXqljta0LZLpjDbnnEMW/c+DbZa/+KM12vmvmf9scLdqAjk0O6Z6G27 J2tikdjRQ3UvXVMPaugKFhnVmEFh1rXSUXrDpWeUT8rRhI/G9IJGjjPZcukPI7RgJmdP B6Gtp2v7FYKemzP3k6e4qbvMJ2ezmrCWtffM4aT9FgueaxIIH9GIEjFRLEZtwtc/kwd3 ZKg3n+S1WJnrrp5J2sXYO854IK5I5uvPxYzR9tA0xPcAECQ74myotnlvp2i5+CBwAB9A DnBqq1TDLJW7+jMu/Y5hfxoG35aeMxQy1SM0GIdxc+VshB2+tdfb/fzb41avFqxQIBmJ F2Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752002139; x=1752606939; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3P98FaovY9iwvLZaff233B7gZMlMWlgSqALv1uUW1KM=; b=GgSR9bLRKy+MD7lUetQfnoGQ2QpDU0uK6td8AfpYK/2dbH++lqgqqqXof0jd+XwIYZ OLDwbd7xK7fn29wnRryC/u1sdnpaX+xKm/zcONkU5VAcnZNd9fqStIVOFEDy+BmT/YWJ 2qoSkYCDrOv4UX5KWAS9xb6e2NXi1MMSGm71giUGJ0ObpJGdl3plcLyGCAki6iNfCrp2 qs0Bycf3KMMfINw+0BS45fuG9oDiLmLwakJ5OOwmIw5VEqt8IDh868tnByM3ijYm0D0X Q9LDN3ix2tIGbZESb1GqRJ3nS52AJmMFxH8aKvd7JolMHdrF+VByOh1/MaG3kEHk6hfS +piQ== X-Gm-Message-State: AOJu0Yzr/e/cogYjiVDx+76dWqqx18mhuTPrmS5UNOxlcW71YtWWW+SG qsg19tV8RfXrbEwic2or9xeGWbSH8ZqEGl8ZWgiryavxXywx9nFrMxtxz9FbD63hm3Qn2HYcJLT L/FlqPyN/0nnc X-Gm-Gg: ASbGncvq0XDSRE+Jycx7iC5369ThZb4edXMfKcW/1pcBFz+RFK8ms3vIWcLKk4K23bj 77xEra1m7qUoEJcp8H9+DIeVxTdJq1opkjgPzJMZEKfjAa3vIxrW47CLxGtTiOR03Dt26iNUqLV nBp3hEiHlxGOmS6SgrcwVktbzi6b2IgGBzc3J5E+rUB8YkCfapA7nE4AHKuyjO6sOHiBL7nC5pb MaYdICuhY0+G9rbyUqwOoI746fwHVEI+Kb0Z5ywF1Iw58TTkO1PDLWLmYaJxEVAeZR7PrOaBxqL XkX4f1hO1qFXz5LEJ23A0BFLvV6S7K+M0uTctTYCzSAP3duPcyVpf5FFlcp4PAEyv+3dCWCr18Z /VZVryzODfQ90JqtN/Cu3B+EzDtQ3cru6GWIA38IiKN8y2yncBiDOQasxpWpjx8Q= X-Google-Smtp-Source: AGHT+IHvtB6vYj6wFnyfQCYOxda0lPOaauw0FfzfauxKR2Sh68A1DlKq5h5R8xS+0gZxDP+XffSyzw== X-Received: by 2002:a05:600c:3b11:b0:43d:ac5:11e8 with SMTP id 5b1f17b1804b1-454b4ead65emr125189815e9.21.1751986713658; Tue, 08 Jul 2025 07:58:33 -0700 (PDT) From: David Milosevic X-Google-Original-From: David Milosevic To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Eduardo Habkost , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , pizhenwei@bytedance.com, marcello.bauer@9elements.com, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , David Milosevic Subject: [PATCH 2/3] video: add GStreamer backend Date: Tue, 8 Jul 2025 16:56:49 +0200 Message-ID: <20250708145828.63295-3-David.Milosevic@9elements.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250708145828.63295-1-David.Milosevic@9elements.com> References: <20250708145828.63295-1-David.Milosevic@9elements.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::430; envelope-from=david.milosevic@9elements.com; helo=mail-wr1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @9elements.com) X-ZM-MESSAGEID: 1752007730631116600 This follow-up extends the video subsystem by adding support for a GStreamer backend. There are some rules to follow when passing a GStreamer pipeline to QEMU: - pipeline must start with a source element (for example, v4l2src) - pipeline must NOT end with a sink element. QEMU will dynamically create and append an appsink to the provided cmdline pipeline In between, one can add an arbitrary number of converter elements to the cmdline pipeline. QEMU will also create a capsfilter right before its appsink, in order to have control over streaming parameters such as pixelformat, width, height, ... Hence, the final pipeline looks like this: <------------------- qemu-cmdline -------------------><--- qemu-runtime -= --> [source] -> [converter #1] -> ... -> [converter #n] -> capsfilter -> apps= ink Example usage: qemu-system-x86_64 \ -device qemu-xhci \ -videodev gstreamer,id=3Dcam0,pipeline=3D"v4l2src device=3D/dev/video0 = ! videoconvert" \ -device usb-video,videodev=3Dcam0 Pipelines like this are also possible: pipeline=3D"videotestsrc pattern=3Dsnow ! " \ "capsfilter caps=3Dvideo/x-raw^format=3DYUY2^width=3D1280^height= =3D720^framerate=3D30/1" Note the use of '^' in the GStreamer pipeline string. QEMU's option parser does not allow commas ',' inside option values, which makes it difficult to pass full GStreamer pipelines directly via the command line. To avoid modifying QEMU's core option parsing logic, we adopt a simple workaround: users are asked to substitute commas with '^' when specifying the pipeline. Signed-off-by: Marcello Sylvester Bauer Signed-off-by: David Milosevic --- meson.build | 11 + meson_options.txt | 2 + qemu-options.hx | 3 + scripts/meson-buildoptions.sh | 3 + video/gstreamer-common.h | 49 +++ video/gstreamer.c | 642 ++++++++++++++++++++++++++++++++++ video/meson.build | 2 + 7 files changed, 712 insertions(+) create mode 100644 video/gstreamer-common.h create mode 100644 video/gstreamer.c diff --git a/meson.build b/meson.build index 9eb6349182..4eb4a93d41 100644 --- a/meson.build +++ b/meson.build @@ -2313,6 +2313,15 @@ if not get_option('libdw').auto() or \ required: get_option('libdw')) endif =20 +gstreamer =3D not_found +gstreamer_app =3D not_found +if not get_option('gstreamer').auto() or (host_os =3D=3D 'linux' and have_= system) + gstreamer =3D dependency('gstreamer-1.0', required: get_option('gstreame= r'), + method: 'pkg-config') + gstreamer_app =3D dependency('gstreamer-app-1.0', required: get_option('= gstreamer'), + method: 'pkg-config') +endif + v4l2 =3D not_found if not get_option('v4l2').auto() or (host_os =3D=3D 'linux' and have_syste= m) v4l2 =3D declare_dependency() @@ -2368,6 +2377,7 @@ config_host_data.set('CONFIG_AUDIO_DRIVERS', =20 if have_system video_backend_available =3D { + 'gstreamer': gstreamer.found() and gstreamer_app.found(), 'v4l2': v4l2.found(), } foreach k, v: video_backend_available @@ -4949,6 +4959,7 @@ summary(summary_info, bool_yn: true, section: 'Networ= k backends') =20 # Video backends summary_info =3D {} +summary_info +=3D {'gstreamer support': gstreamer.found() and gstreame= r_app.found()} summary_info +=3D {'v4l2 support': v4l2} summary(summary_info, bool_yn: true, section: 'Video backends') =20 diff --git a/meson_options.txt b/meson_options.txt index ec2d4236f4..cd37affda2 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -383,5 +383,7 @@ option('rust', type: 'feature', value: 'disabled', option('strict_rust_lints', type: 'boolean', value: false, description: 'Enable stricter set of Rust warnings') =20 +option('gstreamer', type: 'feature', value: 'auto', + description: 'gstreamer video backend support') option('v4l2', type: 'feature', value: 'auto', description: 'v4l2 video backend support') diff --git a/qemu-options.hx b/qemu-options.hx index 4e2cf31d88..f1a82f052d 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -1089,6 +1089,9 @@ SRST ERST =20 DEF("videodev", HAS_ARG, QEMU_OPTION_videodev, +#ifdef CONFIG_VIDEO_GSTREAMER + "-videodev gstreamer,id=3Did,pipeline=3Dpipeline\n" +#endif #ifdef CONFIG_VIDEO_V4L2 "-videodev v4l2,id=3Did,device=3Dpath\n" #endif diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index b7ffa51921..80b530bf01 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -123,6 +123,7 @@ meson_options_help() { printf "%s\n" ' gio use libgio for D-Bus support' printf "%s\n" ' glusterfs Glusterfs block device driver' printf "%s\n" ' gnutls GNUTLS cryptography support' + printf "%s\n" ' gstreamer gstreamer video backend support' printf "%s\n" ' gtk GTK+ user interface' printf "%s\n" ' gtk-clipboard clipboard support for the gtk UI (EXPER= IMENTAL, MAY HANG)' printf "%s\n" ' guest-agent Build QEMU Guest Agent' @@ -330,6 +331,8 @@ _meson_option_parse() { --disable-glusterfs) printf "%s" -Dglusterfs=3Ddisabled ;; --enable-gnutls) printf "%s" -Dgnutls=3Denabled ;; --disable-gnutls) printf "%s" -Dgnutls=3Ddisabled ;; + --enable-gstreamer) printf "%s" -Dgstreamer=3Denabled ;; + --disable-gstreamer) printf "%s" -Dgstreamer=3Ddisabled ;; --enable-gtk) printf "%s" -Dgtk=3Denabled ;; --disable-gtk) printf "%s" -Dgtk=3Ddisabled ;; --enable-gtk-clipboard) printf "%s" -Dgtk_clipboard=3Denabled ;; diff --git a/video/gstreamer-common.h b/video/gstreamer-common.h new file mode 100644 index 0000000000..c90b223f54 --- /dev/null +++ b/video/gstreamer-common.h @@ -0,0 +1,49 @@ +/* + * Copyright 2025 9elements GmbH + * + * Authors: + * David Milosevic + * Marcello Sylvester Bauer + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_VIDEO_GSTREAMER_COMMON_H +#define QEMU_VIDEO_GSTREAMER_COMMON_H + +#define TYPE_VIDEODEV_GSTREAMER TYPE_VIDEODEV"-gstreamer" + +#include +#include + +/* + * GStreamer pipeline: + * + * <------------------- qemu-cmdline -------------------><--- qemu-runtime= ---> + * [source] -> [converter #1] -> ... -> [converter #n] -> capsfilter -> ap= psink + */ +struct GStreamerVideodev { + + Videodev parent; + + GstElement *pipeline; // gstreamer pipeline + GstElement *head; // first element of pipeline (source) + GstElement *tail; // last element of cmdline pipeline + GstElement *filter; // dynamically generated capsfilter + GstElement *sink; // dynamnically generated appsink + + struct GStreamerVideoFrame { + GstSample *sample; + GstBuffer *buffer; + GstMapInfo map_info; + } current_frame; +}; +typedef struct GStreamerVideodev GStreamerVideodev; + +DECLARE_INSTANCE_CHECKER(GStreamerVideodev, GSTREAMER_VIDEODEV, TYPE_VIDEO= DEV_GSTREAMER) + +void video_gstreamer_class_init(ObjectClass *oc, const void *data); +char *video_gstreamer_qemu_opt_get(QemuOpts *opts, const char *name); + +#endif /* QEMU_VIDEO_GSTREAMER_COMMON_H */ diff --git a/video/gstreamer.c b/video/gstreamer.c new file mode 100644 index 0000000000..25576c5e39 --- /dev/null +++ b/video/gstreamer.c @@ -0,0 +1,642 @@ +/* + * Copyright 2025 9elements GmbH + * + * Authors: + * David Milosevic + * Marcello Sylvester Bauer + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qapi/qmp/qerror.h" +#include "qemu/option.h" +#include "video/video.h" +#include "video/gstreamer-common.h" + +typedef struct { + const char *format; + uint32_t fourcc; +} FormatFourCC; + +FormatFourCC formatFourCCMap[] =3D { + {"YUY2", QEMU_VIDEO_PIX_FMT_YUYV}, +}; + +typedef struct VideoGStreamerCtrl { + VideoControlType q; + const char *v; +} VideoGStreamerCtrl; + +static VideoGStreamerCtrl video_gstreamer_ctrl_table[] =3D { + { + .q =3D VideoControlTypeBrightness, + .v =3D "brightness" + }, + { + .q =3D VideoControlTypeContrast, + .v =3D "contrast" + }, + { + .q =3D VideoControlTypeHue, + .v =3D "hue" + }, + { + .q =3D VideoControlTypeSaturation, + .v =3D "saturation" + } +}; + +static const char *video_qemu_control_to_gstreamer(VideoControlType type) +{ + for (int i =3D 0; i < ARRAY_SIZE(video_gstreamer_ctrl_table); i++) { + + if (video_gstreamer_ctrl_table[i].q =3D=3D type) { + return video_gstreamer_ctrl_table[i].v; + } + } + + return NULL; +} + +static GstElement *video_gstreamer_pipeline_head(GstElement *tail) +{ + GstElement *current =3D tail; + + while (true) { + + GstPad *sink_pad, *peer_pad; + GstElement *prev; + + sink_pad =3D gst_element_get_static_pad(current, "sink"); + if (!sink_pad) { + /* no sink pad - source/head found */ + break; + } + + if (!gst_pad_is_linked(sink_pad)) { + /* unlinked sink pad - not a proper source */ + gst_object_unref(sink_pad); + return NULL; + } + + peer_pad =3D gst_pad_get_peer(sink_pad); + gst_object_unref(sink_pad); + if (!peer_pad) { + /* broken pipeline? */ + return NULL; + } + + prev =3D gst_pad_get_parent_element(peer_pad); + gst_object_unref(peer_pad); + if (!prev) { + /* broken pipeline? */ + return NULL; + } + + current =3D prev; + } + + return current; +} + +char *video_gstreamer_qemu_opt_get(QemuOpts *opts, const char *name) +{ + const char *qemu_opt =3D qemu_opt_get(opts, name); + + /* + * QEMU's option parser forbids ',' inside option values, + * making it hard to pass full GStreamer pipelines over the cmdline. + * + * Users replace ',' with '^' as a workaround. This function reverses + * that replacement to restore the original pipeline. + * + * Use it whenever you would expect ',' within your option value. + */ + + if (qemu_opt =3D=3D NULL) { + return NULL; + } + + char *sanitized_opt =3D g_strdup(qemu_opt); + g_strdelimit(sanitized_opt, "^", ','); + return sanitized_opt; +} + +static int video_gstreamer_open(Videodev *vd, QemuOpts *opts, Error **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + char *pipeline =3D video_gstreamer_qemu_opt_get(opts, "pipeline"); + GstStateChangeReturn ret; + GstPad *tail_src_pad; + GError *error =3D NULL; + + if (pipeline =3D=3D NULL) { + vd_error_setg(vd, errp, QERR_MISSING_PARAMETER, "pipeline"); + return VIDEODEV_RC_ERROR; + } + + if (!gst_is_initialized()) + gst_init(NULL, NULL); + + gv->pipeline =3D gst_parse_bin_from_description(pipeline, false, &erro= r); + g_free(pipeline); + if (error) { + vd_error_setg(vd, errp, "unable to parse pipeline: %s", error->mes= sage); + return VIDEODEV_RC_ERROR; + } + + tail_src_pad =3D gst_bin_find_unlinked_pad(GST_BIN(gv->pipeline), GST_= PAD_SRC); + if (!tail_src_pad) { + vd_error_setg(vd, errp, "pipeline has no unlinked src pad"); + return VIDEODEV_RC_ERROR; + } + + gv->tail =3D gst_pad_get_parent_element(tail_src_pad); + gst_object_unref(tail_src_pad); + if (!gv->tail) { + vd_error_setg(vd, errp, "failed to get pipeline's tail element"); + return VIDEODEV_RC_ERROR; + } + + gv->head =3D video_gstreamer_pipeline_head(gv->tail); + if (!gv->head) { + vd_error_setg(vd, errp, "failed to get pipeline's head element"); + return VIDEODEV_RC_ERROR; + } + + gv->filter =3D gst_element_factory_make("capsfilter", "filter"); + if (!gv->filter) { + vd_error_setg(vd, errp, "failed to create capsfilter"); + return VIDEODEV_RC_ERROR; + } + + gst_bin_add(GST_BIN(gv->pipeline), gv->filter); + + if (!gst_element_link(gv->tail, gv->filter)) { + vd_error_setg(vd, errp, "failed to link pipeline to capsfilter"); + return VIDEODEV_RC_ERROR; + } + + gv->sink =3D gst_element_factory_make("appsink", "sink"); + if (!gv->sink) { + vd_error_setg(vd, errp, "failed to create appsink"); + return VIDEODEV_RC_ERROR; + } + + gst_bin_add(GST_BIN(gv->pipeline), gv->sink); + + if (!gst_element_link(gv->filter, gv->sink)) { + vd_error_setg(vd, errp, "failed to link pipeline to appsink"); + return VIDEODEV_RC_ERROR; + } + + ret =3D gst_element_set_state(gv->pipeline, GST_STATE_READY); + if (ret =3D=3D GST_STATE_CHANGE_FAILURE) { + + vd_error_setg(vd, errp, "failed to set pipeline to READY"); + return VIDEODEV_RC_ERROR; + } + + return VIDEODEV_RC_OK; +} + +static uint32_t gst_format_to_fourcc(const char *format) +{ + if (!format) { + return 0; + } + + for (int i =3D 0; i < ARRAY_SIZE(formatFourCCMap); i++) { + + if (!strcmp(formatFourCCMap[i].format, format)) { + return formatFourCCMap[i].fourcc; + } + } + + return 0; +} + +static const char *gst_fourcc_to_format(const uint32_t fourcc) { + + for (int i =3D 0; i < ARRAY_SIZE(formatFourCCMap); i++) { + + if (formatFourCCMap[i].fourcc =3D=3D fourcc) { + return formatFourCCMap[i].format; + } + } + + return NULL; +} + +static int video_gstreamer_enum_modes(Videodev *vd, Error **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + GstPad *tail_src_pad =3D NULL; + GstCaps *tail_src_caps =3D NULL; + const GstStructure *s; + uint32_t pixelformat; + + VideoMode *mode; + VideoFramesize *frmsz; + VideoFramerate *frmival; + + int i, j; + const gchar *name, *format; + const GValue *width_val, *height_val, *framerates; + + tail_src_pad =3D gst_element_get_static_pad(gv->tail, "src"); + if (!tail_src_pad) { + vd_error_setg(vd, errp, "failed to get src pad"); + return VIDEODEV_RC_ERROR; + } + + tail_src_caps =3D gst_pad_query_caps(tail_src_pad, NULL); + if (!tail_src_caps) { + vd_error_setg(vd, errp, "failed to get capabilities from src pad"); + return VIDEODEV_RC_ERROR; + } + + for (i =3D 0; i < gst_caps_get_size(tail_src_caps); i++) { + s =3D gst_caps_get_structure(tail_src_caps, i); + + name =3D gst_structure_get_name(s); + if (strcmp(name, "video/x-raw") !=3D 0) + continue; + + format =3D gst_structure_get_string(s, "format"); + if (!format) + continue; + + pixelformat =3D gst_format_to_fourcc(format); + if (pixelformat =3D=3D 0) + continue; + + if (!gst_structure_has_field(s, "width") || + !gst_structure_has_field(s, "height") || + !gst_structure_has_field(s, "framerate")) + continue; + + width_val =3D gst_structure_get_value(s, "width"); + height_val =3D gst_structure_get_value(s, "height"); + framerates =3D gst_structure_get_value(s, "framerate"); + + if (GST_VALUE_HOLDS_INT_RANGE(width_val) || + GST_VALUE_HOLDS_INT_RANGE(height_val)) + continue; + + // Collect all width values + GArray *widths =3D g_array_new(FALSE, FALSE, sizeof(int)); + if (G_VALUE_TYPE(width_val) =3D=3D G_TYPE_INT) { + int w =3D g_value_get_int(width_val); + g_array_append_val(widths, w); + } else if (GST_VALUE_HOLDS_LIST(width_val)) { + for (j =3D 0; j < gst_value_list_get_size(width_val); j++) { + const GValue *v =3D gst_value_list_get_value(width_val, j); + if (G_VALUE_TYPE(v) =3D=3D G_TYPE_INT) { + int w =3D g_value_get_int(v); + g_array_append_val(widths, w); + } + } + } else { + g_array_free(widths, TRUE); + continue; + } + + // Collect all height values + GArray *heights =3D g_array_new(FALSE, FALSE, sizeof(int)); + if (G_VALUE_TYPE(height_val) =3D=3D G_TYPE_INT) { + int h =3D g_value_get_int(height_val); + g_array_append_val(heights, h); + } else if (GST_VALUE_HOLDS_LIST(height_val)) { + for (j =3D 0; j < gst_value_list_get_size(height_val); j++) { + const GValue *v =3D gst_value_list_get_value(height_val, j= ); + if (G_VALUE_TYPE(v) =3D=3D G_TYPE_INT) { + int h =3D g_value_get_int(v); + g_array_append_val(heights, h); + } + } + } else { + g_array_free(widths, TRUE); + g_array_free(heights, TRUE); + continue; + } + + // Iterate over all width =C3=97 height combinations + for (int wi =3D 0; wi < widths->len; wi++) { + for (int hi =3D 0; hi < heights->len; hi++) { + int w =3D g_array_index(widths, int, wi); + int h =3D g_array_index(heights, int, hi); + + // Find or create VideoMode for this pixelformat + mode =3D NULL; + for (j =3D 0; j < vd->nmodes; j++) { + if (vd->modes[j].pixelformat =3D=3D pixelformat) { + mode =3D &vd->modes[j]; + break; + } + } + + if (!mode) { + vd->nmodes++; + vd->modes =3D g_realloc(vd->modes, vd->nmodes * sizeof= (VideoMode)); + mode =3D &vd->modes[vd->nmodes - 1]; + mode->pixelformat =3D pixelformat; + mode->framesizes =3D NULL; + mode->nframesize =3D 0; + } + + // Add new framesize + mode->nframesize++; + mode->framesizes =3D g_realloc(mode->framesizes, + mode->nframesize * sizeof(Vid= eoFramesize)); + frmsz =3D &mode->framesizes[mode->nframesize - 1]; + + frmsz->width =3D w; + frmsz->height =3D h; + frmsz->framerates =3D NULL; + frmsz->nframerate =3D 0; + + // Handle framerates (list or single fraction) + if (GST_VALUE_HOLDS_LIST(framerates)) { + for (j =3D 0; j < gst_value_list_get_size(framerates);= j++) { + const GValue *fval =3D gst_value_list_get_value(fr= amerates, j); + if (GST_VALUE_HOLDS_FRACTION(fval)) { + frmsz->nframerate++; + frmsz->framerates =3D g_realloc(frmsz->framera= tes, + frmsz->nframerat= e * sizeof(VideoFramerate)); + frmival =3D &frmsz->framerates[frmsz->nframera= te - 1]; + + // intentionally swapped + frmival->denominator =3D gst_value_get_fractio= n_numerator(fval); + frmival->numerator =3D gst_value_get_fraction_= denominator(fval); + } + } + } else if (GST_VALUE_HOLDS_FRACTION(framerates)) { + frmsz->nframerate++; + frmsz->framerates =3D g_realloc(frmsz->framerates, + frmsz->nframerate * size= of(VideoFramerate)); + frmival =3D &frmsz->framerates[frmsz->nframerate - 1]; + + // intentionally swapped + frmival->denominator =3D gst_value_get_fraction_numera= tor(framerates); + frmival->numerator =3D gst_value_get_fraction_denomina= tor(framerates); + } + } + } + + g_array_free(widths, TRUE); + g_array_free(heights, TRUE); + } + + if (vd->modes =3D=3D NULL) { + vd_error_setg(vd, errp, "failed to enumerate modes"); + return VIDEODEV_RC_ERROR; + } + + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_set_options(Videodev *vd, Error **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + const char *pixformat; + GstCaps *caps; + + if ((pixformat =3D gst_fourcc_to_format(vd->selected.mode->pixelformat= )) =3D=3D NULL) { + + vd_error_setg(vd, errp, "unsupported pixelformat"); + return VIDEODEV_RC_ERROR; + } + + caps =3D gst_caps_new_simple( + "video/x-raw", + "width", G_TYPE_INT, vd->selected.frmsz->width, + "height", G_TYPE_INT, vd->selected.frmsz->height, + "format", G_TYPE_STRING, pixformat, + "framerate", GST_TYPE_FRACTION, vd->selected.frmrt.denominator, + vd->selected.frmrt.numerator, NULL + ); + + if (caps =3D=3D NULL) { + + vd_error_setg(vd, errp, "failed to create new caps"); + return VIDEODEV_RC_ERROR; + } + + g_object_set(gv->filter, "caps", caps, NULL); + gst_caps_unref(caps); + + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_stream_on(Videodev *vd, Error **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + GstStateChangeReturn ret; + + if (gv->pipeline =3D=3D NULL) { + + vd_error_setg(vd, errp, "GStreamer pipeline not initialized!"); + return VIDEODEV_RC_ERROR; + } + + if (video_gstreamer_set_options(vd, errp) !=3D VIDEODEV_RC_OK) { + return VIDEODEV_RC_ERROR; + } + + ret =3D gst_element_set_state(gv->pipeline, GST_STATE_PLAYING); + + if (ret =3D=3D GST_STATE_CHANGE_FAILURE) { + + vd_error_setg(vd, errp, "failed to start GStreamer pipeline!"); + return VIDEODEV_RC_ERROR; + } + + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_stream_off(Videodev *vd, Error **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + GstStateChangeReturn ret; + + if (gv->pipeline =3D=3D NULL) { + + vd_error_setg(vd, errp, "GStreamer pipeline not initialized!"); + return VIDEODEV_RC_ERROR; + } + + ret =3D gst_element_set_state(gv->pipeline, GST_STATE_READY); + + if (ret =3D=3D GST_STATE_CHANGE_FAILURE) { + + vd_error_setg(vd, errp, "failed to stop GStreamer pipeline!"); + return VIDEODEV_RC_ERROR; + } + + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_claim_frame(Videodev *vd, Error **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + GstSample *sample; + GstBuffer *buffer; + + if ((sample =3D gst_app_sink_try_pull_sample(GST_APP_SINK(gv->sink), 0= )) =3D=3D NULL) { + + vd_error_setg(vd, errp, "appsink: underrun"); + return VIDEODEV_RC_UNDERRUN; + } + + if ((buffer =3D gst_sample_get_buffer(sample)) =3D=3D NULL) { + + gst_sample_unref(sample); + vd_error_setg(vd, errp, "could not retrieve sample buffer"); + return VIDEODEV_RC_ERROR; + } + + if (gst_buffer_map(buffer, &gv->current_frame.map_info, GST_MAP_READ) = !=3D TRUE) { + + gst_sample_unref(sample); + vd_error_setg(vd, errp, "could not map sample buffer"); + return VIDEODEV_RC_ERROR; + } + + gv->current_frame.sample =3D sample; + gv->current_frame.buffer =3D buffer; + vd->current_frame.data =3D (uint8_t*) gv->current_frame.map_info= .data; + vd->current_frame.bytes_left =3D gv->current_frame.map_info.size; + + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_release_frame(Videodev *vd, Error **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + + gst_buffer_unmap(gv->current_frame.buffer, &gv->current_frame.map_info= ); + gst_sample_unref(gv->current_frame.sample); + + gv->current_frame.sample =3D NULL; + gv->current_frame.buffer =3D NULL; + vd->current_frame.data =3D NULL; + vd->current_frame.bytes_left =3D 0; + + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_probe_control(Videodev *vd, VideoGStreamerCtrl = *ctrl, VideoControl *c) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + GParamSpec *pspec; + GParamSpecInt *ispec; + + /* + * Apparently there is no proper way to find out + * the real minimum and maximum of a video control. + * + * There is GParamSpec, but that one only gives us + * min and max of the underlying datatype. + * + * As a workaround, we could dynamically probe accepted + * values for a given control using g_object_get/g_object_set, + * but that might be an idea for the future. + */ + + pspec =3D g_object_class_find_property(G_OBJECT_GET_CLASS(gv->head), c= trl->v); + + if (pspec =3D=3D NULL) { + return VIDEODEV_RC_NOTSUP; + } + + ispec =3D G_PARAM_SPEC_INT(pspec); + + *c =3D (VideoControl) { + + .type =3D ctrl->q, + .min =3D ispec->minimum, + .max =3D ispec->maximum, + .step =3D 1 + }; + + g_object_get(G_OBJECT(gv->head), ctrl->v, &c->def, NULL); + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_enum_controls(Videodev *vd, Error **errp) +{ + for (int i =3D 0; i < ARRAY_SIZE(video_gstreamer_ctrl_table); i++) { + + VideoGStreamerCtrl *ctrl; + VideoControl ctrl_buffer; + + ctrl =3D &video_gstreamer_ctrl_table[i]; + + if (video_gstreamer_probe_control(vd, ctrl, &ctrl_buffer) !=3D VID= EODEV_RC_OK) { + continue; + } + + vd->ncontrols +=3D 1; + vd->controls =3D g_realloc(vd->controls, vd->ncontrols * sizeof(= VideoControl)); + + vd->controls[vd->ncontrols - 1] =3D ctrl_buffer; + } + + return VIDEODEV_RC_OK; +} + +static int video_gstreamer_set_control(Videodev *vd, VideoControl *ctrl, E= rror **errp) +{ + GStreamerVideodev *gv =3D GSTREAMER_VIDEODEV(vd); + const char *property; + int value; + + if ((property =3D video_qemu_control_to_gstreamer(ctrl->type)) =3D=3D = NULL) { + + vd_error_setg(vd, errp, "invalid control property!"); + return VIDEODEV_RC_INVAL; + } + + g_object_set(G_OBJECT(gv->head), property, ctrl->cur, NULL); + g_object_get(G_OBJECT(gv->head), property, &value, NULL); + + if (value !=3D ctrl->cur) { + + vd_error_setg(vd, errp, "could not apply new setting for '%s'", pr= operty); + return VIDEODEV_RC_INVAL; + } + + return VIDEODEV_RC_OK; +} + +void video_gstreamer_class_init(ObjectClass *oc, const void *data) +{ + VideodevClass *vc =3D VIDEODEV_CLASS(oc); + + vc->open =3D video_gstreamer_open; + vc->enum_modes =3D video_gstreamer_enum_modes; + vc->stream_on =3D video_gstreamer_stream_on; + vc->stream_off =3D video_gstreamer_stream_off; + vc->claim_frame =3D video_gstreamer_claim_frame; + vc->release_frame =3D video_gstreamer_release_frame; + vc->enum_controls =3D video_gstreamer_enum_controls; + vc->set_control =3D video_gstreamer_set_control; +} + +static const TypeInfo video_gstreamer_type_info =3D { + .name =3D TYPE_VIDEODEV_GSTREAMER, + .parent =3D TYPE_VIDEODEV, + .instance_size =3D sizeof(GStreamerVideodev), + .class_init =3D video_gstreamer_class_init, +}; + +static void register_types(void) +{ + type_register_static(&video_gstreamer_type_info); +} + +type_init(register_types); diff --git a/video/meson.build b/video/meson.build index 278ebdc389..33da556ea4 100644 --- a/video/meson.build +++ b/video/meson.build @@ -9,6 +9,8 @@ files =3D 2 =20 video_modules =3D {} foreach m : [ + ['gstreamer', gstreamer, files('gstreamer.c')], + ['gstreamer-app', gstreamer_app, files('gstreamer.c')], ['v4l2', v4l2, files('v4l2.c')], ] if m[dep].found() --=20 2.47.0