From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303212; cv=none; d=zohomail.com; s=zohoarc; b=GtvJEfRIHvQZQEHF9ItsP2JiaHlHGXu3Cgoeyb++ddwqcPFUZ7W/PKyc0wPe6sbDjpGqMVPmyWXvt/+1yAECXZgxESItOhPqc/Vw5tU4pUmkYGHNRlIiZEskTBmZGsNT6OAEaLx8t/6ldKEh4TNf+EPbAma+Y7lgZ0e6V2j3v8I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303212; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Z/ZAIf1bYmBCjSJaQWmGzOEGfik/bC9O2yzUeP6aW7w=; b=NZFMXLoSeVGjnKfsVQM434MfDMLa+StRurAXQW0IAcCzxqytcE7bWXrrCtQCC6OpffPHO4UsqcNkUIp5hREtBviqq4kwkZxXkPNNq1WLjZTiPCgm7YQKKOVRWG6dR0ipCIuoL27raKTGKNqk04uivlzERUANLChAl/+5BfO2XqQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303212480689.4648861067326; Thu, 20 Oct 2022 15:00:12 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-261-B3faKh78OD6rqDyhqhCUKA-1; Thu, 20 Oct 2022 17:59:19 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 78C70833A3F; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A385F40C6EC2; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1469E1946595; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3DD201946587 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 2FF074047AA; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0F1754047A7 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303211; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Z/ZAIf1bYmBCjSJaQWmGzOEGfik/bC9O2yzUeP6aW7w=; b=Zz6KZmabsLjE4OwWBGCsDkY/+X2u2jPsAAYfBWLudOOvvDN03aeGqCxLja2+0RjMRoP5zs vAo7ZIISgepfA+SjRTMTAKi8h9/sJNU6+LUNt4H5oTcYrbVvI76pRjVokcAf7uvhXazm+i BZDNkcfGSscrJAE1eisVxEHnCbrkhxI= X-MC-Unique: B3faKh78OD6rqDyhqhCUKA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 01/18] schema: allow 'ssh' as a protocol for network disks Date: Thu, 20 Oct 2022 16:58:52 -0500 Message-Id: <20221020215909.1751428-2-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303212840100001 Content-Type: text/plain; charset="utf-8"; x-default="true" There was support in the code for parsing protocol=3D'ssh' on network disk sources, but it was not present in the xml schema. Add this to the schema and mention it in the documentation. Signed-off-by: Jonathon Jongsma Reviewed-by: J=C3=A1n Tomko --- src/conf/schemas/domaincommon.rng | 1 + .../disk-network-ssh.x86_64-latest.args | 36 +++++++++++++++++++ tests/qemuxml2argvdata/disk-network-ssh.xml | 31 ++++++++++++++++ tests/qemuxml2argvtest.c | 1 + 4 files changed, 69 insertions(+) create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.a= rgs create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index d346442510..e2d5ff1379 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2144,6 +2144,7 @@ sheepdog tftp + ssh diff --git a/tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args b/t= ests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args new file mode 100644 index 0000000000..045474724b --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args @@ -0,0 +1,36 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"ssh","path":"test.img","server":{"host":"example.org= ","port":"2222"},"node-name":"libvirt-1-storage","auto-read-only":true,"dis= card":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-ssh.xml b/tests/qemuxml2ar= gvdata/disk-network-ssh.xml new file mode 100644 index 0000000000..355add4fea --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh.xml @@ -0,0 +1,31 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + + + + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 7ede68d555..ef32cae2e9 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1325,6 +1325,7 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-network-tlsx509-nbd-hostname"); DO_TEST_CAPS_VER("disk-network-tlsx509-vxhs", "5.0.0"); DO_TEST_CAPS_LATEST("disk-network-http"); + DO_TEST_CAPS_LATEST("disk-network-ssh"); driver.config->vxhsTLS =3D 0; VIR_FREE(driver.config->vxhsTLSx509certdir); DO_TEST_CAPS_LATEST("disk-no-boot"); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303161; cv=none; d=zohomail.com; s=zohoarc; b=QgEte5Zyg2JSxSPmv4UOcF7mDuuiGJCFxBkB+g6TaYRFFGOsLhEbGqO3Zz6Dnvuf/TRd68pbeVtUEsmuraVp8SyuZ0/IO24zf9BdZi9hzZ0PVhvoc4oi/GEo+d8hAViZE8gX6ByVgnOnp+t55aa7mUfpHA4/qbraMJpmmX5CIjw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303161; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jGjAbvMKXUCX75txxFY5mwZcpT4QciyxDeoAnONkkHg=; b=meEBVbGbMMlAFee76m2wGwGQLbk9Yhq6/4zlB8/JgeWic1nKEeOwykh/GS+GrqVGZwdk+O7ziFip4S/x8ChsGQrY39XkUCm5UceYd9y4iKXUEtLifC1PXman0oGYQoVwgFh3pqM9k+VUAyAsZPAygud1kZBlvlezrUH8eaSByao= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1666303161298145.3669413692146; Thu, 20 Oct 2022 14:59:21 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-54-nI-4D45AMcKM-qR5gPe0Xw-1; Thu, 20 Oct 2022 17:59:18 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6F781857D0D; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id C0C922024CBF; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6715E1946A40; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7A7531946587 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 6D25E4047AA; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 435734047A7 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303160; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=jGjAbvMKXUCX75txxFY5mwZcpT4QciyxDeoAnONkkHg=; b=I/Hl4ysyoLsa6zC0BrHu/UJ5PeWJK6eEG/6IpgIE19tCeToHGyMJ8yTol8TfiBoKk1ZTre yfiNZFabxA62+i1R636YVWh/K0aVXC12iURRJDeDJyFRTjbHa9OgvYizqbv2JJAwu8UVM/ A4ItYuU6PYJ9yWQrMCztm1Fb6tsy+Mw= X-MC-Unique: nI-4D45AMcKM-qR5gPe0Xw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 02/18] qemu: Add functions for determining nbdkit availability Date: Thu, 20 Oct 2022 16:58:53 -0500 Message-Id: <20221020215909.1751428-3-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303162411100001 Content-Type: text/plain; charset="utf-8"; x-default="true" In future commits, we will optionally use nbdkit to serve some remote disk sources. This patch queries to see whether nbdkit is installed on the host and queries it for capabilities. The data will be used in later commits. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- po/POTFILES | 1 + src/qemu/meson.build | 1 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_nbdkit.c | 203 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 52 +++++++++++ 5 files changed, 258 insertions(+) create mode 100644 src/qemu/qemu_nbdkit.c create mode 100644 src/qemu/qemu_nbdkit.h diff --git a/po/POTFILES b/po/POTFILES index 169e2a41dc..d96ce4415a 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -180,6 +180,7 @@ src/qemu/qemu_monitor.c src/qemu/qemu_monitor_json.c src/qemu/qemu_monitor_text.c src/qemu/qemu_namespace.c +src/qemu/qemu_nbdkit.c src/qemu/qemu_process.c src/qemu/qemu_qapi.c src/qemu/qemu_saveimage.c diff --git a/src/qemu/meson.build b/src/qemu/meson.build index 96952cc52d..101cf3591f 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -28,6 +28,7 @@ qemu_driver_sources =3D [ 'qemu_monitor_json.c', 'qemu_monitor_text.c', 'qemu_namespace.c', + 'qemu_nbdkit.c', 'qemu_process.c', 'qemu_qapi.c', 'qemu_saveimage.c', diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 8cf2dd2ec5..a39510b0b1 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -36,6 +36,7 @@ #include "virthreadpool.h" #include "locking/lock_manager.h" #include "qemu_capabilities.h" +#include "qemu_nbdkit.h" #include "virclosecallbacks.h" #include "virhostdev.h" #include "virfile.h" diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c new file mode 100644 index 0000000000..7a7248c1ef --- /dev/null +++ b/src/qemu/qemu_nbdkit.c @@ -0,0 +1,203 @@ +/* + * qemu_nbdkit.c: helpers for using nbdkit + * + * Copyright (C) 2021 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + */ + +#include +#include + +#include "vircommand.h" +#include "virerror.h" +#include "virlog.h" +#include "virpidfile.h" +#include "virutil.h" +#include "qemu_block.h" +#include "qemu_conf.h" +#include "qemu_domain.h" +#include "qemu_driver.h" +#include "qemu_extdevice.h" +#include "qemu_nbdkit.h" +#include "qemu_security.h" + +#include + +#define VIR_FROM_THIS VIR_FROM_QEMU + +VIR_LOG_INIT("qemu.nbdkit"); + +VIR_ENUM_IMPL(qemuNbdkitCaps, + QEMU_NBDKIT_CAPS_LAST, + /* 0 */ + "plugin-curl", /* QEMU_NBDKIT_CAPS_PLUGIN_CURL */ + "plugin-ssh", /* QEMU_NBDKIT_CAPS_PLUGIN_SSH */ + "filter-readahead", /* QEMU_NBDKIT_CAPS_FILTER_READAHEAD */ +); + +struct _qemuNbdkitCaps { + GObject parent; + + char *path; + char *version; + + virBitmap *flags; +}; +G_DEFINE_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, G_TYPE_OBJECT); + + +static void +qemuNbdkitCheckCommandCap(qemuNbdkitCaps *nbdkit, + virCommand *cmd, + qemuNbdkitCapsFlags cap) +{ + if (virCommandRun(cmd, NULL) !=3D 0) + return; + + VIR_DEBUG("Setting nbdkit capability %i", cap); + ignore_value(virBitmapSetBit(nbdkit->flags, cap)); +} + + +static void +qemuNbdkitQueryFilter(qemuNbdkitCaps *nbdkit, + const char *filter, + qemuNbdkitCapsFlags cap) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(nbdkit->path, + "--version", + NULL); + + virCommandAddArgPair(cmd, "--filter", filter); + + /* use null plugin to check for filter */ + virCommandAddArg(cmd, "null"); + + qemuNbdkitCheckCommandCap(nbdkit, cmd, cap); +} + + +static void +qemuNbdkitQueryPlugin(qemuNbdkitCaps *nbdkit, + const char *plugin, + qemuNbdkitCapsFlags cap) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(nbdkit->path, + plugin, + "--version", + NULL); + + qemuNbdkitCheckCommandCap(nbdkit, cmd, cap); +} + + +static void +qemuNbdkitCapsQueryPlugins(qemuNbdkitCaps *nbdkit) +{ + qemuNbdkitQueryPlugin(nbdkit, "curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + qemuNbdkitQueryPlugin(nbdkit, "ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); +} + + +static void +qemuNbdkitCapsQueryFilters(qemuNbdkitCaps *nbdkit) +{ + qemuNbdkitQueryFilter(nbdkit, "readahead", + QEMU_NBDKIT_CAPS_FILTER_READAHEAD); +} + + +static int +qemuNbdkitCapsQueryVersion(qemuNbdkitCaps *nbdkit) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(nbdkit->path, + "--version", + NULL); + + virCommandSetOutputBuffer(cmd, &nbdkit->version); + + if (virCommandRun(cmd, NULL) !=3D 0) + return -1; + + VIR_DEBUG("Got nbdkit version %s", nbdkit->version); + return 0; +} + + +static void +qemuNbdkitCapsFinalize(GObject *object) +{ + qemuNbdkitCaps *nbdkit =3D QEMU_NBDKIT_CAPS(object); + + g_clear_pointer(&nbdkit->path, g_free); + g_clear_pointer(&nbdkit->version, g_free); + g_clear_pointer(&nbdkit->flags, virBitmapFree); + + G_OBJECT_CLASS(qemu_nbdkit_caps_parent_class)->finalize(object); +} + + +void +qemu_nbdkit_caps_init(qemuNbdkitCaps *caps) +{ + caps->flags =3D virBitmapNew(QEMU_NBDKIT_CAPS_LAST); + caps->version =3D NULL; +} + + +static void +qemu_nbdkit_caps_class_init(qemuNbdkitCapsClass *klass) +{ + GObjectClass *obj =3D G_OBJECT_CLASS(klass); + + obj->finalize =3D qemuNbdkitCapsFinalize; +} + + +qemuNbdkitCaps * +qemuNbdkitCapsNew(const char *path) +{ + qemuNbdkitCaps *caps =3D g_object_new(QEMU_TYPE_NBDKIT_CAPS, NULL); + caps->path =3D g_strdup(path); + + return caps; +} + + +G_GNUC_UNUSED static void +qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) +{ + qemuNbdkitCapsQueryPlugins(caps); + qemuNbdkitCapsQueryFilters(caps); + qemuNbdkitCapsQueryVersion(caps); +} + + +bool +qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag) +{ + return virBitmapIsBitSet(nbdkitCaps->flags, flag); +} + + +void +qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag) +{ + ignore_value(virBitmapSetBit(nbdkitCaps->flags, flag)); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h new file mode 100644 index 0000000000..8ffb0f7044 --- /dev/null +++ b/src/qemu/qemu_nbdkit.h @@ -0,0 +1,52 @@ +/* + * qemu_nbdkit.h: helpers for using nbdkit + * + * Copyright (C) 2021 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + */ + +#pragma once + +#include "internal.h" +#include "virenum.h" + +typedef struct _qemuNbdkitCaps qemuNbdkitCaps; + +typedef enum { + /* 0 */ + QEMU_NBDKIT_CAPS_PLUGIN_CURL, + QEMU_NBDKIT_CAPS_PLUGIN_SSH, + QEMU_NBDKIT_CAPS_FILTER_READAHEAD, + + QEMU_NBDKIT_CAPS_LAST, +} qemuNbdkitCapsFlags; + +VIR_ENUM_DECL(qemuNbdkitCaps); + +qemuNbdkitCaps * +qemuNbdkitCapsNew(const char *path); + +bool +qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag); + +void +qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag); + +#define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() +G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, = GObject); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303170; cv=none; d=zohomail.com; s=zohoarc; b=XswO2Cuuzn9meO6zI2poSlmT7Jpf6TXl53lNA83FQ0AZjtVFG0q2lNnCeOU1+RfPmh1jiTxrSvR3vG/Y01cDg8008c29hL3VYQItzxP1X1rvJIjeajMZh5T30IIhUOdx1dU7C6NbzW6jqIj/9J+5pIMk7EnwxxNh6I3/WZEAtdU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303170; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=425IlyOXdQ9gpKDG502def87+RORjWsZ/wYn4U+jybQ=; b=R2PfPn4YyNb9E6tmR6s+7jllP5nQgytYFkcFz8cfhe9XCjVdmzVV+40FZhi5RABoNcVujI3pdjDTBpxnIbbeUeOqp7Ao1zSAdTr8U4mUN413iZRqzAZ4Xc/xkw51VcbMeuf0EE7d+xjORIhU1XQf74DnSLeqYtVzJmj82y7eGvA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1666303170968913.9232384191442; Thu, 20 Oct 2022 14:59:30 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-350-7VlEVsQoMyWMpPe-g6FbCw-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 97518857020; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 814CFC15BA5; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D74AF1946A46; Thu, 20 Oct 2022 21:59:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id ADC571946587 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id A1BD74047AA; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 80C864047A7 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303170; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=425IlyOXdQ9gpKDG502def87+RORjWsZ/wYn4U+jybQ=; b=PzKTl0y5f/MnlsaoGgNkJVFtxdjMdHAFKMj2dSwcfZfObG1F7IFd3Azbm+CGBq9ZzLJiq0 mYBDzvKkS1Zqf5dCZr/ILe2PEwggVJEq58SYGuhi6CQTgvxfVxtJel7D5Ktzqc1PJGllLt u/An2AmM3WQ2vAt8q/4s1Kxu2FW7UPI= X-MC-Unique: 7VlEVsQoMyWMpPe-g6FbCw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 03/18] qemu: expand nbdkit capabilities Date: Thu, 20 Oct 2022 16:58:54 -0500 Message-Id: <20221020215909.1751428-4-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303172365100006 Content-Type: text/plain; charset="utf-8"; x-default="true" In order to add caching of the nbdkit capabilities, we will need to compare against file modification times, etc. So look up this information when creating the nbdkit caps. Add a nbdkit_moddir build option to allow the builder to specify the location to look for nbdkit plugins and filters. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- meson.build | 6 ++++++ meson_options.txt | 1 + src/qemu/qemu_nbdkit.c | 40 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 47 insertions(+) diff --git a/meson.build b/meson.build index 93de355430..e4581e74dd 100644 --- a/meson.build +++ b/meson.build @@ -1731,6 +1731,12 @@ if not get_option('driver_qemu').disabled() qemu_dbus_daemon_path =3D '/usr/bin/dbus-daemon' endif conf.set_quoted('QEMU_DBUS_DAEMON', qemu_dbus_daemon_path) + + nbdkit_moddir =3D get_option('nbdkit_moddir') + if nbdkit_moddir =3D=3D '' + nbdkit_moddir =3D libdir / 'nbdkit' + endif + conf.set_quoted('NBDKIT_MODDIR', nbdkit_moddir) endif endif =20 diff --git a/meson_options.txt b/meson_options.txt index 861c5577d2..d5ea4376e0 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -71,6 +71,7 @@ option('driver_vbox', type: 'feature', value: 'auto', des= cription: 'VirtualBox X option('vbox_xpcomc_dir', type: 'string', value: '', description: 'Locatio= n of directory containing VirtualBox XPCOMC library') option('driver_vmware', type: 'feature', value: 'auto', description: 'VMwa= re driver') option('driver_vz', type: 'feature', value: 'auto', description: 'Virtuozz= o driver') +option('nbdkit_moddir', type: 'string', value: '', description: 'set the d= irectory where nbdkit modules are located') =20 option('secdriver_apparmor', type: 'feature', value: 'auto', description: = 'use AppArmor security driver') option('apparmor_profiles', type: 'feature', value: 'auto', description: '= install apparmor profiles') diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 7a7248c1ef..5de1021d89 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -41,6 +41,9 @@ =20 VIR_LOG_INIT("qemu.nbdkit"); =20 +#define NBDKIT_PLUGINDIR NBDKIT_MODDIR "/plugins" +#define NBDKIT_FILTERDIR NBDKIT_MODDIR "/filters" + VIR_ENUM_IMPL(qemuNbdkitCaps, QEMU_NBDKIT_CAPS_LAST, /* 0 */ @@ -54,6 +57,11 @@ struct _qemuNbdkitCaps { =20 char *path; char *version; + time_t ctime; + time_t libvirtCtime; + time_t pluginDirMtime; + time_t filterDirMtime; + unsigned int libvirtVersion; =20 virBitmap *flags; }; @@ -178,9 +186,41 @@ qemuNbdkitCapsNew(const char *path) } =20 =20 +static time_t +qemuNbdkitGetDirMtime(const char *moddir) +{ + struct stat st; + + if (stat(moddir, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit module directory '%s': %s", + moddir, + g_strerror(errno)); + return 0; + } + + return st.st_mtime; +} + + G_GNUC_UNUSED static void qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) { + struct stat st; + + if (stat(caps->path, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit binary '%s': %s", + caps->path, + g_strerror(errno)); + caps->ctime =3D 0; + return; + } + + caps->ctime =3D st.st_ctime; + caps->filterDirMtime =3D qemuNbdkitGetDirMtime(NBDKIT_FILTERDIR); + caps->pluginDirMtime =3D qemuNbdkitGetDirMtime(NBDKIT_PLUGINDIR); + caps->libvirtCtime =3D virGetSelfLastChanged(); + caps->libvirtVersion =3D LIBVIR_VERSION_NUMBER; + qemuNbdkitCapsQueryPlugins(caps); qemuNbdkitCapsQueryFilters(caps); qemuNbdkitCapsQueryVersion(caps); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303170; cv=none; d=zohomail.com; s=zohoarc; b=Mtt86aNHUSJBSskYbtuaqiRdMV6gVEZ6KL6Yqy8HuDHxH94Vlne/sCttdHNEU5o4JMQsv/EVRUySCTYNfcpiZ63kgli4QKANJV8wTTab1NeFQKdKXgMJXk5qux3xt5rmUNgHxAQ1dhl9st0DJcrsKxhoBDD5U+uM+Vr8m2duxNg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303170; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7iy4Aoz7pjN4QfwCqBz0LllyFiMcOL7M2RdJ5KSccms=; b=XulNTVjrYTw89+cJ3VbSbs9BIaFV4sTjCn22Bcbx08lviMHApWZwxOfDIHRxPCOwCmOMSCMxGpYvOStvKpWq8bMIz6y9RvCtWx6e0srA9VVyudZY53wCODjjXRE4k98eV6wkZOh3Tm3PorZxsUBMVceuSq0FL1k/pD5RYbB3PiM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303170077665.2427691859519; Thu, 20 Oct 2022 14:59:30 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-561-nYFY_P9DPTqY0cKulVWP5A-1; Thu, 20 Oct 2022 17:59:23 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AAA0B1C0CE92; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 98CDA2024CBB; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D98A31946597; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id DF3E21946587 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id D47D14047AA; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B41CB4047A7 for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303169; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=7iy4Aoz7pjN4QfwCqBz0LllyFiMcOL7M2RdJ5KSccms=; b=TmToxoNoEG41DXEnq6EYo4mNEIOo/lty5/UJEMdWYX+VpKPgjYYmofdKRgTJz18bJhqym2 MJdX3XQiCEjdBrUdn6C44QI4eX/ZwqwdZzpbQXCEDOP35jPuwZ1FEG1XP8OwMHdYm/pIJ3 oNj+N1Nlb37XvLigOcOTxt0jgSspALE= X-MC-Unique: nYFY_P9DPTqY0cKulVWP5A-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 04/18] util: Allow virFileCache data to be any GObject Date: Thu, 20 Oct 2022 16:58:55 -0500 Message-Id: <20221020215909.1751428-5-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303170601100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Since the libvirt documentation suggests to prefer GObject over virObject, and since virObject is a GObject, change virFileCache to allow GObjects as data. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/util/virfilecache.c | 14 ++++++++------ src/util/virfilecache.h | 2 +- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/src/util/virfilecache.c b/src/util/virfilecache.c index bad37c9f00..eaedf6db7e 100644 --- a/src/util/virfilecache.c +++ b/src/util/virfilecache.c @@ -170,7 +170,7 @@ virFileCacheLoad(virFileCache *cache, *data =3D g_steal_pointer(&loadData); =20 cleanup: - virObjectUnref(loadData); + g_clear_pointer(&loadData, g_object_unref); return ret; } =20 @@ -207,7 +207,7 @@ virFileCacheNewData(virFileCache *cache, return NULL; =20 if (virFileCacheSave(cache, name, data) < 0) { - g_clear_pointer(&data, virObjectUnref); + g_clear_object(&data); } } =20 @@ -239,7 +239,7 @@ virFileCacheNew(const char *dir, if (!(cache =3D virObjectNew(virFileCacheClass))) return NULL; =20 - cache->table =3D virHashNew(virObjectUnref); + cache->table =3D virHashNew(g_object_unref); =20 cache->dir =3D g_strdup(dir); =20 @@ -270,7 +270,7 @@ virFileCacheValidate(virFileCache *cache, if (*data) { VIR_DEBUG("Caching data '%p' for '%s'", *data, name); if (virHashAddEntry(cache->table, name, *data) < 0) { - g_clear_pointer(data, virObjectUnref); + g_clear_pointer(data, g_object_unref); } } } @@ -300,7 +300,8 @@ virFileCacheLookup(virFileCache *cache, data =3D virHashLookup(cache->table, name); virFileCacheValidate(cache, name, &data); =20 - virObjectRef(data); + if (data) + g_object_ref(data); virObjectUnlock(cache); =20 return data; @@ -331,7 +332,8 @@ virFileCacheLookupByFunc(virFileCache *cache, data =3D virHashSearch(cache->table, iter, iterData, &name); virFileCacheValidate(cache, name, &data); =20 - virObjectRef(data); + if (data) + g_object_ref(data); virObjectUnlock(cache); =20 return data; diff --git a/src/util/virfilecache.h b/src/util/virfilecache.h index 81be8feef5..f0d220cc86 100644 --- a/src/util/virfilecache.h +++ b/src/util/virfilecache.h @@ -48,7 +48,7 @@ typedef bool * @priv: private data created together with cache * * Creates a new data based on the @name. The returned data must be - * an instance of virObject. + * an instance of GObject. * * Returns data object or NULL on error. */ --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303217; cv=none; d=zohomail.com; s=zohoarc; b=RUPffXZbEj+0Y0CChMSFgwV6eNVX1MwnhHemXl4XM9eWcC5WOQ3aEa0jdilkWCwEHnl3DyDLT8oH20hjqqv5xzYe8vMGKMdDRjbmS6VbjuFba7DQxNurIALnQU4MGTA+JKNN7Kmf+IUIuPt6xWso3FWEkzEHb4j9KuZnbKD2puw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303217; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nJ2MLW4bMUOJsPG8UIIQ7ce4qE5Uu/NVY01Oa7L2I8M=; b=ZpwGnzRzpp6kkuZsCEOWAbu8//OKwD+fxVdzDemJSk6XSspg8trRLgtAU2Ddv0jDTEBZPRN55TSChuk1eCylLNpQC/yXcbnn8FtBwd1okhKnRNOlkI7TkpLisDSKXgKXp5+bwAsNKOYLEINuijHqrQR3vni7nEzm8H/SHOtG3aE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 166630321764282.79044589940634; Thu, 20 Oct 2022 15:00:17 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-595-wlALsRyXPyif3YAZeP45xw-1; Thu, 20 Oct 2022 17:59:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 906E33817A9A; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7868140CA41E; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 9AF721946A48; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3DECD19465B1 for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 143704047A7; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E77A24047AA for ; Thu, 20 Oct 2022 21:59:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303216; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=nJ2MLW4bMUOJsPG8UIIQ7ce4qE5Uu/NVY01Oa7L2I8M=; b=W+9vvrYhql/CkOyUXFgY4w321toIipBoBZO6dcqRHibACzWRDN34oCa65bounwocj68Q6t +AqkVlNuNyLCtmRX+NL1Dv51jkHiy8Gz/D6NO1wjOhwJhyBjhx4Pe57J06BsASJMqYOIfU fD49lhAOTWZ1We4lczTOxHtu2EN953I= X-MC-Unique: wlALsRyXPyif3YAZeP45xw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 05/18] qemu: implement basic virFileCache for nbdkit caps Date: Thu, 20 Oct 2022 16:58:56 -0500 Message-Id: <20221020215909.1751428-6-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303218735100011 Content-Type: text/plain; charset="utf-8"; x-default="true" Preparatory step for caching nbdkit capabilities. This patch implements the newData and isValid virFileCacheHandlers callback functions. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 93 +++++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_nbdkit.h | 4 ++ 2 files changed, 96 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 5de1021d89..9ab048e9e1 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -202,7 +202,7 @@ qemuNbdkitGetDirMtime(const char *moddir) } =20 =20 -G_GNUC_UNUSED static void +static void qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) { struct stat st; @@ -241,3 +241,94 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, { ignore_value(virBitmapSetBit(nbdkitCaps->flags, flag)); } + + +static bool +virNbkditCapsCheckModdir(const char *moddir, + time_t expectedMtime) +{ + time_t mtime =3D qemuNbdkitGetDirMtime(moddir); + + if (mtime !=3D expectedMtime) { + VIR_DEBUG("Outdated capabilities for nbdkit: module " + "directory '%s' changed (%lld vs %lld)", + moddir, + (long long)mtime, (long long)expectedMtime); + return false; + } + return true; +} + + +static bool +virNbdkitCapsIsValid(void *data, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *nbdkitCaps =3D data; + struct stat st; + + if (!nbdkitCaps->path) + return true; + + if (!virNbkditCapsCheckModdir(NBDKIT_PLUGINDIR, nbdkitCaps->pluginDirM= time)) + return false; + if (!virNbkditCapsCheckModdir(NBDKIT_FILTERDIR, nbdkitCaps->filterDirM= time)) + return false; + + if (nbdkitCaps->libvirtCtime !=3D virGetSelfLastChanged() || + nbdkitCaps->libvirtVersion !=3D LIBVIR_VERSION_NUMBER) { + VIR_DEBUG("Outdated capabilities for '%s': libvirt changed " + "(%lld vs %lld, %lu vs %lu)", + nbdkitCaps->path, + (long long)nbdkitCaps->libvirtCtime, + (long long)virGetSelfLastChanged(), + (unsigned long)nbdkitCaps->libvirtVersion, + (unsigned long)LIBVIR_VERSION_NUMBER); + return false; + } + + if (stat(nbdkitCaps->path, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit binary '%s': %s", + nbdkitCaps->path, + g_strerror(errno)); + return false; + } + + if (st.st_ctime !=3D nbdkitCaps->ctime) { + VIR_DEBUG("Outdated capabilities for '%s': nbdkit binary changed " + "(%lld vs %lld)", + nbdkitCaps->path, + (long long)st.st_ctime, (long long)nbdkitCaps->ctime); + return false; + } + + return true; +} + + +static void* +virNbdkitCapsNewData(const char *binary, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *caps =3D qemuNbdkitCapsNew(binary); + qemuNbdkitCapsQuery(caps); + + return caps; +} + + +virFileCacheHandlers nbdkitCapsCacheHandlers =3D { + .isValid =3D virNbdkitCapsIsValid, + .newData =3D virNbdkitCapsNewData, + .loadFile =3D NULL, + .saveFile =3D NULL, + .privFree =3D NULL, +}; + + +virFileCache* +qemuNbdkitCapsCacheNew(const char *cachedir) +{ + g_autofree char *dir =3D g_build_filename(cachedir, "nbdkitcapabilitie= s", NULL); + return virFileCacheNew(dir, "xml", &nbdkitCapsCacheHandlers); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 8ffb0f7044..f6cbedaa94 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -23,6 +23,7 @@ =20 #include "internal.h" #include "virenum.h" +#include "virfilecache.h" =20 typedef struct _qemuNbdkitCaps qemuNbdkitCaps; =20 @@ -40,6 +41,9 @@ VIR_ENUM_DECL(qemuNbdkitCaps); qemuNbdkitCaps * qemuNbdkitCapsNew(const char *path); =20 +virFileCache * +qemuNbdkitCapsCacheNew(const char *cachedir); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303168; cv=none; d=zohomail.com; s=zohoarc; b=DJtnivQ4wLlgqErSsNW34KZ6BelHWP/MaVqCQrWjT4q/aC2qk54lVImySG/N0oIfrNFqY2pOa+GErViKMZRRjDFs3zdCQly+qyp8G0wX+Q91zBbPZ1L1c0RnKJ7JR0szmXRD+lW6mTw8HZ0MMcEA/Er/i0rIgMSTWIvOu5c0mt8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303168; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fIlylQUwozfgetTl2vG+ssDjDz6JcEUp0yQD6bnvorw=; b=Ijq3FxQoxH8kbq4OecmVCWlyT26kS1Be2xzxhugZhB/DTfMoFJTMfLVn521ZdCx0lGfftRvZof1SwG5eAzF+W8VjkqsTod1cPWU/8FerqhitM7FiJchpFHw3R2jjJP05Z87SbPpg92xd0HzK7JrHOu4d1VzkEbiyQSgL6gLhpMA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303168350188.80855509440983; Thu, 20 Oct 2022 14:59:28 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-557-8y7YDg_JOfqHbcYg3H7iWQ-1; Thu, 20 Oct 2022 17:59:21 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C47963817984; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A8DA31402BD9; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id DC40919465A2; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 66AE719465BA for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 4922F4047AC; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 27CE24047AA for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303166; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=fIlylQUwozfgetTl2vG+ssDjDz6JcEUp0yQD6bnvorw=; b=hlQ216Xh34R2ge8cutDVyP/4rczg1m5Rp7sWh4RfYUWDe3wnOe9cfgarosyyEH6Uw6GF5L oT28YlGf9tEW7en/C6TqHaluclWgVfLt6LB+rbsOGyzZgd+eDB0mYK5hgPUE4rn8j901JO lRXUHCyPIOW4gYGiDsR/bwn4c2jUUWI= X-MC-Unique: 8y7YDg_JOfqHbcYg3H7iWQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 06/18] qemu: implement persistent file cache for nbdkit caps Date: Thu, 20 Oct 2022 16:58:57 -0500 Message-Id: <20221020215909.1751428-7-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303170630100002 Content-Type: text/plain; charset="utf-8"; x-default="true" Implement the loadFile and saveFile virFileCacheHandlers callbacks so that nbdkit capabilities are cached perstistently across daemon restarts. The format and implementation is modeled on the qemu capabilities, but simplified slightly. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_nbdkit.c | 234 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 232 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 9ab048e9e1..a47e169a0f 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -317,11 +317,241 @@ virNbdkitCapsNewData(const char *binary, } =20 =20 +static int +qemuNbdkitCapsValidateBinary(qemuNbdkitCaps *nbdkitCaps, + xmlXPathContextPtr ctxt) +{ + g_autofree char *str =3D NULL; + + if (!(str =3D virXPathString("string(./path)", ctxt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing path in nbdkit capabilities cache")); + return -1; + } + + if (STRNEQ(str, nbdkitCaps->path)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Expected caps for '%s' but saw '%s'"), + nbdkitCaps->path, str); + return -1; + } + + return 0; +} + + +static int +qemuNbdkitCapsParseFlags(qemuNbdkitCaps *nbdkitCaps, + xmlXPathContextPtr ctxt) +{ + g_autofree xmlNodePtr *nodes =3D NULL; + size_t i; + int n; + + if ((n =3D virXPathNodeSet("./flag", ctxt, &nodes)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to parse qemu capabilities flags")); + return -1; + } + + VIR_DEBUG("Got flags %d", n); + for (i =3D 0; i < n; i++) { + unsigned int flag; + + if (virXMLPropEnum(nodes[i], "name", qemuNbdkitCapsTypeFromString, + VIR_XML_PROP_REQUIRED, &flag) < 0) + return -1; + + qemuNbdkitCapsSet(nbdkitCaps, flag); + } + + return 0; +} + + +/* + * Parsing a doc that looks like + * + * + * /some/path + * 234235253 + * 234235253 + * 234235253 + * 234235253 + * 1002016 + * + * + * ... + * + * + * Returns 0 on success, 1 if outdated, -1 on error + */ +static int +qemuNbdkitCapsLoadCache(qemuNbdkitCaps *nbdkitCaps, + const char *filename) +{ + g_autoptr(xmlDoc) doc =3D NULL; + g_autoptr(xmlXPathContext) ctxt =3D NULL; + long long int l; + unsigned long lu; + + if (!(doc =3D virXMLParse(filename, NULL, NULL, "nbdkitCaps", &ctxt, N= ULL, false))) + return -1; + + if (virXPathLongLong("string(./selfctime)", ctxt, &l) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing selfctime in nbdkit capabilities XML")); + return -1; + } + nbdkitCaps->libvirtCtime =3D (time_t)l; + + nbdkitCaps->libvirtVersion =3D 0; + if (virXPathULong("string(./selfvers)", ctxt, &lu) =3D=3D 0) + nbdkitCaps->libvirtVersion =3D lu; + + if (nbdkitCaps->libvirtCtime !=3D virGetSelfLastChanged() || + nbdkitCaps->libvirtVersion !=3D LIBVIR_VERSION_NUMBER) { + VIR_DEBUG("Outdated capabilities in %s: libvirt changed " + "(%lld vs %lld, %lu vs %lu), stopping load", + nbdkitCaps->path, + (long long)nbdkitCaps->libvirtCtime, + (long long)virGetSelfLastChanged(), + (unsigned long)nbdkitCaps->libvirtVersion, + (unsigned long)LIBVIR_VERSION_NUMBER); + return 1; + } + + if (qemuNbdkitCapsValidateBinary(nbdkitCaps, ctxt) < 0) + return -1; + + if (virXPathLongLong("string(./nbdkitctime)", ctxt, &l) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing nbdkitctime in nbdkit capabilities XML")= ); + return -1; + } + nbdkitCaps->ctime =3D (time_t)l; + + if (virXPathLongLong("string(./plugindirmtime)", ctxt, &l) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing plugindirmtime in nbdkit capabilities XM= L")); + return -1; + } + nbdkitCaps->pluginDirMtime =3D (time_t)l; + + if (virXPathLongLong("string(./filterdirmtime)", ctxt, &l) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing filterdirmtime in nbdkit capabilities XM= L")); + return -1; + } + nbdkitCaps->filterDirMtime =3D (time_t)l; + + if (qemuNbdkitCapsParseFlags(nbdkitCaps, ctxt) < 0) + return -1; + + if ((nbdkitCaps->version =3D virXPathString("string(./version)", ctxt)= ) =3D=3D NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing version in nbdkit capabilities cache")); + return -1; + } + + return 0; +} + + +static void* +virNbdkitCapsLoadFile(const char *filename, + const char *binary, + void *privData G_GNUC_UNUSED, + bool *outdated) +{ + g_autoptr(qemuNbdkitCaps) nbdkitCaps =3D qemuNbdkitCapsNew(binary); + int ret; + + if (!nbdkitCaps) + return NULL; + + ret =3D qemuNbdkitCapsLoadCache(nbdkitCaps, filename); + if (ret < 0) + return NULL; + if (ret =3D=3D 1) { + *outdated =3D true; + return NULL; + } + + return g_steal_pointer(&nbdkitCaps); +} + + +static char* +qemuNbdkitCapsFormatCache(qemuNbdkitCaps *nbdkitCaps) +{ + g_auto(virBuffer) buf =3D VIR_BUFFER_INITIALIZER; + size_t i; + + virBufferAddLit(&buf, "\n"); + virBufferAdjustIndent(&buf, 2); + + virBufferEscapeString(&buf, "%s\n", + nbdkitCaps->path); + virBufferAsprintf(&buf, "%lld\n", + (long long)nbdkitCaps->ctime); + virBufferAsprintf(&buf, "%lld\n", + (long long)nbdkitCaps->pluginDirMtime); + virBufferAsprintf(&buf, "%lld\n", + (long long)nbdkitCaps->filterDirMtime); + virBufferAsprintf(&buf, "%lld\n", + (long long)nbdkitCaps->libvirtCtime); + virBufferAsprintf(&buf, "%lu\n", + (unsigned long)nbdkitCaps->libvirtVersion); + + for (i =3D 0; i < QEMU_NBDKIT_CAPS_LAST; i++) { + if (qemuNbdkitCapsGet(nbdkitCaps, i)) { + virBufferAsprintf(&buf, "\n", + qemuNbdkitCapsTypeToString(i)); + } + } + + virBufferAsprintf(&buf, "%s\n", + nbdkitCaps->version); + + virBufferAdjustIndent(&buf, -2); + virBufferAddLit(&buf, "\n"); + + return virBufferContentAndReset(&buf); +} + + +static int +virNbdkitCapsSaveFile(void *data, + const char *filename, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *nbdkitCaps =3D data; + g_autofree char *xml =3D NULL; + + xml =3D qemuNbdkitCapsFormatCache(nbdkitCaps); + + if (virFileWriteStr(filename, xml, 0600) < 0) { + virReportSystemError(errno, + _("Failed to save '%s' for '%s'"), + filename, nbdkitCaps->path); + return -1; + } + + VIR_DEBUG("Saved caps '%s' for '%s' with (%lld, %lld)", + filename, nbdkitCaps->path, + (long long)nbdkitCaps->ctime, + (long long)nbdkitCaps->libvirtCtime); + + return 0; +} + + virFileCacheHandlers nbdkitCapsCacheHandlers =3D { .isValid =3D virNbdkitCapsIsValid, .newData =3D virNbdkitCapsNewData, - .loadFile =3D NULL, - .saveFile =3D NULL, + .loadFile =3D virNbdkitCapsLoadFile, + .saveFile =3D virNbdkitCapsSaveFile, .privFree =3D NULL, }; =20 --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303212; cv=none; d=zohomail.com; s=zohoarc; b=TDIeNSoNJsjlIKjqLzIwXUkvzCTi3U8vDSFYUbo88dDDgc/BVtJX7uydMXPt9DCH4xATYn5ZgAs9XnTDAdeMx0ajlGAY7XT/+aLwdDZ91fat6sejsyArtWkfyMpuZ644VEdDI5XT8s9jLWgTCgcJyRg8jCrrfstpEs++K1sUpFk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303212; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qlvPB8Ze6gV36ouvj/7KdjVMGoFd8AUutGU2ES9s3Ps=; b=KCE5bxiK0spUr1Z5hx60c0k5nJR+aG0uUEYLmWfIFya6utAYrmdoZbbjGZ3rIixzqWx4yYEjX8/yz4ye4DcrSfEj9I/ch23P8x6yYB8iDVd+ryZwhVBETIvdGfiHwhk+dhPTcFKd+FHIlezT0TcuvvGBaTldoYIwbIP1bHfqhhw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303212672690.9708531578239; Thu, 20 Oct 2022 15:00:12 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-504-dFY_WdykPPCr7TKOTN-lZQ-1; Thu, 20 Oct 2022 17:59:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 95932804185; Thu, 20 Oct 2022 21:59:19 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 802052166B5D; Thu, 20 Oct 2022 21:59:19 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E7B5E194704B; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8B5671946A46 for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 80FD04047AA; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5BCFE4047A7 for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303211; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=qlvPB8Ze6gV36ouvj/7KdjVMGoFd8AUutGU2ES9s3Ps=; b=TzsGWJOnDOsX/Kg7Q//+LsTDJZalm0YPUhYu0sxoqfBuS3PdJrcB+ixb14Ko+PBBcXaZDY VCWMue3QdR0PdL3Va7S+mtRd0KOcR2WBU9crxmur8yNYyJv+NYeyKqRdkJ30yJZ3tuglPr 2pzUkhTtUxgSVHKxEeszClDujb5ww7s= X-MC-Unique: dFY_WdykPPCr7TKOTN-lZQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 07/18] qemu: use file cache for nbdkit caps Date: Thu, 20 Oct 2022 16:58:58 -0500 Message-Id: <20221020215909.1751428-8-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303214709100003 Content-Type: text/plain; charset="utf-8"; x-default="true" Add the virFileCache implementation for nbdkit capabilities to the qemu driver. This allows us to determine whether nbdkit is installed and which plugins are supported. it also has persistent caching and the capabilities are re-queried whenever something changes. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_conf.h | 3 +++ src/qemu/qemu_driver.c | 4 ++++ 2 files changed, 7 insertions(+) diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index a39510b0b1..95c05e6888 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -320,6 +320,9 @@ struct _virQEMUDriver { =20 /* Immutable pointer, self-locking APIs */ virHashAtomic *migrationErrors; + + /* Immutable pointer, self-locking APIs */ + virFileCache *nbdkitCapsCache; }; =20 virQEMUDriverConfig *virQEMUDriverConfigNew(bool privileged, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 59a3b37b98..d42f3c2d2a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -833,6 +833,9 @@ qemuStateInitialize(bool privileged, defsecmodel))) goto error; =20 + /* find whether nbdkit is available and query its capabilities */ + qemu_driver->nbdkitCapsCache =3D qemuNbdkitCapsCacheNew(cfg->cacheDir); + /* If hugetlbfs is present, then we need to create a sub-directory wit= hin * it, since we can't assume the root mount point has permissions that * will let our spawned QEMU instances use it. */ @@ -1070,6 +1073,7 @@ qemuStateCleanup(void) VIR_FREE(qemu_driver->qemuImgBinary); virObjectUnref(qemu_driver->domains); virThreadPoolFree(qemu_driver->workerPool); + virObjectUnref(qemu_driver->nbdkitCapsCache); =20 if (qemu_driver->lockFD !=3D -1) virPidFileRelease(qemu_driver->config->stateDir, "driver", qemu_dr= iver->lockFD); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303204; cv=none; d=zohomail.com; s=zohoarc; b=J2O23cHdruzigppj+gJr3pzUv0RUcVEjG7v5hpwKTHSHFuHDDraPnnx+QGS8lEHMj2Pix98EjmGC8EWUP0yCgywXgSF0uW+jlqxLfaR77H8ejNSQo3vC/xKJtrmu2onDl8nAoxbz5X8zg+T2wHVffInyqfN4vIJ+6TacQzy8dno= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303204; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cf4Tg2lXUxLyYXBeQgQ4l6236KKZQtvAWapp6nd3F58=; b=bBDfuq0DMJlM54MsawTDNbpPQo31zzmuYKY+e3c5urpA+6MjNd65es81JgW0EgupFUr+wmCUabf4Lm2tATljrG0N8Ba4lDEqI8RsBT34kJhAlalsOzjp3xYRQ/VDM3UJmLP0eVT9PKSX/B+fZMCASj4nH6TZbP11JofUyxrflrI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1666303204481949.7798994156659; Thu, 20 Oct 2022 15:00:04 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-632-yFm8qU2-MNmbHxREh4vyiw-1; Thu, 20 Oct 2022 17:59:22 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 719773817A90; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5E0314047AD; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7C5841946A49; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id DA8231946A61 for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id C0D404047AA; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9F93A4047A7 for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303203; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=cf4Tg2lXUxLyYXBeQgQ4l6236KKZQtvAWapp6nd3F58=; b=VngG+W8Y3TuiXpdVh1SfkqqVeRapq7kgsglFqxXzwq2Sxut8g0s6q5zqnowIxAGrVHFEG/ ERMoJJa2ZqkSDQnke0A05EMsmqUDrfqQBVNWIL1/hhNuZMAxoEQO5h0JC5mLTHYu29uwq4 yL8qCuef4DjPxZv8i5+EiJHHRx2543w= X-MC-Unique: yFm8qU2-MNmbHxREh4vyiw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 08/18] qemu: Add qemuNbdkitProcess Date: Thu, 20 Oct 2022 16:58:59 -0500 Message-Id: <20221020215909.1751428-9-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303206683100001 Content-Type: text/plain; charset="utf-8"; x-default="true" An object for storing information about a nbdkit process that is serving a specific virStorageSource. At the moment, this information is just stored in the private data of virStorageSource and not used at all. Future commits will use this data to actually start a nbdkit process. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_conf.c | 23 ++++++++++++ src/qemu/qemu_conf.h | 3 ++ src/qemu/qemu_domain.c | 31 ++++++++++++++++ src/qemu/qemu_domain.h | 4 +++ src/qemu/qemu_nbdkit.c | 82 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 26 ++++++++++++++ 6 files changed, 169 insertions(+) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index ae5bbcd138..0370429da0 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -1641,3 +1641,26 @@ qemuHugepageMakeBasedir(virQEMUDriver *driver, =20 return 0; } + + +/* + * qemuGetNbdkitCaps: + * @driver: the qemu driver + * + * Gets the capabilities for Nbdkit for the specified driver. These can be= used + * to determine whether a particular disk source can be served by nbdkit or + * not. + * + * Returns: a reference to qemuNbdkitCaps or NULL + */ +qemuNbdkitCaps* +qemuGetNbdkitCaps(virQEMUDriver *driver) +{ + if (!driver->nbdkitBinary) + driver->nbdkitBinary =3D virFindFileInPath("nbdkit"); + + if (driver->nbdkitBinary) + return virFileCacheLookup(driver->nbdkitCapsCache, driver->nbdkitB= inary); + + return NULL; +} diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 95c05e6888..8b4f6ce669 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -323,6 +323,7 @@ struct _virQEMUDriver { =20 /* Immutable pointer, self-locking APIs */ virFileCache *nbdkitCapsCache; + char *nbdkitBinary; }; =20 virQEMUDriverConfig *virQEMUDriverConfigNew(bool privileged, @@ -379,3 +380,5 @@ int qemuGetMemoryBackingPath(virQEMUDriver *driver, =20 int qemuHugepageMakeBasedir(virQEMUDriver *driver, virHugeTLBFS *hugepage); + +qemuNbdkitCaps* qemuGetNbdkitCaps(virQEMUDriver *driver); diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 9ef6c8bb64..3fca163415 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -851,6 +851,7 @@ qemuDomainStorageSourcePrivateDispose(void *obj) g_clear_pointer(&priv->encinfo, qemuDomainSecretInfoFree); g_clear_pointer(&priv->httpcookie, qemuDomainSecretInfoFree); g_clear_pointer(&priv->tlsKeySecret, qemuDomainSecretInfoFree); + g_clear_pointer(&priv->nbdkitProcess, qemuNbdkitProcessFree); } =20 =20 @@ -10055,6 +10056,34 @@ qemuDomainPrepareStorageSourceNFS(virStorageSource= *src) } =20 =20 +/* qemuPrepareStorageSourceNbdkit: + * @src: source for a disk + * + * If src is an network source that is managed by nbdkit, prepare data so = that + * nbdkit can be launched before the domain is started + * + * Returns true if nbdkit will be used for this source, + */ +static bool +qemuDomainPrepareStorageSourceNbdkit(virStorageSource *src, + virQEMUDriverConfig *cfg, + const char *alias, + qemuDomainObjPrivate *priv) +{ + g_autoptr(qemuNbdkitCaps) nbdkit =3D NULL; + + if (virStorageSourceGetActualType(src) !=3D VIR_STORAGE_TYPE_NETWORK) + return false; + + nbdkit =3D qemuGetNbdkitCaps(priv->driver); + if (!nbdkit) + return false; + + return qemuNbdkitInitStorageSource(nbdkit, src, priv->libDir, + alias, cfg->user, cfg->group); +} + + /* qemuProcessPrepareStorageSourceTLS: * @source: source for a disk * @cfg: driver configuration @@ -10829,6 +10858,8 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virD= omainDiskDef *disk, if (qemuDomainPrepareStorageSourceNFS(src) < 0) return -1; =20 + qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, priv); + return 0; } =20 diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 2bbd492d62..79dd9e4329 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -33,6 +33,7 @@ #include "qemu_conf.h" #include "qemu_capabilities.h" #include "qemu_migration_params.h" +#include "qemu_nbdkit.h" #include "qemu_slirp.h" #include "qemu_fd.h" #include "virchrdev.h" @@ -298,6 +299,9 @@ struct _qemuDomainStorageSourcePrivate { =20 /* key for decrypting TLS certificate */ qemuDomainSecretInfo *tlsKeySecret; + + /* an nbdkit process for serving network storage sources */ + qemuNbdkitProcess *nbdkitProcess; }; =20 virObject *qemuDomainStorageSourcePrivateNew(void); diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index a47e169a0f..291f988c7a 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -562,3 +562,85 @@ qemuNbdkitCapsCacheNew(const char *cachedir) g_autofree char *dir =3D g_build_filename(cachedir, "nbdkitcapabilitie= s", NULL); return virFileCacheNew(dir, "xml", &nbdkitCapsCacheHandlers); } + + +static qemuNbdkitProcess * +qemuNbdkitProcessNew(virStorageSource *source, + const char *pidfile, + const char *socketfile) +{ + qemuNbdkitProcess *nbdkit =3D g_new0(qemuNbdkitProcess, 1); + /* weak reference -- source owns this object, so it will always outliv= e us */ + nbdkit->source =3D source; + nbdkit->user =3D -1; + nbdkit->group =3D -1; + nbdkit->pid =3D -1; + nbdkit->pidfile =3D g_strdup(pidfile); + nbdkit->socketfile =3D g_strdup(socketfile); + + return nbdkit; +} + + +bool +qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, + virStorageSource *source, + char *statedir, + const char *alias, + uid_t user, + gid_t group) +{ + qemuDomainStorageSourcePrivate *srcPriv =3D qemuDomainStorageSourcePri= vateFetch(source); + g_autofree char *pidname =3D g_strdup_printf("nbdkit-%s.pid", alias); + g_autofree char *socketname =3D g_strdup_printf("nbdkit-%s.socket", al= ias); + g_autofree char *pidfile =3D g_build_filename(statedir, pidname, NULL); + g_autofree char *socketfile =3D g_build_filename(statedir, socketname,= NULL); + qemuNbdkitProcess *proc; + + if (srcPriv->nbdkitProcess) + return false; + + switch (source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (!virBitmapIsBitSet(caps->flags, QEMU_NBDKIT_CAPS_PLUGIN_CU= RL)) + return false; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (!virBitmapIsBitSet(caps->flags, QEMU_NBDKIT_CAPS_PLUGIN_SS= H)) + return false; + break; + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + return false; + } + + proc =3D qemuNbdkitProcessNew(source, pidfile, socketfile); + proc->caps =3D g_object_ref(caps); + proc->user =3D user; + proc->group =3D group; + + srcPriv->nbdkitProcess =3D proc; + + return true; +} + + +void +qemuNbdkitProcessFree(qemuNbdkitProcess *proc) +{ + g_clear_pointer(&proc->pidfile, g_free); + g_clear_pointer(&proc->socketfile, g_free); + g_clear_object(&proc->caps); + g_free(proc); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index f6cbedaa94..5cb7b0f21b 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -22,10 +22,12 @@ #pragma once =20 #include "internal.h" +#include "storage_source_conf.h" #include "virenum.h" #include "virfilecache.h" =20 typedef struct _qemuNbdkitCaps qemuNbdkitCaps; +typedef struct _qemuNbdkitProcess qemuNbdkitProcess; =20 typedef enum { /* 0 */ @@ -44,6 +46,14 @@ qemuNbdkitCapsNew(const char *path); virFileCache * qemuNbdkitCapsCacheNew(const char *cachedir); =20 +bool +qemuNbdkitInitStorageSource(qemuNbdkitCaps *nbdkitCaps, + virStorageSource *source, + char *statedir, + const char *alias, + uid_t user, + gid_t group); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); @@ -54,3 +64,19 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, =20 #define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, = GObject); + +struct _qemuNbdkitProcess { + qemuNbdkitCaps *caps; + virStorageSource *source; + + char *pidfile; + char *socketfile; + uid_t user; + gid_t group; + pid_t pid; +}; + +void +qemuNbdkitProcessFree(qemuNbdkitProcess *proc); + +G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuNbdkitProcess, qemuNbdkitProcessFree); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303171; cv=none; d=zohomail.com; s=zohoarc; b=KTpACIY03IgkLp/CebCvF99zYVwlvZ96WIlqPj3KwsBiD5obECzuTm+RP+4BNTg4FWw9diDZJ5ugJVnLa2jLkJh1YvSEWFQX70oOz5lSjOymS7GXb8fbkAuNCDAzdc2FQ+nfyDAb3rkdqK3gdbZXCP4WdpDK1Ju09JrcgSXR3PU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303171; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=P8AjEfAE/0MFG+Mi8bxBTBNp/8FDnNWB0NUw2QK+CSw=; b=eJ72Ol9OPPIxvsydgh4UOiChi//lfJmnZn7t+uMS1IsCy2w1hIw7dRWirjdPOy0UOFROZf5mjgBvLsppxP6XMYxl7d5io4svLhiuU6sAKUHLMhSVPU2uZ0XMLm1RwluH5nkDTVXaXm2rE1Zm0rUD39cH3L/XLRWaKy/Jl5BoLT4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303171730477.2376127550815; Thu, 20 Oct 2022 14:59:31 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-621-vdMz1PpWMcy-okAN1vORCw-1; Thu, 20 Oct 2022 17:59:26 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3E6732A5956B; Thu, 20 Oct 2022 21:59:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20B4D477F62; Thu, 20 Oct 2022 21:59:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E10B41946A50; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1BB231946A61 for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 011964047AA; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D405A4047A7 for ; Thu, 20 Oct 2022 21:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303170; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=P8AjEfAE/0MFG+Mi8bxBTBNp/8FDnNWB0NUw2QK+CSw=; b=TNF+A6ZkmKg88qf7ivW9Ns2b6A7d2sFj5AY7U/Gj+MwomIahiYBZqpZ2+jIYycM+KZwjWn M1/PuNT27ufjfC33MmCG2pdJTcPoo5CksaCrKBw6EYXu+INZvMvgxP+4vTuqNZ7gpcm+My +NYTsi7mhqHHP5PFJuRlAsxcKRtt0HE= X-MC-Unique: vdMz1PpWMcy-okAN1vORCw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 09/18] qemu: add functions to start and stop nbdkit Date: Thu, 20 Oct 2022 16:59:00 -0500 Message-Id: <20221020215909.1751428-10-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303172362100005 Content-Type: text/plain; charset="utf-8"; x-default="true" Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_nbdkit.c | 251 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed, 261 insertions(+) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 291f988c7a..c5b0762f8d 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -26,6 +26,7 @@ #include "virerror.h" #include "virlog.h" #include "virpidfile.h" +#include "virtime.h" #include "virutil.h" #include "qemu_block.h" #include "qemu_conf.h" @@ -636,6 +637,163 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } =20 =20 +static int +qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + g_autoptr(virURI) uri =3D qemuBlockStorageSourceGetURI(proc->source); + g_autofree char *uristring =3D virURIFormat(uri); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "curl"); + virCommandAddArgPair(cmd, "protocols", + virStorageNetProtocolTypeToString(proc->source->p= rotocol)); + virCommandAddArgPair(cmd, "url", uristring); + + if (proc->source->auth) { + g_autoptr(virConnect) conn =3D virGetConnectSecret(); + g_autofree uint8_t *secret =3D NULL; + size_t secretlen =3D 0; + g_autofree char *password =3D NULL; + int secrettype; + + virCommandAddArgPair(cmd, "user", + proc->source->auth->username); + + if ((secrettype =3D virSecretUsageTypeFromString(proc->source->aut= h->secrettype)) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("invalid secret type %s"), + proc->source->auth->secrettype); + return -1; + } + + if (virSecretGetSecretString(conn, + &proc->source->auth->seclookupdef, + secrettype, + &secret, + &secretlen) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to get auth secret for storage")); + return -1; + } + + /* ensure that the secret is a NULL-terminated string */ + password =3D g_strndup((char*)secret, secretlen); + + /* for now, just report an error rather than passing the password = in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, + "%s", + "Password not yet supported for nbdkit sources"); + return -1; + } + + if (proc->source->ncookies > 0) { + /* for now, just report an error rather than passing cookies in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, + "%s", + "Cookies not yet supported for nbdkit sources"); + return -1; + } + + if (proc->source->sslverify =3D=3D VIR_TRISTATE_BOOL_NO) { + virCommandAddArgPair(cmd, "sslverify", "false"); + } + + if (proc->source->timeout > 0) { + g_autofree char *timeout =3D g_strdup_printf("%llu", proc->source-= >timeout); + virCommandAddArgPair(cmd, "timeout", timeout); + } + + return 0; +} + + +static int +qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + const char *user =3D NULL; + virStorageNetHostDef *host =3D &proc->source->hosts[0]; + g_autofree char *portstr =3D g_strdup_printf("%u", host->port); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "ssh"); + + virCommandAddArgPair(cmd, "host", host->name); + virCommandAddArgPair(cmd, "port", portstr); + virCommandAddArgPair(cmd, "path", proc->source->path); + + if (proc->source->auth) + user =3D proc->source->auth->username; + else if (proc->source->ssh_user) + user =3D proc->source->ssh_user; + + if (user) + virCommandAddArgPair(cmd, "user", user); + + if (proc->source->ssh_host_key_check_disabled) + virCommandAddArgPair(cmd, "verify-remote-host", "false"); + + return 0; +} + + +static virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(proc->caps->path, + "--exit-with-parent", + "--unix", + proc->socketfile, + "--foreground", + //"--selinux-label", + //selinux_label, + NULL); + + if (proc->source->readonly) + virCommandAddArg(cmd, "--readonly"); + + if (qemuNbdkitCapsGet(proc->caps, QEMU_NBDKIT_CAPS_FILTER_READAHEAD) && + proc->source->readahead > 0) + virCommandAddArgPair(cmd, "--filter", "readahead"); + + switch (proc->source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (qemuNbdkitProcessBuildCommandCurl(proc, cmd) < 0) + return NULL; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (qemuNbdkitProcessBuildCommandSSH(proc, cmd) < 0) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportError(VIR_ERR_NO_SUPPORT, + _("protocol '%s' is not supported by nbdkit"), + virStorageNetProtocolTypeToString(proc->source-= >protocol)); + return NULL; + } + + virCommandDaemonize(cmd); + + return g_steal_pointer(&cmd); +} + + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { @@ -644,3 +802,96 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) g_clear_object(&proc->caps); g_free(proc); } + + +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver) +{ + g_autoptr(virCommand) cmd =3D NULL; + int rc; + int exitstatus =3D 0; + int cmdret =3D 0; + VIR_AUTOCLOSE errfd =3D -1; + virTimeBackOffVar timebackoff; + bool socketCreated =3D false; + + if (!(cmd =3D qemuNbdkitProcessBuildCommand(proc))) + return -1; + + VIR_DEBUG("starting nbdkit process for %s", proc->source->nodestorage); + virCommandSetErrorFD(cmd, &errfd); + virCommandSetPidFile(cmd, proc->pidfile); + + if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) + goto error; + + if (qemuSecurityCommandRun(driver, vm, cmd, proc->user, proc->group, &= exitstatus, &cmdret) < 0) + goto error; + + if (cmdret < 0 || exitstatus !=3D 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not start 'nbdkit'. exitstatus: %d"), exit= status); + goto error; + } + + if ((rc =3D virPidFileReadPath(proc->pidfile, &proc->pid)) < 0) { + virReportSystemError(-rc, + _("Failed to read pidfile %s"), + proc->pidfile); + goto error; + } + + if (virTimeBackOffStart(&timebackoff, 1, 1000) < 0) + goto error; + + while (virTimeBackOffWait(&timebackoff)) { + if ((socketCreated =3D virFileExists(proc->socketfile))) + break; + + if (virProcessKill(proc->pid, 0) =3D=3D 0) + continue; + + goto error; + } + + if (!socketCreated) { + virReportError(VIR_ERR_OPERATION_TIMEOUT, "%s", + _("nbdkit socket did not show up")); + goto error; + } + + return 0; + + error: + if (errfd > 0) { + g_autofree char *errbuf =3D g_new0(char, 1024); + if (read(errfd, errbuf, 1024) > 0) + virReportError(VIR_ERR_OPERATION_FAILED, + _("nbdkit failed to start and reported: %s"), e= rrbuf); + } + qemuNbdkitProcessStop(proc); + return -1; +} + + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc) +{ + int ret; + + if (proc->pid < 0) + return 0; + + VIR_DEBUG("Stopping nbdkit process %i", proc->pid); + unlink(proc->pidfile); + unlink(proc->socketfile); + + ret =3D virProcessKillPainfully(proc->pid, true); + + if (ret >=3D 0) + proc->pid =3D -1; + + return ret; +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 5cb7b0f21b..30cab744b0 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -40,6 +40,8 @@ typedef enum { =20 VIR_ENUM_DECL(qemuNbdkitCaps); =20 +typedef struct _virQEMUDriver virQEMUDriver; + qemuNbdkitCaps * qemuNbdkitCapsNew(const char *path); =20 @@ -76,6 +78,14 @@ struct _qemuNbdkitProcess { pid_t pid; }; =20 +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver); + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc); + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); =20 --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303208; cv=none; d=zohomail.com; s=zohoarc; b=fAMwEyGaCywjZlTyWk9MP2o6GlTuwbuHEkjSnnZK+mI87FSPp5l2Io5Oy5X2vD56plw4ICwaoZfrEMgMtxuJy3S8+zuf0ASR9U+oWshd0NG96QfM3dU5CBP02YTgDbGq/ues2ua+5eUuFxpeqpS8/+cgM/65qBQgexoleQg7t/g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303208; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UsMqAPEVzdAl7CfDzu2xOhrrEUXggqzJwLzeWozYWhQ=; b=U31+Z6Tdw4fzoQPPjSyhsDPYm32X20ddNvfS2N98VMamULlafBoqy2yr0u3j+VVEW0FlAehc1M9o3BYDtU9VSnLsVqJPwa+B6WZd/BEvBZ+/JTe5ZsKcxSpL8k0BliP1gatsjik7i+21URTGV+0XU00hKQnyUnYM49R2dog97YY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 166630320872340.73724658795163; Thu, 20 Oct 2022 15:00:08 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-45-5n4FZX3ROJicRm6mlJrwUg-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DAF2485A59D; Thu, 20 Oct 2022 21:59:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id C93612027063; Thu, 20 Oct 2022 21:59:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7965E19465BA; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 4E8CC1946587 for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 349D84047AA; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 13A4B4047A7 for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303207; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=UsMqAPEVzdAl7CfDzu2xOhrrEUXggqzJwLzeWozYWhQ=; b=fmCUS63IiwN2z+uCxyuIyWED6gw/DEtq6+MpUO8eg5zTFHGkHsxm43MuS8DLL7pOszRwyt yUGQjb2dc7MSkxOu3Yg+n8HGuHUGgmVHQtBsSFOTKUYqDGHim0Sts8DPCZY0V0gKwdEJek LBAdIlZ0cf8G05sx7TO1Yj9ouQoJxOk= X-MC-Unique: 5n4FZX3ROJicRm6mlJrwUg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 10/18] tests: add ability to test various nbdkit capabilities Date: Thu, 20 Oct 2022 16:59:01 -0500 Message-Id: <20221020215909.1751428-11-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303210714100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Add new DO_TEST_CAPS_LATEST_NBDKIT macro to test xml2argv for various nbdkit capability scenarios. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 20 +++++++++++++++++--- tests/qemuxml2argvtest.c | 11 +++++++++++ tests/testutilsqemu.c | 27 +++++++++++++++++++++++++++ tests/testutilsqemu.h | 5 +++++ 4 files changed, 60 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index c5b0762f8d..59c452a15e 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -263,10 +263,16 @@ virNbkditCapsCheckModdir(const char *moddir, =20 static bool virNbdkitCapsIsValid(void *data, - void *privData G_GNUC_UNUSED) + void *privData) { qemuNbdkitCaps *nbdkitCaps =3D data; struct stat st; + /* when run under test, we will use privData as a signal to indicate t= hat + * we shouldn't touch the filesystem */ + bool skipValidation =3D (privData !=3D NULL); + + if (skipValidation) + return true; =20 if (!nbdkitCaps->path) return true; @@ -309,9 +315,17 @@ virNbdkitCapsIsValid(void *data, =20 static void* virNbdkitCapsNewData(const char *binary, - void *privData G_GNUC_UNUSED) + void *privData) { - qemuNbdkitCaps *caps =3D qemuNbdkitCapsNew(binary); + /* when run under test, we will use privData as a signal to indicate t= hat + * we shouldn't touch the filesystem */ + bool skipNewData =3D (privData !=3D NULL); + qemuNbdkitCaps *caps =3D NULL; + + if (skipNewData) + return NULL; + + caps =3D qemuNbdkitCapsNew(binary); qemuNbdkitCapsQuery(caps); =20 return caps; diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index ef32cae2e9..0032bafdce 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -670,6 +670,14 @@ testCompareXMLToArgv(const void *data) if (rc < 0) goto cleanup; =20 + if (info->nbdkitCaps) { + if (virFileCacheInsertData(driver.nbdkitCapsCache, TEST_NBDKIT_PAT= H, + g_object_ref(info->nbdkitCaps)) < 0) { + g_object_unref(info->nbdkitCaps); + goto cleanup; + } + } + if (info->migrateFrom && !(migrateURI =3D qemuMigrationDstGetURI(info->migrateFrom, info->migrateFd))) @@ -927,6 +935,9 @@ mymain(void) # define DO_TEST_CAPS_ARCH_VER(name, arch, ver) \ DO_TEST_CAPS_ARCH_VER_FULL(name, arch, ver, ARG_END) =20 +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ + DO_TEST_CAPS_ARCH_LATEST_FULL(name, "x86_64", ARG_NBDKIT_CAPS, __VA_AR= GS__, QEMU_NBDKIT_CAPS_LAST, ARG_END) + # define DO_TEST_CAPS_LATEST(name) \ DO_TEST_CAPS_ARCH_LATEST(name, "x86_64") =20 diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index 6d3decdc16..386042aa79 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -131,6 +131,10 @@ virFindFileInPath(const char *file) return g_strdup_printf("/usr/bin/%s", file); } =20 + if (g_str_equal(file, "nbdkit")) { + return g_strdup(TEST_NBDKIT_PATH); + } + /* Nothing in tests should be relying on real files * in host OS, so we return NULL to try to force * an error in such a case @@ -422,6 +426,7 @@ void qemuTestDriverFree(virQEMUDriver *driver) virObjectUnref(driver->caps); virObjectUnref(driver->config); virObjectUnref(driver->securityManager); + g_clear_object(&driver->nbdkitCapsCache); =20 virCPUDefFree(cpuDefault); virCPUDefFree(cpuHaswell); @@ -665,6 +670,12 @@ int qemuTestDriverInit(virQEMUDriver *driver) if (!driver->qemuCapsCache) goto error; =20 + driver->nbdkitCapsCache =3D qemuNbdkitCapsCacheNew("/dev/null"); + /* the nbdkitCapsCache just interprets the presence of a non-null priv= ate + * data pointer as a signal to skip cache validation. This prevents the + * cache from trying to validate the plugindir mtime, etc during test = */ + virFileCacheSetPriv(driver->nbdkitCapsCache, GUINT_TO_POINTER(1)); + driver->xmlopt =3D virQEMUDriverCreateXMLConf(driver, "none"); if (!driver->xmlopt) goto error; @@ -885,6 +896,7 @@ testQemuInfoSetArgs(struct testQemuInfo *info, =20 info->conf =3D conf; info->args.newargs =3D true; + info->args.fakeNbdkitCaps =3D qemuNbdkitCapsNew(TEST_NBDKIT_PATH); =20 va_start(argptr, conf); while ((argname =3D va_arg(argptr, testQemuInfoArgName)) !=3D ARG_END)= { @@ -896,6 +908,13 @@ testQemuInfoSetArgs(struct testQemuInfo *info, virQEMUCapsSet(info->args.fakeCaps, flag); break; =20 + case ARG_NBDKIT_CAPS: + info->args.fakeNbdkitCapsUsed =3D true; + + while ((flag =3D va_arg(argptr, int)) < QEMU_NBDKIT_CAPS_LAST) + qemuNbdkitCapsSet(info->args.fakeNbdkitCaps, flag); + break; + case ARG_GIC: info->args.gic =3D va_arg(argptr, int); break; @@ -1020,6 +1039,12 @@ testQemuInfoInitArgs(struct testQemuInfo *info) info->qemuCaps =3D g_steal_pointer(&info->args.fakeCaps); } =20 + if (info->args.fakeNbdkitCapsUsed) + info->nbdkitCaps =3D g_steal_pointer(&info->args.fakeNbdkitCaps); + else + /* empty caps */ + info->nbdkitCaps =3D qemuNbdkitCapsNew(TEST_NBDKIT_PATH); + if (info->args.gic !=3D GIC_NONE && testQemuCapsSetGIC(info->qemuCaps, info->args.gic) < 0) return -1; @@ -1037,6 +1062,8 @@ testQemuInfoClear(struct testQemuInfo *info) VIR_FREE(info->errfile); virObjectUnref(info->qemuCaps); g_clear_pointer(&info->args.fakeCaps, virObjectUnref); + g_clear_object(&info->nbdkitCaps); + g_clear_object(&info->args.fakeNbdkitCaps); } =20 =20 diff --git a/tests/testutilsqemu.h b/tests/testutilsqemu.h index 943958d02a..618837559c 100644 --- a/tests/testutilsqemu.h +++ b/tests/testutilsqemu.h @@ -28,6 +28,7 @@ # define TEST_TPM_ENV_VAR "VIR_TEST_MOCK_FAKE_TPM_VERSION" # define TPM_VER_1_2 "1.2" # define TPM_VER_2_0 "2.0" +# define TEST_NBDKIT_PATH "/usr/bin/nbdkit" =20 enum { GIC_NONE =3D 0, @@ -52,6 +53,7 @@ typedef enum { ARG_CAPS_VER, ARG_CAPS_HOST_CPU_MODEL, ARG_HOST_OS, + ARG_NBDKIT_CAPS, ARG_END, } testQemuInfoArgName; =20 @@ -82,6 +84,8 @@ struct testQemuArgs { bool newargs; virQEMUCaps *fakeCaps; bool fakeCapsUsed; + qemuNbdkitCaps *fakeNbdkitCaps; + bool fakeNbdkitCapsUsed; char *capsver; char *capsarch; qemuTestCPUDef capsHostCPUModel; @@ -96,6 +100,7 @@ struct testQemuInfo { char *outfile; char *errfile; virQEMUCaps *qemuCaps; + qemuNbdkitCaps *nbdkitCaps; const char *migrateFrom; int migrateFd; unsigned int flags; --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303235; cv=none; d=zohomail.com; s=zohoarc; b=P5hArAB/+DgvzHDFCN+v/ll5dQkmcLCTxf1a13RHhqCjkjeKxxvUd9ViC8o3nS+4cuQD0WKzkvxjvDlxVNgV+iQmqz+3fVi7mGkRrUsbsfcL8FtWJEfV65rOKblK0Prfa+PXSpvN3QZJWqcx9PbCYBTVUHnaiA25dbGwSdgZsF8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303235; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LJrn3FANqHaOnrOm82wUxbzFVHOALfn22jYoj21ewps=; b=TP5X6R6+PamzVqB+Il3hoRb6YTRBzGNZf7R0Je7b9sQRXPZrk07YmksJJY4oeABh83tGq34tfXRs4AeX06LfCMgMhg7b/JtGpBzDrFSfhWLBg9TJ04JJ0m2vrq+oWQB0u0x4Ypp2Op0gfU2iuoD3xQEqTcqpZGg+4piorh7arvE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303235081335.4584288601609; Thu, 20 Oct 2022 15:00:35 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-99-NH3eam0JNpq3RjakBOqLSA-1; Thu, 20 Oct 2022 17:59:30 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2749C102BCBD; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 14730111CD38; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E65291946587; Thu, 20 Oct 2022 21:59:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 87CBA1946587 for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 7C17E4047A7; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4773C4047AC; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303234; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=LJrn3FANqHaOnrOm82wUxbzFVHOALfn22jYoj21ewps=; b=aqrBu1ceSBRlzvBx+7ACYRyHdIpYTDxBMRZQHyZiY1ZZELF9F00T7ToQYFONJ4ee0pbXHb BkacjIxpiP231FKAmevTdtyBBvqf/0hCiw+R6sJE+7U2R83GbxabwYNnJ4Nloo455Rm+w/ HzWVxx/2QQgnGLDDrDQOLqcOih6yg+s= X-MC-Unique: NH3eam0JNpq3RjakBOqLSA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 11/18] qemu: split qemuDomainSecretStorageSourcePrepare Date: Thu, 20 Oct 2022 16:59:02 -0500 Message-Id: <20221020215909.1751428-12-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303236857100001 Content-Type: text/plain; charset="utf-8"; x-default="true" This prepares encryption secrets and authentication secrets. When we add nbdkit-backed network storage sources, we will not need to send authentication secrets to qemu, since they will be sent to nbdkit instead. So split this into two different functions. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 83 ++++++++++++++++++++++++++---------------- 1 file changed, 52 insertions(+), 31 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 3fca163415..ace7ae4c61 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1299,24 +1299,19 @@ qemuDomainSecretStorageSourcePrepareCookies(qemuDom= ainObjPrivate *priv, =20 =20 /** - * qemuDomainSecretStorageSourcePrepare: + * qemuDomainSecretStorageSourcePrepareEncryption: * @priv: domain private object * @src: storage source struct to setup - * @authalias: prefix of the alias for secret holding authentication data - * @encalias: prefix of the alias for secret holding encryption password + * @alias: prefix of the alias for secret holding encryption password * - * Prepares data necessary for encryption and authentication of @src. The = two - * alias prefixes are provided since in the backing chain authentication b= elongs - * to the storage protocol data whereas encryption is relevant to the form= at - * driver in qemu. The two will have different node names. + * Prepares data necessary for encryption of @src. * * Returns 0 on success; -1 on error while reporting an libvirt error. */ static int -qemuDomainSecretStorageSourcePrepare(qemuDomainObjPrivate *priv, - virStorageSource *src, - const char *aliasprotocol, - const char *aliasformat) +qemuDomainSecretStorageSourcePrepareEncryption(qemuDomainObjPrivate *priv, + virStorageSource *src, + const char *alias) { qemuDomainStorageSourcePrivate *srcPriv; bool hasEnc =3D src->encryption && src->encryption->nsecrets > 0; @@ -1324,13 +1319,43 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjP= rivate *priv, if (virStorageSourceIsEmpty(src)) return 0; =20 - if (!src->auth && !hasEnc && src->ncookies =3D=3D 0) + if (!hasEnc) return 0; =20 - if (!(src->privateData =3D qemuDomainStorageSourcePrivateNew())) + srcPriv =3D qemuDomainStorageSourcePrivateFetch(src); + + if (!(srcPriv->encinfo =3D qemuDomainSecretInfoSetupFromSecret(priv, a= lias, + "encrypti= on", + VIR_SECRE= T_USAGE_TYPE_VOLUME, + NULL, + &src->enc= ryption->secrets[0]->seclookupdef))) return -1; =20 - srcPriv =3D QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); + return 0; +} + + +/** + * qemuDomainSecretStorageSourcePrepareAuth: + * @priv: domain private object + * @src: storage source struct to setup + * @alias: prefix of the alias for secret holding authentication data + * + * Prepares data necessary for authentication of @src. + * + * Returns 0 on success; -1 on error while reporting an libvirt error. + */ +static int +qemuDomainSecretStorageSourcePrepareAuth(qemuDomainObjPrivate *priv, + virStorageSource *src, + const char *alias) +{ + qemuDomainStorageSourcePrivate *srcPriv; + + if (virStorageSourceIsEmpty(src)) + return 0; + + srcPriv =3D qemuDomainStorageSourcePrivateFetch(src); =20 if (src->auth) { virSecretUsageType usageType =3D VIR_SECRET_USAGE_TYPE_ISCSI; @@ -1338,7 +1363,7 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjPri= vate *priv, if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_RBD) usageType =3D VIR_SECRET_USAGE_TYPE_CEPH; =20 - if (!(srcPriv->secinfo =3D qemuDomainSecretInfoSetupFromSecret(pri= v, aliasprotocol, + if (!(srcPriv->secinfo =3D qemuDomainSecretInfoSetupFromSecret(pri= v, alias, "auth= ", usage= Type, src->= auth->username, @@ -1346,19 +1371,10 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjP= rivate *priv, return -1; } =20 - if (hasEnc) { - if (!(srcPriv->encinfo =3D qemuDomainSecretInfoSetupFromSecret(pri= v, aliasformat, - "encr= yption", - VIR_S= ECRET_USAGE_TYPE_VOLUME, - NULL, - &src-= >encryption->secrets[0]->seclookupdef))) - return -1; - } - if (src->ncookies && !(srcPriv->httpcookie =3D qemuDomainSecretStorageSourcePrepareCook= ies(priv, = src, - = aliasprotocol))) + = alias))) return -1; =20 return 0; @@ -10805,9 +10821,12 @@ qemuDomainPrepareDiskSourceLegacy(virDomainDiskDef= *disk, qemuDomainPrepareStorageSourceConfig(disk->src, cfg); qemuDomainPrepareDiskSourceData(disk, disk->src); =20 - if (qemuDomainSecretStorageSourcePrepare(priv, disk->src, - disk->info.alias, - disk->info.alias) < 0) + if (qemuDomainSecretStorageSourcePrepareEncryption(priv, disk->src, + disk->info.alias) <= 0) + return -1; + + if (qemuDomainSecretStorageSourcePrepareAuth(priv, disk->src, + disk->info.alias) < 0) return -1; =20 if (qemuDomainPrepareStorageSourcePR(disk->src, priv, disk->info.alias= ) < 0) @@ -10843,9 +10862,11 @@ qemuDomainPrepareStorageSourceBlockdevNodename(vir= DomainDiskDef *disk, qemuDomainPrepareStorageSourceConfig(src, cfg); qemuDomainPrepareDiskSourceData(disk, src); =20 - if (qemuDomainSecretStorageSourcePrepare(priv, src, - src->nodestorage, - src->nodeformat) < 0) + if (qemuDomainSecretStorageSourcePrepareEncryption(priv, src, + src->nodeformat) < = 0) + return -1; + if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, + src->nodestorage) < 0) return -1; =20 if (qemuDomainPrepareStorageSourcePR(src, priv, src->nodestorage) < 0) --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303215; cv=none; d=zohomail.com; s=zohoarc; b=firRjfPknc287Ppjbl0DhYEbjqnV0M5wfSVD8sWVPnnZoQoTm2x3h+fhaM9s+pWmCEaZDr6GCthXrlBRbqPMJVmoQM70YO5nVh33RiGTDqwdGdM3XUN0FnYP2zQXQ7vqp/kcGeAHgVtQVVfM7un4n0EqdHLK7CvViUYuhr/j5yU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303215; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nadPwdzk9z+bIaWdQ0uBObxpTCIh3dDMahOhz91lpVk=; b=Z4IwClAnPqB0kVCDTuN+ocK0Ux5DhDN+1iXZK5u6QEZGNEI2AInmgHlXKvjbXkj+1UUkazaj2jmFnHYmNwaKK6YSznV+SmfPqBG4EsszJFzWrRldIJY6cefJt3WrLYD5YGtY2Ma7FvAMubTI5YWQN9SjibYpNUvZ84I5g8lkWjM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 166630321569559.51056144277061; Thu, 20 Oct 2022 15:00:15 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-10-XeBE43LnO8GZylCb8iQHcw-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AC449868A33; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9170B1402BDD; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 640C91946A43; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id AFD0B1946587 for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id A5AAE4047A7; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 84AB14047AC for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303214; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=nadPwdzk9z+bIaWdQ0uBObxpTCIh3dDMahOhz91lpVk=; b=U9LO9QVvNAsNS226JgesVvLVBQBilCu3NtALKw6yvg6Xrh9bFSDrGIEuampnj4HEUlkf9e /yWBfpWYacHkKI8a1fOst8BaTexrIqVK4FTO+FGMCWye1yn6/YTNAgu2S+sLn931/4I6Te GYJo0gaAfVMR3gtLIi48Aa6Tarfj3u8= X-MC-Unique: XeBE43LnO8GZylCb8iQHcw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 12/18] qemu: include nbdkit state in private xml Date: Thu, 20 Oct 2022 16:59:03 -0500 Message-Id: <20221020215909.1751428-13-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303216781100009 Content-Type: text/plain; charset="utf-8"; x-default="true" Add xml to the private data for a disk source to represent the nbdkit process so that the state can be re-created if the libvirt daemon is restarted. Format: /path/to/nbdkit.pid /path/to/nbdkit.socket Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_domain.c | 53 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.c | 21 +++++++++++++++++ src/qemu/qemu_nbdkit.h | 5 ++++ 3 files changed, 79 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index ace7ae4c61..2ae87392cb 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1849,6 +1849,34 @@ qemuStorageSourcePrivateDataAssignSecinfo(qemuDomain= SecretInfo **secinfo, } =20 =20 +static int +qemuStorageSourcePrivateDataParseNbdkit(xmlNodePtr node, + xmlXPathContextPtr ctxt, + virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *srcpriv =3D qemuDomainStorageSourcePri= vateFetch(src); + g_autofree char *pidfile =3D NULL; + g_autofree char *socketfile =3D NULL; + VIR_XPATH_NODE_AUTORESTORE(ctxt); + + if (srcpriv->nbdkitProcess) + return 0; + + ctxt->node =3D node; + + if (!(pidfile =3D virXPathString("string(./pidfile)", ctxt))) + return -1; + + if (!(socketfile =3D virXPathString("string(./socketfile)", ctxt))) + return -1; + + if (!(srcpriv->nbdkitProcess =3D qemuNbdkitProcessLoad(src, pidfile, s= ocketfile))) + return -1; + + return 0; +} + + static int qemuStorageSourcePrivateDataParse(xmlXPathContextPtr ctxt, virStorageSource *src) @@ -1859,6 +1887,7 @@ qemuStorageSourcePrivateDataParse(xmlXPathContextPtr = ctxt, g_autofree char *httpcookiealias =3D NULL; g_autofree char *tlskeyalias =3D NULL; g_autofree char *thresholdEventWithIndex =3D NULL; + xmlNodePtr nbdkitnode =3D NULL; =20 src->nodestorage =3D virXPathString("string(./nodenames/nodename[@type= =3D'storage']/@name)", ctxt); src->nodeformat =3D virXPathString("string(./nodenames/nodename[@type= =3D'format']/@name)", ctxt); @@ -1902,6 +1931,10 @@ qemuStorageSourcePrivateDataParse(xmlXPathContextPtr= ctxt, virTristateBoolTypeFromString(thresholdEventWithIndex) =3D=3D VIR_= TRISTATE_BOOL_YES) src->thresholdEventWithIndex =3D true; =20 + if ((nbdkitnode =3D virXPathNode("nbdkit", ctxt))) { + if (qemuStorageSourcePrivateDataParseNbdkit(nbdkitnode, ctxt, src)= < 0) + return -1; + } return 0; } =20 @@ -1919,6 +1952,23 @@ qemuStorageSourcePrivateDataFormatSecinfo(virBuffer = *buf, } =20 =20 +static void +qemuStorageSourcePrivateDataFormatNbdkit(qemuNbdkitProcess *nbdkit, + virBuffer *buf) +{ + g_auto(virBuffer) childBuf =3D VIR_BUFFER_INIT_CHILD(buf); + + if (!nbdkit) + return; + + virBufferEscapeString(&childBuf, "%s\n", + nbdkit->pidfile); + virBufferEscapeString(&childBuf, "%s\n", + nbdkit->socketfile); + virXMLFormatElement(buf, "nbdkit", NULL, &childBuf); +} + + static int qemuStorageSourcePrivateDataFormat(virStorageSource *src, virBuffer *buf) @@ -1957,6 +2007,9 @@ qemuStorageSourcePrivateDataFormat(virStorageSource *= src, if (src->thresholdEventWithIndex) virBufferAddLit(buf, "\n"); =20 + if (srcPriv) + qemuStorageSourcePrivateDataFormatNbdkit(srcPriv->nbdkitProcess, b= uf); + return 0; } =20 diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 59c452a15e..5b47a15112 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -597,6 +597,27 @@ qemuNbdkitProcessNew(virStorageSource *source, } =20 =20 +qemuNbdkitProcess * +qemuNbdkitProcessLoad(virStorageSource *source, + const char *pidfile, + const char *socketfile) +{ + int rc; + g_autoptr(qemuNbdkitProcess) nbdkit =3D qemuNbdkitProcessNew(source, p= idfile, socketfile); + + if ((rc =3D virPidFileReadPath(nbdkit->pidfile, &nbdkit->pid)) < 0) + return NULL; + + if (virProcessKill(nbdkit->pid, 0) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("nbdkit process %i is not alive"), nbdkit->pid); + return NULL; + } + + return g_steal_pointer(&nbdkit); +} + + bool qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, virStorageSource *source, diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 30cab744b0..ca7f1dcf31 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -89,4 +89,9 @@ qemuNbdkitProcessStop(qemuNbdkitProcess *proc); void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); =20 +qemuNbdkitProcess * +qemuNbdkitProcessLoad(virStorageSource *source, + const char *pidfile, + const char *socketfile); + G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuNbdkitProcess, qemuNbdkitProcessFree); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303172; cv=none; d=zohomail.com; s=zohoarc; b=eIF2tZvaRNz2lqUM3JO8uJOlAjADFiTLL0kgOqpNV59dfRLfEtJHXtXdHcLUIoshYqWuDUsGqbW6lIM6X4Ei5WBMYPDpxw7q1cC+J3tannqGjnYgpXHKfJGGfaPZRaY6Wv0XyaStUL8kD7VLpA+cod1abVNpOkQYVL3RxkNB3QE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303172; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wxtsjdzt9mbSCBof1gDsXJkyp6Af7r8VHi2Z7seIXPY=; b=TOzRMu/bdKhqqbj1EXqhzPmIiCYk0Zn7AbqM+LtcKovuu5cbJXJZgEo6qLjy0yyMbL2lFaN9w2RepyftPC4du6hgmswogIqQOteRoyXaGvKmNhGbbPaybbY0IK1tIFKhSE4JNzCRtVl4lcxqsLPiS/lbGJnbXrzO1/5BFtjxqh0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303172016761.736436087554; Thu, 20 Oct 2022 14:59:32 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-15-dqT7L139M-qg9ZzWFxCaBw-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B3F763C0F809; Thu, 20 Oct 2022 21:59:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9EB0B2024CBB; Thu, 20 Oct 2022 21:59:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 59EE51946A62; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1682B1946A44 for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id ED6AC4047AD; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BC7B64047AC for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303170; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=wxtsjdzt9mbSCBof1gDsXJkyp6Af7r8VHi2Z7seIXPY=; b=aXucDn4fYP4/lSmpYP0qu+no+A7RYNWA6DeczODYmHnCyFAdW/mWtqENvWeBYqA87JOOc7 gAMWgiBCTV1GdPHViaeQnmodzMZ2jwDDzIWqkn+qvrqfGmgZkx2AEFeyvdd9Y5MKw3/4yv tUXX1THbeSeFbuulgWMPG7P/eBhsATg= X-MC-Unique: dqT7L139M-qg9ZzWFxCaBw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 13/18] qemu: use nbdkit to serve network disks if available Date: Thu, 20 Oct 2022 16:59:04 -0500 Message-Id: <20221020215909.1751428-14-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303172412100010 Content-Type: text/plain; charset="utf-8"; x-default="true" For virStorageSource objects that contain an nbdkitProcess, start that nbdkit process to serve that network drive and then pass the nbdkit socket to qemu rather than sending the network url to qemu directly. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_block.c | 162 +++++++++++------- src/qemu/qemu_domain.c | 21 ++- src/qemu/qemu_extdevice.c | 48 ++++++ src/qemu/qemu_nbdkit.c | 63 +++++++ src/qemu/qemu_nbdkit.h | 13 ++ ...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 +++++ .../disk-cdrom-network-nbdkit.xml | 1 + ...isk-network-http-nbdkit.x86_64-latest.args | 45 +++++ .../disk-network-http-nbdkit.xml | 1 + ...rce-curl-nbdkit-backing.x86_64-latest.args | 38 ++++ ...isk-network-source-curl-nbdkit-backing.xml | 45 +++++ ...work-source-curl-nbdkit.x86_64-latest.args | 50 ++++++ .../disk-network-source-curl-nbdkit.xml | 1 + ...isk-network-source-curl.x86_64-latest.args | 53 ++++++ .../disk-network-source-curl.xml | 71 ++++++++ ...disk-network-ssh-nbdkit.x86_64-latest.args | 36 ++++ .../disk-network-ssh-nbdkit.xml | 1 + tests/qemuxml2argvtest.c | 6 + 18 files changed, 627 insertions(+), 70 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64= -latest.args create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-= latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-= backing.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-= backing.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.= x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.= xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-= latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-l= atest.args create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index b82e3311e1..224d468799 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -439,6 +439,32 @@ qemuBlockStorageSourceGetCURLProps(virStorageSource *s= rc, } =20 =20 +static virJSONValue * +qemuBlockStorageSourceGetNbdkitProps(virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *srcPriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(src); + virJSONValue *ret =3D NULL; + g_autoptr(virJSONValue) serverprops =3D NULL; + virStorageNetHostDef host =3D { .transport =3D VIR_STORAGE_NET_HOST_TR= ANS_UNIX }; + + /* srcPriv->nbdkitProcess will already be initialized if we can use nb= dkit + * to proxy this storage source */ + if (!(srcPriv && srcPriv->nbdkitProcess)) + return NULL; + + host.socket =3D srcPriv->nbdkitProcess->socketfile; + serverprops =3D qemuBlockStorageSourceBuildJSONSocketAddress(&host); + + if (!serverprops) + return NULL; + + if (virJSONValueObjectAdd(&ret, "a:server", &serverprops, NULL) < 0) + return NULL; + + return ret; +} + + static virJSONValue * qemuBlockStorageSourceGetISCSIProps(virStorageSource *src, bool onlytarget) @@ -851,69 +877,75 @@ qemuBlockStorageSourceGetBackendProps(virStorageSourc= e *src, return NULL; =20 case VIR_STORAGE_TYPE_NETWORK: - switch ((virStorageNetProtocol) src->protocol) { - case VIR_STORAGE_NET_PROTOCOL_GLUSTER: - driver =3D "gluster"; - if (!(fileprops =3D qemuBlockStorageSourceGetGlusterProps(src,= onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_VXHS: - driver =3D "vxhs"; - if (!(fileprops =3D qemuBlockStorageSourceGetVxHSProps(src, on= lytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_HTTP: - case VIR_STORAGE_NET_PROTOCOL_HTTPS: - case VIR_STORAGE_NET_PROTOCOL_FTP: - case VIR_STORAGE_NET_PROTOCOL_FTPS: - case VIR_STORAGE_NET_PROTOCOL_TFTP: - driver =3D virStorageNetProtocolTypeToString(src->protocol); - if (!(fileprops =3D qemuBlockStorageSourceGetCURLProps(src, on= lytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_ISCSI: - driver =3D "iscsi"; - if (!(fileprops =3D qemuBlockStorageSourceGetISCSIProps(src, o= nlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_NBD: + /* prefer using nbdkit for sources that are supported */ + if ((fileprops =3D qemuBlockStorageSourceGetNbdkitProps(src))) { driver =3D "nbd"; - if (!(fileprops =3D qemuBlockStorageSourceGetNBDProps(src, onl= ytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_RBD: - driver =3D "rbd"; - if (!(fileprops =3D qemuBlockStorageSourceGetRBDProps(src, onl= ytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: - driver =3D "sheepdog"; - if (!(fileprops =3D qemuBlockStorageSourceGetSheepdogProps(src= ))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_SSH: - driver =3D "ssh"; - if (!(fileprops =3D qemuBlockStorageSourceGetSshProps(src))) - return NULL; break; - - case VIR_STORAGE_NET_PROTOCOL_NFS: - driver =3D "nfs"; - if (!(fileprops =3D qemuBlockStorageSourceGetNFSProps(src))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_NONE: - case VIR_STORAGE_NET_PROTOCOL_LAST: - virReportEnumRangeError(virStorageNetProtocol, src->protocol); - return NULL; + } else { + switch ((virStorageNetProtocol) src->protocol) { + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + driver =3D "gluster"; + if (!(fileprops =3D qemuBlockStorageSourceGetGlusterPr= ops(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_VXHS: + driver =3D "vxhs"; + if (!(fileprops =3D qemuBlockStorageSourceGetVxHSProps= (src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + driver =3D virStorageNetProtocolTypeToString(src->prot= ocol); + if (!(fileprops =3D qemuBlockStorageSourceGetCURLProps= (src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + driver =3D "iscsi"; + if (!(fileprops =3D qemuBlockStorageSourceGetISCSIProp= s(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NBD: + driver =3D "nbd"; + if (!(fileprops =3D qemuBlockStorageSourceGetNBDProps(= src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_RBD: + driver =3D "rbd"; + if (!(fileprops =3D qemuBlockStorageSourceGetRBDProps(= src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + driver =3D "sheepdog"; + if (!(fileprops =3D qemuBlockStorageSourceGetSheepdogP= rops(src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_SSH: + driver =3D "ssh"; + if (!(fileprops =3D qemuBlockStorageSourceGetSshProps(= src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NFS: + driver =3D "nfs"; + if (!(fileprops =3D qemuBlockStorageSourceGetNFSProps(= src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportEnumRangeError(virStorageNetProtocol, src->pr= otocol); + return NULL; + } } break; } @@ -2196,6 +2228,7 @@ qemuBlockStorageSourceCreateGetStorageProps(virStorag= eSource *src, g_autoptr(virJSONValue) location =3D NULL; const char *driver =3D NULL; const char *filename =3D NULL; + qemuDomainStorageSourcePrivate *srcPriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(src); =20 switch (actualType) { case VIR_STORAGE_TYPE_FILE: @@ -2224,6 +2257,13 @@ qemuBlockStorageSourceCreateGetStorageProps(virStora= geSource *src, break; =20 case VIR_STORAGE_NET_PROTOCOL_SSH: + if (srcPriv->nbdkitProcess) { + /* disk creation not yet supported with nbdkit, and even i= f it + * was supported, it would not be done with blockdev-create + * props */ + return 0; + } + driver =3D "ssh"; if (!(location =3D qemuBlockStorageSourceGetSshProps(src))) return -1; diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 2ae87392cb..a3fafdd9e3 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -10918,21 +10918,24 @@ qemuDomainPrepareStorageSourceBlockdevNodename(vi= rDomainDiskDef *disk, if (qemuDomainSecretStorageSourcePrepareEncryption(priv, src, src->nodeformat) < = 0) return -1; - if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, - src->nodestorage) < 0) - return -1; =20 if (qemuDomainPrepareStorageSourcePR(src, priv, src->nodestorage) < 0) return -1; =20 - if (qemuDomainPrepareStorageSourceTLS(src, cfg, src->nodestorage, - priv) < 0) - return -1; + if (!qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, = priv)) { + /* If we're using nbdkit to serve the storage source, we don't pass + * authentication secrets to qemu, but will pass them to nbdkit in= stead */ + if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, + src->nodestorage) < 0) + return -1; =20 - if (qemuDomainPrepareStorageSourceNFS(src) < 0) - return -1; + if (qemuDomainPrepareStorageSourceTLS(src, cfg, src->nodestorage, + priv) < 0) + return -1; =20 - qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, priv); + if (qemuDomainPrepareStorageSourceNFS(src) < 0) + return -1; + } =20 return 0; } diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c index 24a57b0f74..4a3d082f1e 100644 --- a/src/qemu/qemu_extdevice.c +++ b/src/qemu/qemu_extdevice.c @@ -219,6 +219,17 @@ qemuExtDevicesStart(virQEMUDriver *driver, return -1; } =20 + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + if (qemuNbdkitStartStorageSource(driver, vm, disk->src) < 0) + return -1; + } + + if (def->os.loader && def->os.loader->nvram) { + if (qemuNbdkitStartStorageSource(driver, vm, def->os.loader->nvram= ) < 0) + return -1; + } + return 0; } =20 @@ -263,6 +274,14 @@ qemuExtDevicesStop(virQEMUDriver *driver, fs->fsdriver =3D=3D VIR_DOMAIN_FS_DRIVER_TYPE_VIRTIOFS) qemuVirtioFSStop(driver, vm, fs); } + + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + qemuNbdkitStopStorageSource(disk->src); + } + + if (def->os.loader && def->os.loader->nvram) + qemuNbdkitStopStorageSource(def->os.loader->nvram); } =20 =20 @@ -292,6 +311,24 @@ qemuExtDevicesHasDevice(virDomainDef *def) } =20 =20 +/* recursively setup nbdkit cgroups for backing chain of src */ +static int qemuExtDevicesSetupCgroupNbdkit(virStorageSource *src, + virCgroup *cgroup) +{ + qemuDomainStorageSourcePrivate *priv =3D QEMU_DOMAIN_STORAGE_SOURC= E_PRIVATE(src); + + if (src->backingStore) + if (qemuExtDevicesSetupCgroupNbdkit(src->backingStore, cgroup)= < 0) + return -1; + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessSetupCgroup(priv->nbdkitProcess, cgroup) < 0) + return -1; + + return 0; +} + + int qemuExtDevicesSetupCgroup(virQEMUDriver *driver, virDomainObj *vm, @@ -325,6 +362,17 @@ qemuExtDevicesSetupCgroup(virQEMUDriver *driver, return -1; } =20 + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + if (qemuExtDevicesSetupCgroupNbdkit(disk->src, cgroup) < 0) + return -1; + } + + if (def->os.loader && def->os.loader->nvram) { + if (qemuExtDevicesSetupCgroupNbdkit(def->os.loader->nvram, cgroup)= < 0) + return -1; + } + for (i =3D 0; i < def->nfss; i++) { virDomainFSDef *fs =3D def->fss[i]; =20 diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 5b47a15112..8cb9e0e604 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -672,6 +672,61 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } =20 =20 +static int +qemuNbdkitStartStorageSourceOne(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *priv =3D QEMU_DOMAIN_STORAGE_SOURCE_PR= IVATE(src); + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessStart(priv->nbdkitProcess, vm, driver) < 0) + return -1; + + return 0; +} + + +/* recursively start nbdkit for backing chain of src */ +int +qemuNbdkitStartStorageSource(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src) +{ + virStorageSource *backing; + + for (backing =3D src->backingStore; backing !=3D NULL; backing =3D bac= king->backingStore) + if (qemuNbdkitStartStorageSourceOne(driver, vm, backing) < 0) + return -1; + + return qemuNbdkitStartStorageSourceOne(driver, vm, src); +} + + +static void +qemuNbdkitStopStorageSourceOne(virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *priv =3D QEMU_DOMAIN_STORAGE_SOURCE_PR= IVATE(src); + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessStop(priv->nbdkitProcess) < 0) + VIR_WARN("Unable to stop nbdkit for storage source '%s'", src->nod= estorage); +} + + +/* recursively stop nbdkit processes for backing chain of src */ +void +qemuNbdkitStopStorageSource(virStorageSource *src) +{ + virStorageSource *backing; + + qemuNbdkitStopStorageSourceOne(src); + + for (backing =3D src->backingStore; backing !=3D NULL; backing =3D bac= king->backingStore) + qemuNbdkitStopStorageSourceOne(backing); +} + + static int qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, virCommand *cmd) @@ -839,6 +894,14 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) } =20 =20 +int +qemuNbdkitProcessSetupCgroup(qemuNbdkitProcess *proc, + virCgroup *cgroup) +{ + return virCgroupAddProcess(cgroup, proc->pid); +} + + int qemuNbdkitProcessStart(qemuNbdkitProcess *proc, virDomainObj *vm, diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index ca7f1dcf31..c9af6efcfa 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -23,6 +23,7 @@ =20 #include "internal.h" #include "storage_source_conf.h" +#include "vircgroup.h" #include "virenum.h" #include "virfilecache.h" =20 @@ -56,6 +57,14 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *nbdkitCaps, uid_t user, gid_t group); =20 +int +qemuNbdkitStartStorageSource(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src); + +void +qemuNbdkitStopStorageSource(virStorageSource *src); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); @@ -89,6 +98,10 @@ qemuNbdkitProcessStop(qemuNbdkitProcess *proc); void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); =20 +int +qemuNbdkitProcessSetupCgroup(qemuNbdkitProcess *proc, + virCgroup *cgroup); + qemuNbdkitProcess * qemuNbdkitProcessLoad(virStorageSource *source, const char *pidfile, diff --git a/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest= .args b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..eec7ef2af7 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args @@ -0,0 +1,42 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 1024 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":1073741824}= ' \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw"= ,"file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-3-form= at","id":"ide0-0-0","bootindex":1}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw"= ,"file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-2-form= at","id":"ide0-0-1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw"= ,"file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.1","unit":0,"drive":"libvirt-1-form= at","id":"ide0-1-0"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","add= r":"0x2"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml b/tests/q= emuxml2argvdata/disk-cdrom-network-nbdkit.xml new file mode 120000 index 0000000000..55f677546f --- /dev/null +++ b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml @@ -0,0 +1 @@ +disk-cdrom-network.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.= args b/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..25d476d3ce --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args @@ -0,0 +1,45 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-4-storage.socket"},"node-name":"libvirt-4-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"raw= ","file":"libvirt-4-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-4-format","id":"virtio-disk0","bootindex":1}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":false,"driver":"raw= ","file":"libvirt-3-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-3-format","id":"virtio-disk1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"raw= ","file":"libvirt-2-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-2-format","id":"virtio-disk2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x5","drive":"li= bvirt-1-format","id":"virtio-disk3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-http-nbdkit.xml b/tests/qe= muxml2argvdata/disk-network-http-nbdkit.xml new file mode 120000 index 0000000000..6a05204e8a --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-http-nbdkit.xml @@ -0,0 +1 @@ +disk-network-http.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing= .x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-source-curl-nbdki= t-backing.x86_64-latest.args new file mode 100644 index 0000000000..98cfcd219a --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64= -latest.args @@ -0,0 +1,38 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"qcow= 2","file":"libvirt-2-storage"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"qcow= 2","file":"libvirt-1-storage","backing":"libvirt-2-format"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing= .xml b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml new file mode 100644 index 0000000000..37a30fcbd6 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml @@ -0,0 +1,45 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-x86_64 + + + + + + cookievalue1 + cookievalue2 + + + + + + + + cookievalue3 + cookievalue4 + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-= latest.args b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64= -latest.args new file mode 100644 index 0000000000..ec193bb10a --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.= args @@ -0,0 +1,50 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-device '{"driver":"ahci","id":"sata0","bus":"pci.0","addr":"0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-5-storage.socket"},"node-name":"libvirt-5-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw"= ,"file":"libvirt-5-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","= data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","k= eyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-4-storage.socket"},"node-name":"libvirt-4-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luk= s","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-sto= rage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-4-format","id":"virtio-disk4"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw"= ,"file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.1","drive":"libvirt-3-format","id= ":"sata0-0-1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw"= ,"file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.2","drive":"libvirt-2-format","id= ":"sata0-0-2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw"= ,"file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.3","drive":"libvirt-1-format","id= ":"sata0-0-3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml b/t= ests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml new file mode 120000 index 0000000000..4a1e40bd70 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml @@ -0,0 +1 @@ +disk-network-source-curl.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.= args b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args new file mode 100644 index 0000000000..ec6dd13f6c --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args @@ -0,0 +1,53 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-device '{"driver":"ahci","id":"sata0","bus":"pci.0","addr":"0x2"}' \ +-object '{"qom-type":"secret","id":"libvirt-5-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","= keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}'= \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk1.iso","cookie-secret":"libvirt-5-storage-httpcookie-secret0","node-na= me":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw"= ,"file":"libvirt-5-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","= data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","k= eyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ +-object '{"qom-type":"secret","id":"libvirt-4-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","= keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}'= \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk5.iso?foo=3Dbar","sslverify":false,"cookie-secret":"libvirt-4-storage-= httpcookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"= discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luk= s","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-sto= rage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-4-format","id":"virtio-disk4"}' \ +-object '{"qom-type":"secret","id":"libvirt-3-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBBv7TuTgTkyAyOPpC2P5qLbOIy= pLoHpppjz+u5O+X8oT+jA1m7q/OJQ8dk2EFD5c0A=3D","keyid":"masterKey0","iv":"AAE= CAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ +-blockdev '{"driver":"http","url":"http://http.example.org:8080/path/to/di= sk2.iso","cookie-secret":"libvirt-3-storage-httpcookie-secret0","node-name"= :"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw"= ,"file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.1","drive":"libvirt-3-format","id= ":"sata0-0-1"}' \ +-blockdev '{"driver":"ftp","url":"ftp://ftp.example.org:20/path/to/disk3.i= so","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"= }' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw"= ,"file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.2","drive":"libvirt-2-format","id= ":"sata0-0-2"}' \ +-blockdev '{"driver":"ftps","url":"ftps://ftps.example.org:22/path/to/disk= 4.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unm= ap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw"= ,"file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.3","drive":"libvirt-1-format","id= ":"sata0-0-3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.xml b/tests/qe= muxml2argvdata/disk-network-source-curl.xml new file mode 100644 index 0000000000..1e50314abe --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl.xml @@ -0,0 +1,71 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-x86_64 + + + + + cookievalue1 + cookievalue2 + + + + + + + + + + cookievalue1 + cookievalue2 + cookievalue3 + + + + + + + + + + + + + + + + + + + + + + cookievalue1 + cookievalue2 + + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.a= rgs b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..e22ba095b1 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args @@ -0,0 +1,36 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml b/tests/qem= uxml2argvdata/disk-network-ssh-nbdkit.xml new file mode 120000 index 0000000000..b0589bdfb5 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml @@ -0,0 +1 @@ +disk-network-ssh.xml \ No newline at end of file diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0032bafdce..60ec42428c 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1286,6 +1286,7 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-cdrom-empty-network-invalid"); DO_TEST_CAPS_LATEST("disk-cdrom-bus-other"); DO_TEST_CAPS_LATEST("disk-cdrom-network"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-cdrom-network-nbdkit", QEMU_NBDKIT_CA= PS_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-cdrom-tray"); DO_TEST_CAPS_LATEST("disk-floppy"); DO_TEST_CAPS_LATEST("disk-floppy-q35"); @@ -1326,6 +1327,9 @@ mymain(void) /* qemu-6.0 is the last qemu version supporting sheepdog */ DO_TEST_CAPS_VER("disk-network-sheepdog", "6.0.0"); DO_TEST_CAPS_LATEST("disk-network-source-auth"); + DO_TEST_CAPS_LATEST("disk-network-source-curl"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-source-curl-nbdkit", QEMU_NBD= KIT_CAPS_PLUGIN_CURL); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-source-curl-nbdkit-backing", = QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-network-nfs"); driver.config->vxhsTLS =3D 1; driver.config->nbdTLSx509secretUUID =3D g_strdup("6fd3f62d-9fe7-4a4e-a= 869-7acd6376d8ea"); @@ -1336,7 +1340,9 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-network-tlsx509-nbd-hostname"); DO_TEST_CAPS_VER("disk-network-tlsx509-vxhs", "5.0.0"); DO_TEST_CAPS_LATEST("disk-network-http"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-http-nbdkit", QEMU_NBDKIT_CAP= S_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-network-ssh"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-ssh-nbdkit", QEMU_NBDKIT_CAPS= _PLUGIN_SSH); driver.config->vxhsTLS =3D 0; VIR_FREE(driver.config->vxhsTLSx509certdir); DO_TEST_CAPS_LATEST("disk-no-boot"); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303213; cv=none; d=zohomail.com; s=zohoarc; b=F2Fuy4kXprPCkxTyKg9Pa+Zx6pfYzWZFJocv8ynY+2nKa5Nn7IYXDhckJoHuHePWJ0Chy8p/nbiYzY7DJg/THj4DalYRK2GdcPzg9FsZTLfMXOGElQ2pY67DxIgty40pCF+pkLirfwLhnimQwJBpubblBX2hUqhCgmW0kz6o3JU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303213; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3lV0YQ0HupjBAiFgIxqxk1UGohpTLHdTjV1RzLVK2g0=; b=MK8TabywGJNDDSHYN1ogRQSYUYgXu1m4qfF7/c6ISBWf1L1g1lwkvXbKvCumpdrp8qmpdsxHE0JXgVK9nGKHqnFnFKDwB+6R3m+Rn9yQoorlAJn6LO5b44Zrm0qMm+YdhfhUB+j9ur5jR+JRPP8iK/xjM3TKVg3dWfNeBp07qwE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1666303213547207.67997616714206; Thu, 20 Oct 2022 15:00:13 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-389-YUMUBmtqPIiNXq4llIgSuA-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1CD4C85A5A6; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0A69E40C6EC3; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7806719465A8; Thu, 20 Oct 2022 21:59:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3AD351946A44 for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 2EB994047A7; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0CF534047AA for ; Thu, 20 Oct 2022 21:59:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303211; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3lV0YQ0HupjBAiFgIxqxk1UGohpTLHdTjV1RzLVK2g0=; b=Jb9huxTVlMjA3cTJe0L0M+75uthatlFKFBI4102enJyed6sSbgLBu0+4iLeaHZn2u/PWWR PxSGU2tdKkxzg/pAj1lcgRj26S/MZDcQP2Meh9HJOF0lOTvfYmQAzMAI1IA62qEjjjpyKW mS2SNDh1allv2S4T9iy/ufEH96GF/Y0= X-MC-Unique: YUMUBmtqPIiNXq4llIgSuA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 14/18] tests: add tests for nbdkit invocation Date: Thu, 20 Oct 2022 16:59:05 -0500 Message-Id: <20221020215909.1751428-15-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303214755100004 Content-Type: text/plain; charset="utf-8"; x-default="true" We were testing the arguments that were being passed to qemu when a disk was being served by nbdkit, but the arguments used to start nbdkit itself were not testable. This adds a test to ensure that we're invoking nbdkit correctly for various disk source definitions. For now, expect failure for tests that use sensitive data such as passwords and cookies. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 4 +- src/qemu/qemu_nbdkitpriv.h | 31 +++ tests/meson.build | 1 + .../disk-network-ssh.args.disk0 | 7 + tests/qemunbdkittest.c | 239 ++++++++++++++++++ 5 files changed, 281 insertions(+), 1 deletion(-) create mode 100644 src/qemu/qemu_nbdkitpriv.h create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0 create mode 100644 tests/qemunbdkittest.c diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 8cb9e0e604..882a074211 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -34,6 +34,8 @@ #include "qemu_driver.h" #include "qemu_extdevice.h" #include "qemu_nbdkit.h" +#define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +#include "qemu_nbdkitpriv.h" #include "qemu_security.h" =20 #include @@ -830,7 +832,7 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *pro= c, } =20 =20 -static virCommand * +virCommand * qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) { g_autoptr(virCommand) cmd =3D virCommandNewArgList(proc->caps->path, diff --git a/src/qemu/qemu_nbdkitpriv.h b/src/qemu/qemu_nbdkitpriv.h new file mode 100644 index 0000000000..64f9bb99d8 --- /dev/null +++ b/src/qemu/qemu_nbdkitpriv.h @@ -0,0 +1,31 @@ +/* + * qemu_nbdkitpriv.h: exposing some functions for testing + * + * Copyright (C) 2021 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + */ + +#ifndef LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +# error "qemu_nbdkitpriv.h may only be included by qemu_nbdkit.c or test s= uites" +#endif /* LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW */ + +#pragma once + +#include "qemu_nbdkit.h" + +virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc); diff --git a/tests/meson.build b/tests/meson.build index 1149211756..87b29ab5b4 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -451,6 +451,7 @@ if conf.has('WITH_QEMU') { 'name': 'qemuvhostusertest', 'link_with': [ test_qemu_driver_lib ], = 'link_whole': [ test_file_wrapper_lib ] }, { 'name': 'qemuxml2argvtest', 'link_with': [ test_qemu_driver_lib, tes= t_utils_qemu_monitor_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_= wrapper_lib ] }, { 'name': 'qemuxml2xmltest', 'link_with': [ test_qemu_driver_lib ], 'l= ink_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, + { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'li= nk_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, ] endif =20 diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk0 b/tests/qemun= bdkitdata/disk-network-ssh.args.disk0 new file mode 100644 index 0000000000..e0020dff6a --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk0 @@ -0,0 +1,7 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground ssh \ +host=3Dexample.org \ +port=3D2222 \ +path=3Dtest.img diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c new file mode 100644 index 0000000000..c7fa80b9c5 --- /dev/null +++ b/tests/qemunbdkittest.c @@ -0,0 +1,239 @@ +#include + +#include "internal.h" +#include "testutils.h" +#include "testutilsqemu.h" +#include "qemu/qemu_domain.h" +#include "qemu/qemu_nbdkit.h" +#define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +#include "qemu/qemu_nbdkitpriv.h" +#include "vircommand.h" +#define LIBVIRT_VIRCOMMANDPRIV_H_ALLOW +#include "vircommandpriv.h" +#include "virutil.h" +#include "virsecret.h" +#include "datatypes.h" + +#define VIR_FROM_THIS VIR_FROM_QEMU + +static virQEMUDriver driver; + + +/* Some mock implementations for testing */ +int +virSecretGetSecretString(virConnectPtr conn G_GNUC_UNUSED, + virSecretLookupTypeDef *seclookupdef, + virSecretUsageType secretUsageType, + uint8_t **secret, + size_t *secret_size) +{ + char uuidstr[VIR_UUID_BUFLEN]; + const char *secretname =3D NULL; + char *tmp =3D NULL; + + switch (seclookupdef->type) { + case VIR_SECRET_LOOKUP_TYPE_UUID: + virUUIDFormat(seclookupdef->u.uuid, uuidstr); + secretname =3D uuidstr; + break; + case VIR_SECRET_LOOKUP_TYPE_USAGE: + secretname =3D seclookupdef->u.usage; + break; + case VIR_SECRET_LOOKUP_TYPE_NONE: + case VIR_SECRET_LOOKUP_TYPE_LAST: + default: + virReportEnumRangeError(virSecretLookupType, seclookupdef->typ= e); + return -1; + }; + + /* For testing, just generate a value for the secret that includes the= type + * and the id of the secret */ + tmp =3D g_strdup_printf("%s-%s-secret", virSecretUsageTypeToString(sec= retUsageType), secretname); + *secret =3D (uint8_t*)tmp; + *secret_size =3D strlen(tmp) + 1; + + return 0; +} + +virConnectPtr virGetConnectSecret(void) +{ + return virGetConnect(); +} + +/* end of mock implementations */ + + +typedef struct { + const char *name; + char* infile; + char* outtemplate; + qemuNbdkitCaps *nbdkitcaps; + bool expectFail; +} TestInfo; + + +typedef enum { + NBDKIT_ARG_CAPS, + NBDKIT_ARG_EXPECT_FAIL, + NBDKIT_ARG_END +} NbdkitArgName; + + +static void +testInfoSetPaths(TestInfo *info) +{ + info->infile =3D g_strdup_printf("%s/qemuxml2argvdata/%s.xml", + abs_srcdir, info->name); + info->outtemplate =3D g_strdup_printf("%s/qemunbdkitdata/%s", + abs_srcdir, info->name); +} + +static void +testInfoClear(TestInfo *info) +{ + g_free(info->infile); + g_free(info->outtemplate); + g_clear_object(&info->nbdkitcaps); +} + +static void +testInfoSetArgs(TestInfo *info, ...) +{ + va_list argptr; + NbdkitArgName argname; + unsigned int cap; + + va_start(argptr, info); + while ((argname =3D va_arg(argptr, NbdkitArgName)) !=3D NBDKIT_ARG_END= ) { + switch (argname) { + case NBDKIT_ARG_CAPS: + while ((cap =3D va_arg(argptr, unsigned int)) < QEMU_NBDKI= T_CAPS_LAST) + qemuNbdkitCapsSet(info->nbdkitcaps, cap); + break; + case NBDKIT_ARG_EXPECT_FAIL: + info->expectFail =3D va_arg(argptr, unsigned int); + break; + case NBDKIT_ARG_END: + default: + break; + } + } +} + + +static int +testNbdkit(const void *data) +{ + const TestInfo *info =3D data; + g_autoptr(virDomainDef) def =3D NULL; + size_t i; + int ret =3D 0; + + if (!virFileExists(info->infile)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "Test input file '%s' is missing", info->infile); + return -1; + } + + if (!(def =3D virDomainDefParseFile(info->infile, driver.xmlopt, NULL, + VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE))) + return -1; + + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + g_autofree char *statedir =3D g_strdup_printf("/tmp/statedir-%zi",= i); + g_autofree char *alias =3D g_strdup_printf("test-disk-%zi", i); + g_autofree char *cmdfile =3D g_strdup_printf("%s.args.disk%zi", + info->outtemplate, i); + + if (qemuNbdkitInitStorageSource(info->nbdkitcaps, disk->src, state= dir, + alias, 101, 101)) { + qemuDomainStorageSourcePrivate *srcPriv =3D + qemuDomainStorageSourcePrivateFetch(disk->src); + g_autoptr(virCommand) cmd =3D NULL; + g_autoptr(virCommandDryRunToken) dryRunToken =3D virCommandDry= RunTokenNew(); + g_auto(virBuffer) buf =3D VIR_BUFFER_INITIALIZER; + const char *actualCmdline =3D NULL; + + virCommandSetDryRun(dryRunToken, &buf, true, true, NULL, NULL); + cmd =3D qemuNbdkitProcessBuildCommand(srcPriv->nbdkitProcess); + + if (virCommandRun(cmd, NULL) < 0) { + ret =3D -1; + continue; + } + + if (!(actualCmdline =3D virBufferContentAndReset(&buf))) { + ret =3D -1; + continue; + } + + if (virTestCompareToFileFull(actualCmdline, cmdfile, false) < = 0) { + ret =3D -1; + continue; + } + } else { + if (virFileExists(cmdfile)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + "qemuNbdkitInitStorageSource() was not expe= cted to fail"); + ret =3D -1; + } + } + } + + if (info->expectFail) { + if (ret =3D=3D 0) { + ret =3D -1; + VIR_TEST_DEBUG("Error expected but there wasn't any."); + } else { + ret =3D 0; + } + } + return ret; +} + +static int +mymain(void) +{ + int ret =3D 0; + + if (qemuTestDriverInit(&driver) < 0) + return EXIT_FAILURE; + + +#define DO_TEST_FULL(_name, ...) \ + do { \ + TestInfo info =3D { \ + .name =3D _name, \ + .nbdkitcaps =3D qemuNbdkitCapsNew(TEST_NBDKIT_PATH), \ + }; \ + testInfoSetPaths(&info); \ + testInfoSetArgs(&info, __VA_ARGS__); \ + virTestRunLog(&ret, "nbdkit " _name, testNbdkit, &info); \ + testInfoClear(&info); \ + } while (0) + +#define DO_TEST(_name, ...) \ + DO_TEST_FULL(_name, NBDKIT_ARG_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAS= T, NBDKIT_ARG_END) + +#define DO_TEST_FAILURE(_name, ...) \ + DO_TEST_FULL(_name, \ + NBDKIT_ARG_EXPECT_FAIL, 1, \ + NBDKIT_ARG_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, NBDK= IT_ARG_END) + +#define DO_TEST_NOCAPS(_name) \ + DO_TEST_FULL(_name, NBDKIT_ARG_END) + + /* disks with cookies / passwords are not yet supported */ + DO_TEST_FAILURE("disk-cdrom-network", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST_FAILURE("disk-network-http", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST_FAILURE("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT= _CAPS_PLUGIN_CURL); + DO_TEST_FAILURE("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CU= RL); + DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + + qemuTestDriverFree(&driver); + + return ret =3D=3D 0 ? EXIT_SUCCESS : EXIT_FAILURE; +} + +VIR_TEST_MAIN(mymain) --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303172; cv=none; d=zohomail.com; s=zohoarc; b=CZo2xqiJ8DT2g4ygnb9JaptAlVh8NCnBh/Bj05J/DXajL7BAu72eco0ZftdhBNZ6kxfCuMiv5D7bAVysEkE1gVMHBfye2XjSZQkIPb2HbDYG0SGtdfnEh6GWIfNBqkxZluMxMvXWnUVKHh0EzbCsGCR5vODTR2kP1t8rqjv0LBk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303172; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=F0i1aHmSfyyanFCQkDCSdtQkqoswk4w2jq7KXkjSRPo=; b=N+OoTQVWBjLWG9GJ+Revl3WxjUvmGXoDG5/F83+iFOpnF46gy6UevA1eVeoYeduIca237xV7qUPGcQoiptpAD9Q/tYo1p8Q73byuldTR2Y3rFfiMnp8beTWNEpLzTF3taxnXzTzTuVIEdIs1YDT0vBZ6wIFdmsJW2yOswNjFoPA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1666303172773627.7860311766174; Thu, 20 Oct 2022 14:59:32 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-448-Qo00lCsMNKiflWT43YFILQ-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0AEAD3C10253; Thu, 20 Oct 2022 21:59:21 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id E48244A927D; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 806A719465BA; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 77CC81946595 for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 6A1B64047B0; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 41B6D4047AC for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303171; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=F0i1aHmSfyyanFCQkDCSdtQkqoswk4w2jq7KXkjSRPo=; b=DT+HeX24qp8SmRngy2hVQ8Xhs5+Gea8bos35Ld90JA0w6aKvhw7CRZuDhy1ScwA6HLxOrZ H/3e9G+rGhnwoR+CtgZBuW0tSSb4QhqqYO/zn8mZF7OuhmQnSDmycMuJZ3mDfTvBgZ7HId wD0i/+m3a15+ThihsDVO216UmzmXF7w= X-MC-Unique: Qo00lCsMNKiflWT43YFILQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 15/18] util: make virCommandSetSendBuffer testable Date: Thu, 20 Oct 2022 16:59:06 -0500 Message-Id: <20221020215909.1751428-16-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303174505100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Add a private function to peek at the list of send buffers in virCommand so that it is testable Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/libvirt_private.syms | 1 + src/util/vircommand.c | 16 ++++++++-------- src/util/vircommand.h | 8 ++++++++ src/util/vircommandpriv.h | 4 ++++ 4 files changed, 21 insertions(+), 8 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0b0ccbafe5..bf24d65b08 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2080,6 +2080,7 @@ virCommandNewArgs; virCommandNewVAList; virCommandNonblockingFDs; virCommandPassFD; +virCommandPeekSendBuffers; virCommandRawStatus; virCommandRequireHandshake; virCommandRun; diff --git a/src/util/vircommand.c b/src/util/vircommand.c index bbfbe19706..014bab9196 100644 --- a/src/util/vircommand.c +++ b/src/util/vircommand.c @@ -77,14 +77,6 @@ struct _virCommandFD { unsigned int flags; }; =20 -typedef struct _virCommandSendBuffer virCommandSendBuffer; -struct _virCommandSendBuffer { - int fd; - unsigned char *buffer; - size_t buflen; - off_t offset; -}; - struct _virCommand { int has_error; /* 0 on success, -1 on error */ =20 @@ -3451,3 +3443,11 @@ virCommandSetRunAmong(virCommand *cmd, =20 cmd->schedCore =3D pid; } + +void virCommandPeekSendBuffers(virCommand *cmd, + virCommandSendBuffer **buffers, + int *nbuffers) +{ + *buffers =3D cmd->sendBuffers; + *nbuffers =3D cmd->numSendBuffers; +} diff --git a/src/util/vircommand.h b/src/util/vircommand.h index 98788bcbf7..fc2ca0f3ff 100644 --- a/src/util/vircommand.h +++ b/src/util/vircommand.h @@ -24,6 +24,14 @@ #include "internal.h" #include "virbuffer.h" =20 +typedef struct _virCommandSendBuffer virCommandSendBuffer; +struct _virCommandSendBuffer { + int fd; + unsigned char *buffer; + size_t buflen; + off_t offset; +}; + typedef struct _virCommand virCommand; =20 /* This will execute in the context of the first child diff --git a/src/util/vircommandpriv.h b/src/util/vircommandpriv.h index ff17fa5ded..d579810bb5 100644 --- a/src/util/vircommandpriv.h +++ b/src/util/vircommandpriv.h @@ -47,3 +47,7 @@ void virCommandSetDryRun(virCommandDryRunToken *tok, bool bufCommandStripPath, virCommandDryRunCallback cb, void *opaque); + +void virCommandPeekSendBuffers(virCommand *cmd, + virCommandSendBuffer **buffers, + int *nbuffers); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303204; cv=none; d=zohomail.com; s=zohoarc; b=IH7Gbj5ufMa15+lmGj1Yu8L5qqD9nsjHXL2o3wzyRNbZCGkStS9GdOHHoGtGFqfp0RRKkg0C1urLy3ywAQW4SaIcLSxHss/2h4nhH6XEsyiO3SdhzAHv2UtjyHfKHqW/DkhzEMsszedJZ6R6uZDqbmbY6k0sFfJeINxbsQOSFNE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303204; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0fwzLRzJNBF/P+q01W+aKtddXbLG+dGE6cMHcJIF1pY=; b=NrcTJ/5NF0yKkl/sXvx4tIqNA6V9eFlUHTbex10Nd2ewLOTrUDbPeczJn92DnzmOR7V/F5XqXr5S5CaH5AJXFdnl7e6pevODEnrLGp7BwlpdGhgJTfJmUOHdhpdKhcZvIb9plkGlglvzXNr8rMgw977jR16yQyMjdOdpnSFQ9Ng= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1666303204813928.1282631346884; Thu, 20 Oct 2022 15:00:04 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-554-nH6xOg-CPAKYTi8XI1iKAg-1; Thu, 20 Oct 2022 17:59:30 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0661B1C199C6; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id E44501401C23; Thu, 20 Oct 2022 21:59:22 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id CE89A1946595; Thu, 20 Oct 2022 21:59:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id AAFFD1946597 for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id A1DD94047AA; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7646B4047A7 for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303202; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=0fwzLRzJNBF/P+q01W+aKtddXbLG+dGE6cMHcJIF1pY=; b=E0Z4tu/BsARiu+6GFxyl/dQpMCwCHVtKMrpceGbGs1whnoNWoR6s521bYu8+dDYDWLwZUL Fgx3QWjlYwlXiQpf7i5eFe4STPhJHJ+KyzSl75qgSWOkUmFaShT+ExxMPm2Sye28CnWACB Zx6fOqeruUIifspCbUNgW/sP5PYbg1E= X-MC-Unique: nH6xOg-CPAKYTi8XI1iKAg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 16/18] qemu: pass sensitive data to nbdkit via pipe Date: Thu, 20 Oct 2022 16:59:07 -0500 Message-Id: <20221020215909.1751428-17-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303206745100003 Content-Type: text/plain; charset="utf-8"; x-default="true" Rather than passing passwords and cookies (which could contain passwords) to nbdkit via commandline arguments, use the alternate format that nbdkit supports where we can specify a file descriptor which nbdkit will read to get the password or cookies. Signed-off-by: Jonathon Jongsma --- build-aux/syntax-check.mk | 4 +- src/qemu/qemu_nbdkit.c | 64 ++++++++++++----- src/util/vircommand.c | 3 +- src/util/virutil.h | 2 +- .../disk-cdrom-network.args.disk0 | 7 ++ .../disk-cdrom-network.args.disk1 | 9 +++ .../disk-cdrom-network.args.disk1.pipe.1778 | 1 + .../disk-cdrom-network.args.disk2 | 9 +++ .../disk-cdrom-network.args.disk2.pipe.1780 | 1 + .../disk-network-http.args.disk0 | 7 ++ .../disk-network-http.args.disk1 | 6 ++ .../disk-network-http.args.disk2 | 7 ++ .../disk-network-http.args.disk2.pipe.1778 | 1 + .../disk-network-http.args.disk3 | 8 +++ .../disk-network-http.args.disk3.pipe.1780 | 1 + ...work-source-curl-nbdkit-backing.args.disk0 | 8 +++ ...e-curl-nbdkit-backing.args.disk0.pipe.1778 | 1 + .../disk-network-source-curl.args.disk0 | 8 +++ ...k-network-source-curl.args.disk0.pipe.1778 | 1 + .../disk-network-source-curl.args.disk1 | 8 +++ ...k-network-source-curl.args.disk1.pipe.1780 | 1 + .../disk-network-source-curl.args.disk2 | 8 +++ ...k-network-source-curl.args.disk2.pipe.1782 | 1 + .../disk-network-source-curl.args.disk3 | 7 ++ .../disk-network-source-curl.args.disk4 | 7 ++ tests/qemunbdkittest.c | 69 +++++++++++++++++-- 26 files changed, 219 insertions(+), 30 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe= .1778 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe= .1780 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.= 1778 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.= 1780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-ba= cking.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-ba= cking.args.disk0.pipe.1778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 0.pipe.1778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 1.pipe.1780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 2.pipe.1782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4 diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 68cd9dff5f..d44b1e5b17 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -1355,10 +1355,10 @@ exclude_file_name_regexp--sc_prohibit_strdup =3D \ ^(docs/|examples/|tests/virnetserverclientmock.c|tests/commandhelper.c|t= ools/nss/libvirt_nss_(leases|macs)\.c$$) =20 exclude_file_name_regexp--sc_prohibit_close =3D \ - (\.p[yl]$$|\.spec\.in$$|^docs/|^(src/util/vir(file|event)\.c|src/libvirt= -stream\.c|tests/(vir.+mock\.c|commandhelper\.c|qemusecuritymock\.c)|tools/= nss/libvirt_nss_(leases|macs)\.c)|tools/virt-qemu-qmp-proxy$$) + (\.p[yl]$$|\.spec\.in$$|^docs/|^(src/util/vir(file|event)\.c|src/libvirt= -stream\.c|tests/(vir.+mock\.c|commandhelper\.c|qemusecuritymock\.c|qemunbd= kittest\.c)|tools/nss/libvirt_nss_(leases|macs)\.c)|tools/virt-qemu-qmp-pro= xy$$) =20 exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF =3D \ - (^tests/(nodedevmdevctl|virhostcpu|virpcitest|virstoragetest)data/|docs/= js/.*\.js|docs/fonts/.*\.woff|\.diff|tests/virconfdata/no-newline\.conf$$) + (^tests/(nodedevmdevctl|virhostcpu|virpcitest|virstoragetest|qemunbdkit)= data/|docs/js/.*\.js|docs/fonts/.*\.woff|\.diff|tests/virconfdata/no-newlin= e\.conf$$) =20 exclude_file_name_regexp--sc_prohibit_fork_wrappers =3D \ (^(src/(util/(vircommand|virdaemon)|lxc/lxc_controller)|tests/testutils)= \.c$$) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 882a074211..0a0dc5d2a4 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -55,6 +55,7 @@ VIR_ENUM_IMPL(qemuNbdkitCaps, "filter-readahead", /* QEMU_NBDKIT_CAPS_FILTER_READAHEAD */ ); =20 + struct _qemuNbdkitCaps { GObject parent; =20 @@ -71,6 +72,12 @@ struct _qemuNbdkitCaps { G_DEFINE_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, G_TYPE_OBJECT); =20 =20 +enum { + PIPE_FD_READ =3D 0, + PIPE_FD_WRITE =3D 1 +}; + + static void qemuNbdkitCheckCommandCap(qemuNbdkitCaps *nbdkit, virCommand *cmd, @@ -729,6 +736,29 @@ qemuNbdkitStopStorageSource(virStorageSource *src) } =20 =20 +static int +qemuNbdkitCommandPassDataByPipe(virCommand *cmd, + const char *argName, + unsigned char *buf, + size_t buflen) +{ + g_autofree char *fdfmt =3D NULL; + int fd =3D virCommandSetSendBuffer(cmd, buf, buflen); + + if (fd < 0) + return -1; + + /* some nbdkit arguments accept a variation where nbdkit will read the= data + * from a file descriptor, e.g. password=3D-FD */ + fdfmt =3D g_strdup_printf("-%i", fd); + virCommandAddArgPair(cmd, argName, fdfmt); + + virCommandDoAsyncIO(cmd); + + return 0; +} + + static int qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, virCommand *cmd) @@ -744,10 +774,10 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *= proc, =20 if (proc->source->auth) { g_autoptr(virConnect) conn =3D virGetConnectSecret(); - g_autofree uint8_t *secret =3D NULL; + uint8_t *secret =3D NULL; size_t secretlen =3D 0; - g_autofree char *password =3D NULL; int secrettype; + virStorageAuthDef *authdef =3D proc->source->auth; =20 virCommandAddArgPair(cmd, "user", proc->source->auth->username); @@ -760,7 +790,7 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *pr= oc, } =20 if (virSecretGetSecretString(conn, - &proc->source->auth->seclookupdef, + &authdef->seclookupdef, secrettype, &secret, &secretlen) < 0) { @@ -769,24 +799,20 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *= proc, return -1; } =20 - /* ensure that the secret is a NULL-terminated string */ - password =3D g_strndup((char*)secret, secretlen); - - /* for now, just report an error rather than passing the password = in - * cleartext on the commandline */ - virReportError(VIR_ERR_INTERNAL_ERROR, - "%s", - "Password not yet supported for nbdkit sources"); - return -1; + if (qemuNbdkitCommandPassDataByPipe(cmd, "password", + secret, secretlen) < 0) + return -1; } =20 - if (proc->source->ncookies > 0) { - /* for now, just report an error rather than passing cookies in - * cleartext on the commandline */ - virReportError(VIR_ERR_INTERNAL_ERROR, - "%s", - "Cookies not yet supported for nbdkit sources"); - return -1; + /* Create a pipe to send the cookies to the nbdkit process. */ + if (proc->source->ncookies) { + char *cookies =3D + qemuBlockStorageSourceGetCookieString(proc->source); + + if (qemuNbdkitCommandPassDataByPipe(cmd, "cookie", + (unsigned char*)cookies, + strlen(cookies)) < 0) + return -1; } =20 if (proc->source->sslverify =3D=3D VIR_TRISTATE_BOOL_NO) { diff --git a/src/util/vircommand.c b/src/util/vircommand.c index 014bab9196..838eb6bd16 100644 --- a/src/util/vircommand.c +++ b/src/util/vircommand.c @@ -1703,7 +1703,8 @@ virCommandSetSendBuffer(virCommand *cmd, return -1; } =20 - if (fcntl(pipefd[1], F_SETFL, O_NONBLOCK) < 0) { + if (!(dryRunBuffer || dryRunCallback) && + fcntl(pipefd[1], F_SETFL, O_NONBLOCK) < 0) { cmd->has_error =3D errno; VIR_FORCE_CLOSE(pipefd[0]); VIR_FORCE_CLOSE(pipefd[1]); diff --git a/src/util/virutil.h b/src/util/virutil.h index ab8511bf8d..094b399859 100644 --- a/src/util/virutil.h +++ b/src/util/virutil.h @@ -186,7 +186,7 @@ char *virGetPassword(void); * * Returns: -1 on error, 0 on success */ -int virPipe(int fds[2]); +int virPipe(int fds[2]) G_NO_INLINE; =20 /* * virPipeQuiet: diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk0 b/tests/qem= unbdkitdata/disk-cdrom-network.args.disk0 new file mode 100644 index 0000000000..5f3a795ba0 --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk0 @@ -0,0 +1,7 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftp \ +url=3Dftp://host.name:21/url/path/file.iso diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk1 b/tests/qem= unbdkitdata/disk-cdrom-network.args.disk1 new file mode 100644 index 0000000000..12c0dcaf0e --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1 @@ -0,0 +1,9 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftps \ +url=3Dftps://host.name:990/url/path/file.iso \ +user=3Dtestuser \ +password=3D-1777 diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.1778 b= /tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.1778 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.1778 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk2 b/tests/qem= unbdkitdata/disk-cdrom-network.args.disk2 new file mode 100644 index 0000000000..d41337a089 --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2 @@ -0,0 +1,9 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttps \ +'url=3Dhttps://host.name:443/url/path/file.iso?test=3Dval' \ +user=3Dtestuser \ +password=3D-1779 diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.1780 b= /tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.1780 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.1780 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk0 b/tests/qemu= nbdkitdata/disk-network-http.args.disk0 new file mode 100644 index 0000000000..fa8ef90cd1 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk0 @@ -0,0 +1,7 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground curl \ +protocols=3Dhttp \ +url=3Dhttp://example.org:80/test.img \ +timeout=3D1234 diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk1 b/tests/qemu= nbdkitdata/disk-network-http.args.disk1 new file mode 100644 index 0000000000..9bac3fe229 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk1 @@ -0,0 +1,6 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground curl \ +protocols=3Dhttps \ +url=3Dhttps://example.org:443/test2.img diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk2 b/tests/qemu= nbdkitdata/disk-network-http.args.disk2 new file mode 100644 index 0000000000..2d39b6c259 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk2 @@ -0,0 +1,7 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground curl \ +protocols=3Dhttp \ +url=3Dhttp://example.org:1234/test3.img \ +cookie=3D-1777 diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.1778 b/= tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.1778 new file mode 100644 index 0000000000..2c42c95930 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.1778 @@ -0,0 +1 @@ +test=3Dtestcookievalue; test2=3D"blurb" \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk3 b/tests/qemu= nbdkitdata/disk-network-http.args.disk3 new file mode 100644 index 0000000000..54f12f5c9e --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk3 @@ -0,0 +1,8 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-3/nbdkit-test-disk-3.socket \ +--foreground curl \ +protocols=3Dhttps \ +'url=3Dhttps://example.org:1234/test4.img?par=3Dval&other=3Dble' \ +cookie=3D-1779 \ +sslverify=3Dfalse diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.1780 b/= tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.1780 new file mode 100644 index 0000000000..2c42c95930 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.1780 @@ -0,0 +1 @@ +test=3Dtestcookievalue; test2=3D"blurb" \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.a= rgs.disk0 b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.ar= gs.disk0 new file mode 100644 index 0000000000..eb479b996f --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.dis= k0 @@ -0,0 +1,8 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttps \ +url=3Dhttps://https.example.org:8443/path/to/disk1.qcow2 \ +cookie=3D-1777 diff --git a/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.a= rgs.disk0.pipe.1778 b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-= backing.args.disk0.pipe.1778 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.dis= k0.pipe.1778 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk0 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk0 new file mode 100644 index 0000000000..cf2c0b7184 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0 @@ -0,0 +1,8 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttps \ +url=3Dhttps://https.example.org:8443/path/to/disk1.iso \ +cookie=3D-1777 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.= 1778 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.1778 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.1778 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk1 new file mode 100644 index 0000000000..13f03c545e --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 @@ -0,0 +1,8 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground curl \ +protocols=3Dhttps \ +'url=3Dhttps://https.example.org:8443/path/to/disk5.iso?foo=3Dbar' \ +cookie=3D-1779 \ +sslverify=3Dfalse diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.= 1780 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1780 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1780 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk2 new file mode 100644 index 0000000000..490aea3393 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 @@ -0,0 +1,8 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttp \ +url=3Dhttp://http.example.org:8080/path/to/disk2.iso \ +cookie=3D-1781 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.= 1782 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.1782 new file mode 100644 index 0000000000..5c035e84c5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.1782 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2; cookie3=3Dcookievalue3 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk3 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk3 new file mode 100644 index 0000000000..bc28f04564 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk3 @@ -0,0 +1,7 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-3/nbdkit-test-disk-3.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftp \ +url=3Dftp://ftp.example.org:20/path/to/disk3.iso diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk4 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk4 new file mode 100644 index 0000000000..7c3cc711ae --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk4 @@ -0,0 +1,7 @@ +nbdkit \ +--exit-with-parent \ +--unix /tmp/statedir-4/nbdkit-test-disk-4.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftps \ +url=3Dftps://ftps.example.org:22/path/to/disk4.iso diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c index c7fa80b9c5..49888ab8a1 100644 --- a/tests/qemunbdkittest.c +++ b/tests/qemunbdkittest.c @@ -1,5 +1,6 @@ #include =20 +#include #include "internal.h" #include "testutils.h" #include "testutilsqemu.h" @@ -13,6 +14,7 @@ #include "virutil.h" #include "virsecret.h" #include "datatypes.h" +#include "virmock.h" =20 #define VIR_FROM_THIS VIR_FROM_QEMU =20 @@ -20,6 +22,45 @@ static virQEMUDriver driver; =20 =20 /* Some mock implementations for testing */ +#define PIPE_FD_START 1777 +static int mockpipefd =3D PIPE_FD_START; +int +virPipeQuiet(int fds[2]) +{ + fds[0] =3D mockpipefd++; + fds[1] =3D mockpipefd++; + + if (fcntl(fds[0], F_GETFD) !=3D -1 || + fcntl(fds[1], F_GETFD) !=3D -1) + abort(); + + return 0; +} + +static int (*real_close)(int fd); +static void +init_syms(void) +{ + VIR_MOCK_REAL_INIT(close); +} + +int +close(int fd) +{ + int ret; + + init_syms(); + + if (fd >=3D PIPE_FD_START) + ret =3D 0; + else + ret =3D real_close(fd); + + return ret; +} + + + int virSecretGetSecretString(virConnectPtr conn G_GNUC_UNUSED, virSecretLookupTypeDef *seclookupdef, @@ -129,6 +170,9 @@ testNbdkit(const void *data) size_t i; int ret =3D 0; =20 + /* restart mock pipe fds so tests are consistent */ + mockpipefd =3D PIPE_FD_START; + if (!virFileExists(info->infile)) { virReportError(VIR_ERR_INTERNAL_ERROR, "Test input file '%s' is missing", info->infile); @@ -154,6 +198,9 @@ testNbdkit(const void *data) g_autoptr(virCommandDryRunToken) dryRunToken =3D virCommandDry= RunTokenNew(); g_auto(virBuffer) buf =3D VIR_BUFFER_INITIALIZER; const char *actualCmdline =3D NULL; + virCommandSendBuffer *sendbuffers; + int nsendbuffers; + size_t j; =20 virCommandSetDryRun(dryRunToken, &buf, true, true, NULL, NULL); cmd =3D qemuNbdkitProcessBuildCommand(srcPriv->nbdkitProcess); @@ -162,15 +209,24 @@ testNbdkit(const void *data) ret =3D -1; continue; } + virCommandPeekSendBuffers(cmd, &sendbuffers, &nsendbuffers); =20 if (!(actualCmdline =3D virBufferContentAndReset(&buf))) { ret =3D -1; continue; } =20 - if (virTestCompareToFileFull(actualCmdline, cmdfile, false) < = 0) { + if (virTestCompareToFileFull(actualCmdline, cmdfile, false) < = 0) ret =3D -1; - continue; + + for (j =3D 0; j < nsendbuffers; j++) { + virCommandSendBuffer *buffer =3D &sendbuffers[j]; + g_autofree char *pipefile =3D g_strdup_printf("%s.pipe.%i", + cmdfile, + buffer->fd); + + if (virTestCompareToFile((const char*)buffer->buffer, pipe= file) < 0) + ret =3D -1; } } else { if (virFileExists(cmdfile)) { @@ -224,11 +280,10 @@ mymain(void) #define DO_TEST_NOCAPS(_name) \ DO_TEST_FULL(_name, NBDKIT_ARG_END) =20 - /* disks with cookies / passwords are not yet supported */ - DO_TEST_FAILURE("disk-cdrom-network", QEMU_NBDKIT_CAPS_PLUGIN_CURL); - DO_TEST_FAILURE("disk-network-http", QEMU_NBDKIT_CAPS_PLUGIN_CURL); - DO_TEST_FAILURE("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT= _CAPS_PLUGIN_CURL); - DO_TEST_FAILURE("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CU= RL); + DO_TEST("disk-cdrom-network", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-http", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT_CAPS_PL= UGIN_CURL); + DO_TEST("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); =20 qemuTestDriverFree(&driver); --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303212; cv=none; d=zohomail.com; s=zohoarc; b=WD/qeYDHMb67J+Paw+MSSXvQGmHkjbhwaJHSxA/RJI/R8ZxvnVBZ0bBdsf1OZ2YBjKuL4nOTg8h4KS46Q/9xUg+N+UhLafa9IpVhCLMWiidkrWiya0RRKHTuyzIy/SoxTvFgTPjg/8Cho/FCqh3s9qTNtw/hy5AlsdERjPm+PGA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303212; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MByktizcum7ERXSEZGEaYskZPYfuUA++gFywmluG0Ts=; b=M/6hHUICDVfENqm8Au3JgcyUH+xSx4ZuPDTIAi4geLm7OVmNa21bjzPy5Sntsb6lulUyLeQ9JTSK4LrhuVIStekDQhRxr3BRjrq66QQ4/f0xqr6bAZ32dlwNOcV8+Jhuu1DEEtL8XBt4RjFq+euP7P7H39THOwBqUdeusvkkdQg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1666303212929765.2334610776471; Thu, 20 Oct 2022 15:00:12 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-629-bUmQXWWkOAqtoKQROmBemw-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C73BC85CD2E; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A31A840CA429; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 35B3B1946A4C; Thu, 20 Oct 2022 21:59:20 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 02CE11946A61 for ; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id D4EE84B4015; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B42064047AA for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303210; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=MByktizcum7ERXSEZGEaYskZPYfuUA++gFywmluG0Ts=; b=eF9SnJVn0iTmyfLwOry4GfrsOmgNcuLQNBEmrloHIL43IReNDEdCCrGz/xPq4mnJxC2mYI hRDBj7pfwhDglOLIzKVS63KLJLPcaCIW+2ljZVGGFoV0ZfEzw/NygtL56wcDjXZ3S1Kar9 dJ52MUYZFbTnXb4g1He18q8OlUQ/uj8= X-MC-Unique: bUmQXWWkOAqtoKQROmBemw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 17/18] qemu: add test for authenticating a https network disk Date: Thu, 20 Oct 2022 16:59:08 -0500 Message-Id: <20221020215909.1751428-18-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303214774100005 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- tests/qemunbdkitdata/disk-network-source-curl.args.1.pipe.1 | 1 + tests/qemunbdkitdata/disk-network-source-curl.args.disk1 | 4 +++- .../disk-network-source-curl.args.disk1.pipe.1780 | 2 +- .../disk-network-source-curl.args.disk1.pipe.1782 | 1 + .../disk-network-source-curl.args.disk1.pipe.49 | 1 + tests/qemunbdkitdata/disk-network-source-curl.args.disk2 | 2 +- .../disk-network-source-curl.args.disk2.pipe.1784 | 1 + .../disk-network-source-curl.args.disk2.pipe.51 | 1 + .../disk-network-source-curl.x86_64-latest.args | 3 ++- tests/qemuxml2argvdata/disk-network-source-curl.xml | 3 +++ 10 files changed, 15 insertions(+), 4 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.1.pi= pe.1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 1.pipe.1782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 1.pipe.49 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 2.pipe.1784 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 2.pipe.51 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.1.pipe.1 b/= tests/qemunbdkitdata/disk-network-source-curl.args.1.pipe.1 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.1.pipe.1 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk1 index 13f03c545e..4b6eef8a86 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 @@ -4,5 +4,7 @@ nbdkit \ --foreground curl \ protocols=3Dhttps \ 'url=3Dhttps://https.example.org:8443/path/to/disk5.iso?foo=3Dbar' \ -cookie=3D-1779 \ +user=3Dmyname \ +password=3D-1779 \ +cookie=3D-1781 \ sslverify=3Dfalse diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.= 1780 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1780 index 20af4ae383..ccdd4033fc 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1780 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1780 @@ -1 +1 @@ -cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.= 1782 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1782 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1782 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.= 49 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.49 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.49 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk2 index 490aea3393..c950eaf6ae 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 @@ -5,4 +5,4 @@ nbdkit \ --readonly curl \ protocols=3Dhttp \ url=3Dhttp://http.example.org:8080/path/to/disk2.iso \ -cookie=3D-1781 +cookie=3D-1783 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.= 1784 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.1784 new file mode 100644 index 0000000000..5c035e84c5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.1784 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2; cookie3=3Dcookievalue3 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.= 51 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.51 new file mode 100644 index 0000000000..5c035e84c5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.51 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2; cookie3=3Dcookievalue3 \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.= args b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args index ec6dd13f6c..7f09e84227 100644 --- a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args +++ b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args @@ -33,9 +33,10 @@ XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ -blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk1.iso","cookie-secret":"libvirt-5-storage-httpcookie-secret0","node-na= me":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw"= ,"file":"libvirt-5-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-storage-auth-secret0","data"= :"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","keyid"= :"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ -object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","= data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","k= eyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ -object '{"qom-type":"secret","id":"libvirt-4-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","= keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}'= \ --blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk5.iso?foo=3Dbar","sslverify":false,"cookie-secret":"libvirt-4-storage-= httpcookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"= discard":"unmap"}' \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk5.iso?foo=3Dbar","username":"myname","password-secret":"libvirt-4-stor= age-auth-secret0","sslverify":false,"cookie-secret":"libvirt-4-storage-http= cookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"disc= ard":"unmap"}' \ -blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luk= s","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-sto= rage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-4-format","id":"virtio-disk4"}' \ -object '{"qom-type":"secret","id":"libvirt-3-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBBv7TuTgTkyAyOPpC2P5qLbOIy= pLoHpppjz+u5O+X8oT+jA1m7q/OJQ8dk2EFD5c0A=3D","keyid":"masterKey0","iv":"AAE= CAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.xml b/tests/qe= muxml2argvdata/disk-network-source-curl.xml index 1e50314abe..8c3982cd73 100644 --- a/tests/qemuxml2argvdata/disk-network-source-curl.xml +++ b/tests/qemuxml2argvdata/disk-network-source-curl.xml @@ -59,6 +59,9 @@ + + + --=20 2.37.3 From nobody Fri May 3 07:01:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1666303171; cv=none; d=zohomail.com; s=zohoarc; b=B3/5k+odFxN9XTEI9WKQPbW4DwJ9c6P1oCkvt/ffjrN62zsOYFexMFjfTNJFhXO2tkuDSbAs2lKzTapzI7Zbt/8bcc4jp11RCIFftL3+8aVss1OZoeMf5WYi8ZIOFwYg1ZcJj1MnpXNSUHraFDHm+3e34zehBXL8mB0+244pyb4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666303171; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=19L3m+z2u9t+do6fvjkkYVTxsFmokBradwsAjqUYjvk=; b=TTiwFZ0XIwv2PjghRHYvODekwM6ecIHu4zg1B5ZpKfhYKaXMZW/zYPl3aPxnxG0aIz13iHwgZgtK4u0jL6WIG9Rh4M3Jiy358zn+/ny21YJ9/BuNSi3GhQFTRQrgvjfpTqjjSFk/EGWhBUMQiFZhBTuv4+Z6gSDVE3tHg+mwMZg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 166630317117551.349172317906664; Thu, 20 Oct 2022 14:59:31 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-530-XWhfDA73MZ6v5SxUinBNmw-1; Thu, 20 Oct 2022 17:59:27 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F3BC73C106B3; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id D963317593; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id B3B6D19465B2; Thu, 20 Oct 2022 21:59:23 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3135B1946A61 for ; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 16FA24047AA; Thu, 20 Oct 2022 21:59:17 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.111]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E96494047A7 for ; Thu, 20 Oct 2022 21:59:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666303169; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=19L3m+z2u9t+do6fvjkkYVTxsFmokBradwsAjqUYjvk=; b=TdMwxuUwtgl0R505RZ9ULsMq2PU4IUYLr6jH27yYAOJyZmot+BHj+vM1CHjeddcp77jOOl bdBajEQVVSGpN2H78VziwUr9Yyj8o0jJUwrtID0T04mMw8QyNWFOvWroLRkYhiapmCrosy g0UzGjzVZkp1aqqv903BAEKZdwJg9y8= X-MC-Unique: XWhfDA73MZ6v5SxUinBNmw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v3 18/18] qemu: Monitor nbdkit process for exit Date: Thu, 20 Oct 2022 16:59:09 -0500 Message-Id: <20221020215909.1751428-19-jjongsma@redhat.com> In-Reply-To: <20221020215909.1751428-1-jjongsma@redhat.com> References: <20221020215909.1751428-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1666303172398100008 Content-Type: text/plain; charset="utf-8"; x-default="true" Adds the ability to monitor the nbdkit process so that we can take action in case the child exits unexpectedly. When the nbdkit process exits, we pause the vm, restart nbdkit, and then resume the vm. This allows the vm to continue working in the event of a nbdkit failure. Eventually we may want to generalize this functionality since we may need something similar for e.g. qemu-storage-daemon, etc. The process is monitored with the pidfd_open() syscall if it exists (since linux 5.3). Otherwise it resorts to checking whether the process is alive once a second. The one-second time period was chosen somewhat arbitrarily. Signed-off-by: Jonathon Jongsma --- meson.build | 3 + src/qemu/qemu_nbdkit.c | 220 ++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ src/qemu/qemu_process.c | 13 +++ 4 files changed, 246 insertions(+) diff --git a/meson.build b/meson.build index e4581e74dd..b4ed170ca1 100644 --- a/meson.build +++ b/meson.build @@ -686,6 +686,9 @@ if host_machine.system() =3D=3D 'linux' # Check if we have new enough kernel to support BPF devices for cgroup= s v2 [ 'linux/bpf.h', 'BPF_PROG_QUERY' ], [ 'linux/bpf.h', 'BPF_CGROUP_DEVICE' ], + + # process management + [ 'sys/syscall.h', 'SYS_pidfd_open' ], ] endif =20 diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 0a0dc5d2a4..f17fe022ec 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -21,9 +21,11 @@ =20 #include #include +#include =20 #include "vircommand.h" #include "virerror.h" +#include "virevent.h" #include "virlog.h" #include "virpidfile.h" #include "virtime.h" @@ -36,6 +38,7 @@ #include "qemu_nbdkit.h" #define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW #include "qemu_nbdkitpriv.h" +#include "qemu_process.h" #include "qemu_security.h" =20 #include @@ -72,6 +75,13 @@ struct _qemuNbdkitCaps { G_DEFINE_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, G_TYPE_OBJECT); =20 =20 +struct _qemuNbdkitProcessPrivate { + int monitor; + virQEMUDriver *driver; + virDomainObj *vm; +}; + + enum { PIPE_FD_READ =3D 0, PIPE_FD_WRITE =3D 1 @@ -588,6 +598,168 @@ qemuNbdkitCapsCacheNew(const char *cachedir) } =20 =20 +static int +qemuNbdkitProcessStartMonitor(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver); + + +static void +qemuNbdkitProcessHandleExit(qemuNbdkitProcess *proc) +{ + qemuNbdkitProcessPrivate *priv =3D proc->priv; + bool was_running =3D false; + + VIR_DEBUG("nbdkit process %i died", proc->pid); + + /* clean up resources associated with process */ + qemuNbdkitProcessStop(proc); + + if (!(priv->vm && priv->driver)) { + VIR_WARN("Unable to restart nbdkit -- vm and driver not set"); + return; + } + + VIR_DEBUG("restarting nbdkit process"); + + virObjectLock(priv->vm); + if (virDomainObjBeginJob(priv->vm, VIR_JOB_SUSPEND) < 0) { + VIR_WARN("can't begin job"); + goto cleanup; + } + + /* Pause domain */ + if (virDomainObjGetState(priv->vm, NULL) =3D=3D VIR_DOMAIN_RUNNING) { + was_running =3D true; + if (qemuProcessStopCPUs(priv->driver, priv->vm, + VIR_DOMAIN_PAUSED_IOERROR, + VIR_ASYNC_JOB_NONE) < 0) + goto endjob; + VIR_DEBUG("Paused vm while we restart nbdkit backend"); + } + + if (qemuNbdkitProcessStart(proc, priv->vm, priv->driver) < 0) + VIR_WARN("Unable to restart nbkdit process"); + + if (was_running && virDomainObjIsActive(priv->vm)) { + if (qemuProcessStartCPUs(priv->driver, priv->vm, + VIR_DOMAIN_RUNNING_UNPAUSED, + VIR_ASYNC_JOB_NONE) < 0) { + VIR_WARN("Unable to resume guest CPUs after nbdkit restart"); + goto endjob; + } + VIR_DEBUG("Resumed vm"); + } + qemuNbdkitProcessStartMonitor(proc, NULL, NULL); + + endjob: + virDomainObjEndJob(priv->vm); + cleanup: + virObjectUnlock(priv->vm); +} + + +#if WITH_DECL_SYS_PIDFD_OPEN +static void +qemuNbdkitProcessPidfdCb(int watch G_GNUC_UNUSED, + int fd, + int events G_GNUC_UNUSED, + void *opaque) +{ + qemuNbdkitProcess *proc =3D opaque; + + VIR_FORCE_CLOSE(fd); + qemuNbdkitProcessHandleExit(proc); +} +#else +static void +qemuNbdkitProcessTimeoutCb(int timer G_GNUC_UNUSED, + void *opaque) +{ + qemuNbdkitProcess *proc =3D opaque; + + if (virProcessKill(proc->pid, 0) < 0) + qemuNbdkitProcessHandleExit(proc); +} +#endif /* WITH_DECL_SYS_PIDFD_OPEN */ + + +static int +qemuNbdkitProcessStartMonitor(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver) +{ + qemuNbdkitProcessPrivate *priv =3D proc->priv; +#if WITH_DECL_SYS_PIDFD_OPEN + int pidfd; +#endif + + if (vm) { + virObjectRef(vm); + + if (priv->vm) + virObjectUnref(priv->vm); + + priv->vm =3D vm; + } + + if (driver) + priv->driver =3D driver; + + if (!(priv->vm && priv->driver)) { + VIR_WARN("set vm and driver before calling %s", G_STRFUNC); + return -1; + } + +#if WITH_DECL_SYS_PIDFD_OPEN + pidfd =3D syscall(SYS_pidfd_open, proc->pid, 0); + if (pidfd < 0) + return -1; + + priv->monitor =3D virEventAddHandle(pidfd, + VIR_EVENT_HANDLE_READABLE, + qemuNbdkitProcessPidfdCb, + proc, NULL); +#else + /* fall back to checking once a second */ + priv->monitor =3D virEventAddTimeout(1000, + qemuNbdkitProcessTimeoutCb, + proc, NULL); +#endif /* WITH_DECL_SYS_PIDFD_OPEN */ + + if (priv->monitor < 0) + return -1; + + VIR_DEBUG("Monitoring nbdkit process %i for exit", proc->pid); + + return 0; +} + + +static void +qemuNbdkitProcessStopMonitor(qemuNbdkitProcess *proc) +{ + qemuNbdkitProcessPrivate *priv =3D proc->priv; + + if (priv->monitor > 0) { +#if WITH_DECL_SYS_PIDFD_OPEN + virEventRemoveHandle(priv->monitor); +#else + virEventRemoveTimeout(priv->monitor); +#endif /* WITH_DECL_SYS_PIDFD_OPEN */ + priv->monitor =3D 0; + } +} + + +static void +qemuNbdkitProcessPrivateFree(qemuNbdkitProcessPrivate *priv) +{ + virObjectUnref(priv->vm); + g_free(priv); +} + + static qemuNbdkitProcess * qemuNbdkitProcessNew(virStorageSource *source, const char *pidfile, @@ -601,6 +773,7 @@ qemuNbdkitProcessNew(virStorageSource *source, nbdkit->pid =3D -1; nbdkit->pidfile =3D g_strdup(pidfile); nbdkit->socketfile =3D g_strdup(socketfile); + nbdkit->priv =3D g_new0(qemuNbdkitProcessPrivate, 1); =20 return nbdkit; } @@ -627,6 +800,45 @@ qemuNbdkitProcessLoad(virStorageSource *source, } =20 =20 +static int +qemuNbdkitStorageSourceManageProcessOne(virStorageSource *src, + virDomainObj *vm, + virQEMUDriver *driver) +{ + qemuDomainStorageSourcePrivate *srcPriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(src); + qemuNbdkitProcess *nbdkit; + + if (!srcPriv) + return 0; + + nbdkit =3D srcPriv->nbdkitProcess; + if (nbdkit) { + nbdkit->caps =3D qemuGetNbdkitCaps(nbdkit->priv->driver); + + if (qemuNbdkitProcessStartMonitor(nbdkit, vm, driver) < 0) + return -1; + } + + return 0; +} + + +int +qemuNbdkitStorageSourceManageProcess(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src) +{ + virStorageSource *backing; + + for (backing =3D src->backingStore; backing !=3D NULL; backing =3D bac= king->backingStore) { + if (qemuNbdkitStorageSourceManageProcessOne(backing, vm, driver) <= 0) + return -1; + } + + return qemuNbdkitStorageSourceManageProcessOne(src, vm, driver); +} + + bool qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, virStorageSource *source, @@ -915,9 +1127,12 @@ qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { + qemuNbdkitProcessStopMonitor(proc); + g_clear_pointer(&proc->pidfile, g_free); g_clear_pointer(&proc->socketfile, g_free); g_clear_object(&proc->caps); + g_clear_pointer(&proc->priv, qemuNbdkitProcessPrivateFree); g_free(proc); } =20 @@ -988,6 +1203,9 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, goto error; } =20 + if (qemuNbdkitProcessStartMonitor(proc, vm, driver) < 0) + goto error; + return 0; =20 error: @@ -1007,6 +1225,8 @@ qemuNbdkitProcessStop(qemuNbdkitProcess *proc) { int ret; =20 + qemuNbdkitProcessStopMonitor(proc); + if (proc->pid < 0) return 0; =20 diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index c9af6efcfa..da53138d13 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -65,6 +65,11 @@ qemuNbdkitStartStorageSource(virQEMUDriver *driver, void qemuNbdkitStopStorageSource(virStorageSource *src); =20 +int +qemuNbdkitStorageSourceManageProcess(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); @@ -76,6 +81,8 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, #define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, = GObject); =20 +typedef struct _qemuNbdkitProcessPrivate qemuNbdkitProcessPrivate; + struct _qemuNbdkitProcess { qemuNbdkitCaps *caps; virStorageSource *source; @@ -85,6 +92,8 @@ struct _qemuNbdkitProcess { uid_t user; gid_t group; pid_t pid; + + qemuNbdkitProcessPrivate *priv; }; =20 int @@ -107,4 +116,5 @@ qemuNbdkitProcessLoad(virStorageSource *source, const char *pidfile, const char *socketfile); =20 + G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuNbdkitProcess, qemuNbdkitProcessFree); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index f405326312..43e828d42f 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -9009,6 +9009,19 @@ qemuProcessReconnect(void *opaque) } } =20 + for (i =3D 0; i < obj->def->ndisks; i++) { + virDomainDiskDef *disk =3D obj->def->disks[i]; + if (qemuNbdkitStorageSourceManageProcess(driver, obj, disk->src) <= 0) + goto error; + } + + if (obj->def->os.loader && obj->def->os.loader->nvram) { + if (qemuNbdkitStorageSourceManageProcess(driver, obj, + obj->def->os.loader->nvr= am) < 0) + goto error; + } + + /* update domain state XML with possibly updated state in virDomainObj= */ if (virDomainObjSave(obj, driver->xmlopt, cfg->stateDir) < 0) goto error; --=20 2.37.3