From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252244; cv=none; d=zohomail.com; s=zohoarc; b=H4PXwq3r5Wv0VUvD8wJN9xx74N105i28gto6V216/yjw7oeJvx7Rs+qkDxSd6GssurTM55BHBinQOw+cuLhPOyVMPdP3koaNiW5gtdUTOWwOLfh1+hgkFJ/vbATj2dt4P324ne/6REbmZsq49/fALu74LpJNwYuCpDMKTx0Z+Xo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252244; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tKmZ+rIPAyXCja1beqG/ty7dU1tk7Hqn2e3ZQ4OusC4=; b=nYbjHx1tPuiWiywIxzTp72EFMXimhr1uQjSyJaOQO0DgVk0QUHxQk2+oPaVapFyT3rPTHjvgnb24Drhtxvycw1I37AxHNA2lrWab7Mnm669WVg+tP0EL+ZMobJx+E3l7UUsiZ4b9qfT0qvwtyx5WXPaPI9Xb+gOoDd9496lkvFI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252244366523.8431270268761; Fri, 20 Jan 2023 14:04:04 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-533-Ru_xV2-SM1i8j-6isE62Zw-1; Fri, 20 Jan 2023 17:03:35 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5D3211802D4E; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4972540AE1E9; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 460911947040; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 33C2019465B6 for ; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 07FA41415113; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DB3461415108 for ; Fri, 20 Jan 2023 22:03:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252243; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=tKmZ+rIPAyXCja1beqG/ty7dU1tk7Hqn2e3ZQ4OusC4=; b=XrwAK2JdM+MZ16P+DL8js5V30NYTM8ZB/GYy5eLMN0jjnLlzXF+HyMUcgYsKce7VqTFjSK eayacwnBIIU8DvfDKTw3ObCBrvR0dQt1TMFRSyhUHlWmazYiF+Xwoh1xrqMBkth+YR0O22 XChRLCoFNaclWoOlFz2/F+vlzpezJ2Y= X-MC-Unique: Ru_xV2-SM1i8j-6isE62Zw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 01/31] schema: allow 'ssh' as a protocol for network disks Date: Fri, 20 Jan 2023 16:02:55 -0600 Message-Id: <20230120220325.1015090-2-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252245832100002 Content-Type: text/plain; charset="utf-8"; x-default="true" There was support in the code for parsing protocol=3D'ssh' on network disk sources, but it was not present in the xml schema. Add this to the schema and mention it in the documentation. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/conf/schemas/domaincommon.rng | 1 + tests/qemublocktest.c | 2 +- ...w2-invalid.json =3D> network-ssh-qcow2.json} | 0 ...cow2-invalid.xml =3D> network-ssh-qcow2.xml} | 0 .../disk-network-ssh.x86_64-latest.args | 36 +++++++++++++++++++ tests/qemuxml2argvdata/disk-network-ssh.xml | 31 ++++++++++++++++ tests/qemuxml2argvtest.c | 1 + 7 files changed, 70 insertions(+), 1 deletion(-) rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.json= =3D> network-ssh-qcow2.json} (100%) rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.xml = =3D> network-ssh-qcow2.xml} (100%) create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.a= rgs create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index 6cb0a20e1e..f1068c2272 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2155,6 +2155,7 @@ sheepdog tftp + ssh diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c index 010b52f4b3..1c1013d4d9 100644 --- a/tests/qemublocktest.c +++ b/tests/qemublocktest.c @@ -1211,7 +1211,7 @@ mymain(void) =20 TEST_IMAGE_CREATE("network-gluster-qcow2", NULL); TEST_IMAGE_CREATE("network-rbd-qcow2", NULL); - TEST_IMAGE_CREATE("network-ssh-qcow2-invalid", NULL); + TEST_IMAGE_CREATE("network-ssh-qcow2", NULL); =20 #define TEST_BITMAP_DETECT(testname) \ do { \ diff --git a/tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.= json b/tests/qemublocktestdata/imagecreate/network-ssh-qcow2.json similarity index 100% rename from tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.j= son rename to tests/qemublocktestdata/imagecreate/network-ssh-qcow2.json diff --git a/tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.= xml b/tests/qemublocktestdata/imagecreate/network-ssh-qcow2.xml similarity index 100% rename from tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.x= ml rename to tests/qemublocktestdata/imagecreate/network-ssh-qcow2.xml diff --git a/tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args b/t= ests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args new file mode 100644 index 0000000000..045474724b --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args @@ -0,0 +1,36 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"ssh","path":"test.img","server":{"host":"example.org= ","port":"2222"},"node-name":"libvirt-1-storage","auto-read-only":true,"dis= card":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-ssh.xml b/tests/qemuxml2ar= gvdata/disk-network-ssh.xml new file mode 100644 index 0000000000..355add4fea --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh.xml @@ -0,0 +1,31 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + + + + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 8c52feb83c..f3f57c44d0 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1301,6 +1301,7 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-network-tlsx509-nbd-hostname"); DO_TEST_CAPS_VER("disk-network-tlsx509-vxhs", "5.0.0"); DO_TEST_CAPS_LATEST("disk-network-http"); + DO_TEST_CAPS_LATEST("disk-network-ssh"); driver.config->vxhsTLS =3D 0; VIR_FREE(driver.config->vxhsTLSx509certdir); DO_TEST_CAPS_LATEST("disk-no-boot"); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252219; cv=none; d=zohomail.com; s=zohoarc; b=iKgczdDY7RWv2biG4eyG0G3FyLE1LtxHbaq/bjT3WHPrWkYwhYa9LRDk+cTMMixlXdZTIVqPLf9DGXs2qgxOaPLrfTwDPJA7+2B5yoeuGBSdr7B5gQhd/VbVYH66W3I/0dxNQV7769trBDty8cRzE4gvCIQAke7y1RnpNQ7G7Cc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252219; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3opJzx3NO3JMyQdxP6h0CWt1zvCrriJJBQ84Ii/Ggq8=; b=AplSHnpwNbbizOB1/5bIJdqQeAW4Qh63j7iAXPU7QOIShuzXvWn+7FrMKWck5T1Y2PWdUi944/Uff1Zuo7TixhTxbkQ9tOHwklzzxBVNuLGWbFFI3W5EZetSafiN8+O0xyDdMjdg1gsLqFzqb88yav7JQYi5Il56DL4jEl9NGyA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252219699934.6121567638473; Fri, 20 Jan 2023 14:03:39 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-606-CBJyA00gPH2ST0U47QvBzg-1; Fri, 20 Jan 2023 17:03:37 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E84C7100F90C; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id C9A162027004; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A2BE41946A7D; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7906D1946588 for ; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 4F4A1140EBF6; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1AA051415108; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252218; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3opJzx3NO3JMyQdxP6h0CWt1zvCrriJJBQ84Ii/Ggq8=; b=hT2osS81GCgqySa7YvOsVsSnrRkj2sfbgAIx/tczsea+YG4Xqaq8byymNDTAjtQJ/336FK 6qam2gkJrsSp/YN/S+a7Bo06Av66OkIJWgwMri3bP5W/L0rYpkMhi+t+jZkta7H0G0dJYB 45YsZV4VerDsQgeCLUj2pkVmgEGYS4Y= X-MC-Unique: CBJyA00gPH2ST0U47QvBzg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 02/31] qemu: Add functions for determining nbdkit availability Date: Fri, 20 Jan 2023 16:02:56 -0600 Message-Id: <20230120220325.1015090-3-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252221680100001 Content-Type: text/plain; charset="utf-8"; x-default="true" In future commits, we will optionally use nbdkit to serve some remote disk sources. This patch queries to see whether nbdkit is installed on the host and queries it for capabilities. The data will be used in later commits. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/meson.build | 1 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_nbdkit.c | 201 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 50 ++++++++++ 4 files changed, 253 insertions(+) create mode 100644 src/qemu/qemu_nbdkit.c create mode 100644 src/qemu/qemu_nbdkit.h diff --git a/src/qemu/meson.build b/src/qemu/meson.build index c8806bbc36..9be6996195 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -28,6 +28,7 @@ qemu_driver_sources =3D [ 'qemu_monitor_json.c', 'qemu_monitor_text.c', 'qemu_namespace.c', + 'qemu_nbdkit.c', 'qemu_passt.c', 'qemu_process.c', 'qemu_qapi.c', diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 11c740d28f..4f610d86a1 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -36,6 +36,7 @@ #include "virthreadpool.h" #include "locking/lock_manager.h" #include "qemu_capabilities.h" +#include "qemu_nbdkit.h" #include "virclosecallbacks.h" #include "virhostdev.h" #include "virfile.h" diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c new file mode 100644 index 0000000000..9ff293698d --- /dev/null +++ b/src/qemu/qemu_nbdkit.c @@ -0,0 +1,201 @@ +/* + * qemu_nbdkit.c: helpers for using nbdkit + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + */ + +#include +#include + +#include "vircommand.h" +#include "virerror.h" +#include "virlog.h" +#include "virpidfile.h" +#include "virutil.h" +#include "qemu_block.h" +#include "qemu_conf.h" +#include "qemu_domain.h" +#include "qemu_driver.h" +#include "qemu_extdevice.h" +#include "qemu_nbdkit.h" +#include "qemu_security.h" + +#include + +#define VIR_FROM_THIS VIR_FROM_QEMU + +VIR_LOG_INIT("qemu.nbdkit"); + +VIR_ENUM_IMPL(qemuNbdkitCaps, + QEMU_NBDKIT_CAPS_LAST, + /* 0 */ + "plugin-curl", /* QEMU_NBDKIT_CAPS_PLUGIN_CURL */ + "plugin-ssh", /* QEMU_NBDKIT_CAPS_PLUGIN_SSH */ + "filter-readahead", /* QEMU_NBDKIT_CAPS_FILTER_READAHEAD */ +); + +struct _qemuNbdkitCaps { + GObject parent; + + char *path; + char *version; + + virBitmap *flags; +}; +G_DEFINE_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, G_TYPE_OBJECT); + + +static void +qemuNbdkitCheckCommandCap(qemuNbdkitCaps *nbdkit, + virCommand *cmd, + qemuNbdkitCapsFlags cap) +{ + if (virCommandRun(cmd, NULL) !=3D 0) + return; + + VIR_DEBUG("Setting nbdkit capability %i", cap); + ignore_value(virBitmapSetBit(nbdkit->flags, cap)); +} + + +static void +qemuNbdkitQueryFilter(qemuNbdkitCaps *nbdkit, + const char *filter, + qemuNbdkitCapsFlags cap) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(nbdkit->path, + "--version", + NULL); + + virCommandAddArgPair(cmd, "--filter", filter); + + /* use null plugin to check for filter */ + virCommandAddArg(cmd, "null"); + + qemuNbdkitCheckCommandCap(nbdkit, cmd, cap); +} + + +static void +qemuNbdkitQueryPlugin(qemuNbdkitCaps *nbdkit, + const char *plugin, + qemuNbdkitCapsFlags cap) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(nbdkit->path, + plugin, + "--version", + NULL); + + qemuNbdkitCheckCommandCap(nbdkit, cmd, cap); +} + + +static void +qemuNbdkitCapsQueryPlugins(qemuNbdkitCaps *nbdkit) +{ + qemuNbdkitQueryPlugin(nbdkit, "curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + qemuNbdkitQueryPlugin(nbdkit, "ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); +} + + +static void +qemuNbdkitCapsQueryFilters(qemuNbdkitCaps *nbdkit) +{ + qemuNbdkitQueryFilter(nbdkit, "readahead", + QEMU_NBDKIT_CAPS_FILTER_READAHEAD); +} + + +static int +qemuNbdkitCapsQueryVersion(qemuNbdkitCaps *nbdkit) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(nbdkit->path, + "--version", + NULL); + + virCommandSetOutputBuffer(cmd, &nbdkit->version); + + if (virCommandRun(cmd, NULL) !=3D 0) + return -1; + + VIR_DEBUG("Got nbdkit version %s", nbdkit->version); + return 0; +} + + +static void +qemuNbdkitCapsFinalize(GObject *object) +{ + qemuNbdkitCaps *nbdkit =3D QEMU_NBDKIT_CAPS(object); + + g_clear_pointer(&nbdkit->path, g_free); + g_clear_pointer(&nbdkit->version, g_free); + g_clear_pointer(&nbdkit->flags, virBitmapFree); + + G_OBJECT_CLASS(qemu_nbdkit_caps_parent_class)->finalize(object); +} + + +void +qemu_nbdkit_caps_init(qemuNbdkitCaps *caps) +{ + caps->flags =3D virBitmapNew(QEMU_NBDKIT_CAPS_LAST); + caps->version =3D NULL; +} + + +static void +qemu_nbdkit_caps_class_init(qemuNbdkitCapsClass *klass) +{ + GObjectClass *obj =3D G_OBJECT_CLASS(klass); + + obj->finalize =3D qemuNbdkitCapsFinalize; +} + + +qemuNbdkitCaps * +qemuNbdkitCapsNew(const char *path) +{ + qemuNbdkitCaps *caps =3D g_object_new(QEMU_TYPE_NBDKIT_CAPS, NULL); + caps->path =3D g_strdup(path); + + return caps; +} + + +G_GNUC_UNUSED static void +qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) +{ + qemuNbdkitCapsQueryPlugins(caps); + qemuNbdkitCapsQueryFilters(caps); + qemuNbdkitCapsQueryVersion(caps); +} + + +bool +qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag) +{ + return virBitmapIsBitSet(nbdkitCaps->flags, flag); +} + + +void +qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag) +{ + ignore_value(virBitmapSetBit(nbdkitCaps->flags, flag)); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h new file mode 100644 index 0000000000..e191e1fdb4 --- /dev/null +++ b/src/qemu/qemu_nbdkit.h @@ -0,0 +1,50 @@ +/* + * qemu_nbdkit.h: helpers for using nbdkit + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + */ + +#pragma once + +#include "internal.h" +#include "virenum.h" + +typedef struct _qemuNbdkitCaps qemuNbdkitCaps; + +typedef enum { + /* 0 */ + QEMU_NBDKIT_CAPS_PLUGIN_CURL, + QEMU_NBDKIT_CAPS_PLUGIN_SSH, + QEMU_NBDKIT_CAPS_FILTER_READAHEAD, + + QEMU_NBDKIT_CAPS_LAST, +} qemuNbdkitCapsFlags; + +VIR_ENUM_DECL(qemuNbdkitCaps); + +qemuNbdkitCaps * +qemuNbdkitCapsNew(const char *path); + +bool +qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag); + +void +qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag); + +#define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() +G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, = GObject); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252365; cv=none; d=zohomail.com; s=zohoarc; b=i5cbmZl4x9bjoPvtj8FJRPz9y0ViBsCfsdXPCT3EwNRt+BhHBBDlz38TZpTIf5Wg+mvTL8jiAgELindAczOjatQkkrRa3s56Kz4pFwZMXxBaYhEhmIWJFyHHLz9BiIqqhHBZQt0A+EL1H+ey2L8pC5+MX/qeR2cTP8OdcHzbRRY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252365; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=E34y9a43pcAVc5vQP+wQpEqBk7qNiln1kJnmkdR3BWs=; b=dtQNXZf37RSYs3akwMj0OLS/43I6YIh6IXKoCECQ5VxmGTMTWC2tIA1X34harrzk+OAbWalo5ydh46FkIWc6ZAWz/GsIXe9+itMI7/X5WF8nnkc+RVnfGp9nlr3l3hEQ42Sx8JgfaxWAVwiyAUNWrFYOSGSFe2VozQFRRbvdEPE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252365970902.1278677296184; Fri, 20 Jan 2023 14:06:05 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-651-CtSXfl8VPEqRT1zP8_WL4Q-1; Fri, 20 Jan 2023 17:03:46 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E9BCA830F5C; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id D3599422A4; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 176B41946A77; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A90251946588 for ; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 8C2F6140EBF6; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 61DFB14171C0; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252364; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=E34y9a43pcAVc5vQP+wQpEqBk7qNiln1kJnmkdR3BWs=; b=MC5kntqShsKEZyMxbUFFVxvdcsq+Zju4OSiD5Wmyh0h3YrD77X8gg7wgUEpup6Xdn24fEm +9rmVuwoPF6QxCY2dZrxzObsPYuodUv6gIhIKqUjpj7mtNEPVwO14cE8GRSd0NVpanW69G JKTmbzKTaKceZKZPagmXQKW24+t/xCU= X-MC-Unique: CtSXfl8VPEqRT1zP8_WL4Q-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 03/31] qemu: expand nbdkit capabilities Date: Fri, 20 Jan 2023 16:02:57 -0600 Message-Id: <20230120220325.1015090-4-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252367846100001 Content-Type: text/plain; charset="utf-8"; x-default="true" In order to add caching of the nbdkit capabilities, we will need to compare against file modification times, etc. So look up this information when creating the nbdkit caps. Add a nbdkit_moddir build option to allow the builder to specify the location to look for nbdkit plugins and filters. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- meson.build | 6 ++++++ meson_options.txt | 1 + src/qemu/qemu_nbdkit.c | 40 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 47 insertions(+) diff --git a/meson.build b/meson.build index e498b49be4..ca02fde91b 100644 --- a/meson.build +++ b/meson.build @@ -1664,6 +1664,12 @@ if not get_option('driver_qemu').disabled() qemu_dbus_daemon_path =3D '/usr/bin/dbus-daemon' endif conf.set_quoted('QEMU_DBUS_DAEMON', qemu_dbus_daemon_path) + + nbdkit_moddir =3D get_option('nbdkit_moddir') + if nbdkit_moddir =3D=3D '' + nbdkit_moddir =3D libdir / 'nbdkit' + endif + conf.set_quoted('NBDKIT_MODDIR', nbdkit_moddir) endif endif =20 diff --git a/meson_options.txt b/meson_options.txt index 861c5577d2..d5ea4376e0 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -71,6 +71,7 @@ option('driver_vbox', type: 'feature', value: 'auto', des= cription: 'VirtualBox X option('vbox_xpcomc_dir', type: 'string', value: '', description: 'Locatio= n of directory containing VirtualBox XPCOMC library') option('driver_vmware', type: 'feature', value: 'auto', description: 'VMwa= re driver') option('driver_vz', type: 'feature', value: 'auto', description: 'Virtuozz= o driver') +option('nbdkit_moddir', type: 'string', value: '', description: 'set the d= irectory where nbdkit modules are located') =20 option('secdriver_apparmor', type: 'feature', value: 'auto', description: = 'use AppArmor security driver') option('apparmor_profiles', type: 'feature', value: 'auto', description: '= install apparmor profiles') diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 9ff293698d..bb0e76ecbc 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -39,6 +39,9 @@ =20 VIR_LOG_INIT("qemu.nbdkit"); =20 +#define NBDKIT_PLUGINDIR NBDKIT_MODDIR "/plugins" +#define NBDKIT_FILTERDIR NBDKIT_MODDIR "/filters" + VIR_ENUM_IMPL(qemuNbdkitCaps, QEMU_NBDKIT_CAPS_LAST, /* 0 */ @@ -52,6 +55,11 @@ struct _qemuNbdkitCaps { =20 char *path; char *version; + time_t ctime; + time_t libvirtCtime; + time_t pluginDirMtime; + time_t filterDirMtime; + unsigned int libvirtVersion; =20 virBitmap *flags; }; @@ -176,9 +184,41 @@ qemuNbdkitCapsNew(const char *path) } =20 =20 +static time_t +qemuNbdkitGetDirMtime(const char *moddir) +{ + struct stat st; + + if (stat(moddir, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit module directory '%s': %s", + moddir, + g_strerror(errno)); + return 0; + } + + return st.st_mtime; +} + + G_GNUC_UNUSED static void qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) { + struct stat st; + + if (stat(caps->path, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit binary '%s': %s", + caps->path, + g_strerror(errno)); + caps->ctime =3D 0; + return; + } + + caps->ctime =3D st.st_ctime; + caps->filterDirMtime =3D qemuNbdkitGetDirMtime(NBDKIT_FILTERDIR); + caps->pluginDirMtime =3D qemuNbdkitGetDirMtime(NBDKIT_PLUGINDIR); + caps->libvirtCtime =3D virGetSelfLastChanged(); + caps->libvirtVersion =3D LIBVIR_VERSION_NUMBER; + qemuNbdkitCapsQueryPlugins(caps); qemuNbdkitCapsQueryFilters(caps); qemuNbdkitCapsQueryVersion(caps); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252253; cv=none; d=zohomail.com; s=zohoarc; b=E8Guiu7OiAkIIp1goeKNlTIke7oFv4XCQ1wFj2kLQF8qJFfgCWrE7SHzZXhI1Mu9xAKcIUWG+Y45VT0XapwonKl85lwWUzEy+yrUnIAHNKWny5tEvJn4120KIqByC/Lzm3O6Gtnj8W4jjky5iJ/FzJ+1c0XllG2Xlx6pu8kWRmU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252253; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=M8KL39MkGeXRYvMRqXMOSqLHJ/ovbw1SvI5Ix/8SW8M=; b=HcsAHdElJJllV1qb43aNZ2sZXesMWCICAwUVharg/ZcnmbBH0Of+K1wWqJeDVeVHVYQUs1SS70jNYyoL/mMEF90c9WxfYGZycsejfVIB7a34lilXkgM7HZuHBhK+MgMicdFARpmSW40+kShAL3U0TzLSVVaG8QqdPMyNX85VTzo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252253054553.8965377598194; Fri, 20 Jan 2023 14:04:13 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-583-d15v1ziuOt-n_qon3Qe5Rg-1; Fri, 20 Jan 2023 17:03:43 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C331D100F92B; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A718B112132E; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7E8AE1946A6E; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E32191946588 for ; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id C7B67140EBF6; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9DE1C14171C0; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252252; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=M8KL39MkGeXRYvMRqXMOSqLHJ/ovbw1SvI5Ix/8SW8M=; b=G+P+dqApmGoY3LOdleBZ9/5WgOiDrNRF2kpVmtc6oKgKgPZxYrOxLxAaVP7dO6T3w6m9Zd 0yjrv0MUX9vJtxV+5KeoOq6jMYgeTDpgsQoN3cIifTR0rvN8THK0zrodp4kAz7aSLg9IXA jmXmc9Kf5bHGoAU50LfwLcJ9ntECxps= X-MC-Unique: d15v1ziuOt-n_qon3Qe5Rg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 04/31] util: Allow virFileCache data to be any GObject Date: Fri, 20 Jan 2023 16:02:58 -0600 Message-Id: <20230120220325.1015090-5-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252253863100003 Content-Type: text/plain; charset="utf-8"; x-default="true" Since the libvirt documentation suggests to prefer GObject over virObject, and since virObject is a GObject, change virFileCache to allow GObjects as data. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/util/virfilecache.c | 14 ++++++++------ src/util/virfilecache.h | 2 +- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/src/util/virfilecache.c b/src/util/virfilecache.c index bad37c9f00..eaedf6db7e 100644 --- a/src/util/virfilecache.c +++ b/src/util/virfilecache.c @@ -170,7 +170,7 @@ virFileCacheLoad(virFileCache *cache, *data =3D g_steal_pointer(&loadData); =20 cleanup: - virObjectUnref(loadData); + g_clear_pointer(&loadData, g_object_unref); return ret; } =20 @@ -207,7 +207,7 @@ virFileCacheNewData(virFileCache *cache, return NULL; =20 if (virFileCacheSave(cache, name, data) < 0) { - g_clear_pointer(&data, virObjectUnref); + g_clear_object(&data); } } =20 @@ -239,7 +239,7 @@ virFileCacheNew(const char *dir, if (!(cache =3D virObjectNew(virFileCacheClass))) return NULL; =20 - cache->table =3D virHashNew(virObjectUnref); + cache->table =3D virHashNew(g_object_unref); =20 cache->dir =3D g_strdup(dir); =20 @@ -270,7 +270,7 @@ virFileCacheValidate(virFileCache *cache, if (*data) { VIR_DEBUG("Caching data '%p' for '%s'", *data, name); if (virHashAddEntry(cache->table, name, *data) < 0) { - g_clear_pointer(data, virObjectUnref); + g_clear_pointer(data, g_object_unref); } } } @@ -300,7 +300,8 @@ virFileCacheLookup(virFileCache *cache, data =3D virHashLookup(cache->table, name); virFileCacheValidate(cache, name, &data); =20 - virObjectRef(data); + if (data) + g_object_ref(data); virObjectUnlock(cache); =20 return data; @@ -331,7 +332,8 @@ virFileCacheLookupByFunc(virFileCache *cache, data =3D virHashSearch(cache->table, iter, iterData, &name); virFileCacheValidate(cache, name, &data); =20 - virObjectRef(data); + if (data) + g_object_ref(data); virObjectUnlock(cache); =20 return data; diff --git a/src/util/virfilecache.h b/src/util/virfilecache.h index 81be8feef5..f0d220cc86 100644 --- a/src/util/virfilecache.h +++ b/src/util/virfilecache.h @@ -48,7 +48,7 @@ typedef bool * @priv: private data created together with cache * * Creates a new data based on the @name. The returned data must be - * an instance of virObject. + * an instance of GObject. * * Returns data object or NULL on error. */ --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252399; cv=none; d=zohomail.com; s=zohoarc; b=XCmpr/l1Eaf9kg21OjJfN1pYFr2COjQXcoZ1MGhCeFRJemgJBrAzUGTuf6NyLuJgMZs/bxAFp5ujKyf0yz0x5+WeR5ic3YxY2pbzNKXBwqkC6g/s55xp4AWPhc/PB5mkC3KiZFe4HuUDMyoLb5g2c9NDqOjh991F+LU1+QUjKOQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252399; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bIPz9fYvCt9IOW9SCNLvgq1pd4hTwH/CIw56+MEP1tE=; b=ILIj0WhZEfLlYtqWB6q9qKNVUaOhEDYUKHZKf3XU8i4JTzCH22OC1QXltPtT3IjiVsxFjXKcukDRMt+Nmlq1phsVBWmmXATdINliztqnCVbXYo4M1liMIwC7vJ4wI0aaPFPFY1/MqgYti41XvSjjkHEdjf8LQh3bZLAzB8lQ5N4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252399488682.2644449664858; Fri, 20 Jan 2023 14:06:39 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-262-RObw_nGcPjWNT0gQaseGLw-1; Fri, 20 Jan 2023 17:03:49 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9670F2817252; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 841A8492B14; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id ECEE91947043; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 2F9501946A7A for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 109E61401C33; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D9E1014171C0; Fri, 20 Jan 2023 22:03:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252395; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=bIPz9fYvCt9IOW9SCNLvgq1pd4hTwH/CIw56+MEP1tE=; b=TV3LsYzrk9rrv+6kbhmVohyDRbjQDdC7qrcW+SZ6JmX9yLXNotAjvE9ACT+jjwHew3o7rO iUZYsZ74LZQTMFPrskUFQ3wM4bZcDzciLoDt+6n5rbTIGQg+glMLxwcdtjkhph9cNHCdkG wmczkqw7mzNYIKCt9cLzYxyigr9bQ78= X-MC-Unique: RObw_nGcPjWNT0gQaseGLw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 05/31] qemu: implement basic virFileCache for nbdkit caps Date: Fri, 20 Jan 2023 16:02:59 -0600 Message-Id: <20230120220325.1015090-6-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252400004100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Preparatory step for caching nbdkit capabilities. This patch implements the newData and isValid virFileCacheHandlers callback functions. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 89 +++++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_nbdkit.h | 4 ++ 2 files changed, 92 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index bb0e76ecbc..37a2a8ca38 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -200,7 +200,7 @@ qemuNbdkitGetDirMtime(const char *moddir) } =20 =20 -G_GNUC_UNUSED static void +static void qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) { struct stat st; @@ -239,3 +239,90 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, { ignore_value(virBitmapSetBit(nbdkitCaps->flags, flag)); } + + +static bool +virNbkditCapsCheckModdir(const char *moddir, + time_t expectedMtime) +{ + time_t mtime =3D qemuNbdkitGetDirMtime(moddir); + + if (mtime !=3D expectedMtime) { + VIR_DEBUG("Outdated capabilities for nbdkit: module directory '%s'= changed (%lld vs %lld)", + moddir, (long long)mtime, (long long)expectedMtime); + return false; + } + return true; +} + + +static bool +virNbdkitCapsIsValid(void *data, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *nbdkitCaps =3D data; + struct stat st; + + if (!nbdkitCaps->path) + return true; + + if (!virNbkditCapsCheckModdir(NBDKIT_PLUGINDIR, nbdkitCaps->pluginDirM= time)) + return false; + if (!virNbkditCapsCheckModdir(NBDKIT_FILTERDIR, nbdkitCaps->filterDirM= time)) + return false; + + if (nbdkitCaps->libvirtCtime !=3D virGetSelfLastChanged() || + nbdkitCaps->libvirtVersion !=3D LIBVIR_VERSION_NUMBER) { + VIR_DEBUG("Outdated capabilities for '%s': libvirt changed (%lld v= s %lld, %lu vs %lu)", + nbdkitCaps->path, + (long long)nbdkitCaps->libvirtCtime, + (long long)virGetSelfLastChanged(), + (unsigned long)nbdkitCaps->libvirtVersion, + (unsigned long)LIBVIR_VERSION_NUMBER); + return false; + } + + if (stat(nbdkitCaps->path, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit binary '%s': %s", + nbdkitCaps->path, + g_strerror(errno)); + return false; + } + + if (st.st_ctime !=3D nbdkitCaps->ctime) { + VIR_DEBUG("Outdated capabilities for '%s': nbdkit binary changed (= %lld vs %lld)", + nbdkitCaps->path, + (long long)st.st_ctime, (long long)nbdkitCaps->ctime); + return false; + } + + return true; +} + + +static void* +virNbdkitCapsNewData(const char *binary, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *caps =3D qemuNbdkitCapsNew(binary); + qemuNbdkitCapsQuery(caps); + + return caps; +} + + +virFileCacheHandlers nbdkitCapsCacheHandlers =3D { + .isValid =3D virNbdkitCapsIsValid, + .newData =3D virNbdkitCapsNewData, + .loadFile =3D NULL, + .saveFile =3D NULL, + .privFree =3D NULL, +}; + + +virFileCache* +qemuNbdkitCapsCacheNew(const char *cachedir) +{ + g_autofree char *dir =3D g_build_filename(cachedir, "nbdkitcapabilitie= s", NULL); + return virFileCacheNew(dir, "xml", &nbdkitCapsCacheHandlers); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index e191e1fdb4..4aba7c8455 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -21,6 +21,7 @@ =20 #include "internal.h" #include "virenum.h" +#include "virfilecache.h" =20 typedef struct _qemuNbdkitCaps qemuNbdkitCaps; =20 @@ -38,6 +39,9 @@ VIR_ENUM_DECL(qemuNbdkitCaps); qemuNbdkitCaps * qemuNbdkitCapsNew(const char *path); =20 +virFileCache * +qemuNbdkitCapsCacheNew(const char *cachedir); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252229; cv=none; d=zohomail.com; s=zohoarc; b=AFXlZoBo6BNSNgq5UMP3wJ1PkExY0GntAtKDZwZBeJhtm5ub//TMvXc86VABD2N3vO48teeJD9UcNrmmMtPtBLS27PY5M2uAbnYSiog2w+5n7Dkww5P3d3l0+d6IHetvZa1NxjSSIRRtAZVPimPNI4mEvvxzVNHfHOc+1ZwWSlU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252229; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GaCXKmGm2828KuO7dp/aKJjSo7jmzaXCvuBbPvn6FTs=; b=COJt8S83mre4NhdY9Pjwm6fdFDXCKGi8RCQcvoRPEAMQsOl3dja9ZXr+nok0Fy15TJEBYs/SYf90zs9f5kdEUnnlgTGVYX3kGcdUjeM5VBQzFHM5h/EgAVpkZ/PzwG7MPCTIB/gSwyvu3nCkC9HYE6gfLN8VWSbSYJqQaIE9wWQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252229918179.0058688256289; Fri, 20 Jan 2023 14:03:49 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-648-M_Iz8WUyOf-eI9cqr6k_LQ-1; Fri, 20 Jan 2023 17:03:43 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3A8AF810BD2; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 223A51121325; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C29671946A6E; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 5F1DC1947044 for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 43E341415108; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 22469140EBF6 for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252228; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=GaCXKmGm2828KuO7dp/aKJjSo7jmzaXCvuBbPvn6FTs=; b=VaI7+8PE2Y7Bg+e8ZrPGJwwlOap9aFebxHwvTBa2R1CP6sd+nogO5SoqCdVz6T7/mt2htB /JlTz8LJYpXPdNuar9QyVyZ+NUUvGJ5w8z93n1SM/a7WkATRtVUte6smqB05m1iDM9AS2t G2Em1ShcNh/sJ4FGaIVoKjN0WAMwKfg= X-MC-Unique: M_Iz8WUyOf-eI9cqr6k_LQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 06/31] qemu: implement persistent file cache for nbdkit caps Date: Fri, 20 Jan 2023 16:03:00 -0600 Message-Id: <20230120220325.1015090-7-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252231759100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Implement the loadFile and saveFile virFileCacheHandlers callbacks so that nbdkit capabilities are cached perstistently across daemon restarts. The format and implementation is modeled on the qemu capabilities, but simplified slightly. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- po/POTFILES | 1 + src/qemu/qemu_nbdkit.c | 226 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 225 insertions(+), 2 deletions(-) diff --git a/po/POTFILES b/po/POTFILES index 4e446aaf40..a45259c0d8 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -179,6 +179,7 @@ src/qemu/qemu_monitor.c src/qemu/qemu_monitor_json.c src/qemu/qemu_monitor_text.c src/qemu/qemu_namespace.c +src/qemu/qemu_nbdkit.c src/qemu/qemu_passt.c src/qemu/qemu_process.c src/qemu/qemu_qapi.c diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 37a2a8ca38..82586379d0 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -311,11 +311,233 @@ virNbdkitCapsNewData(const char *binary, } =20 =20 +static int +qemuNbdkitCapsValidateBinary(qemuNbdkitCaps *nbdkitCaps, + xmlXPathContextPtr ctxt) +{ + g_autofree char *str =3D NULL; + + if (!(str =3D virXPathString("string(./path)", ctxt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing path in nbdkit capabilities cache")); + return -1; + } + + if (STRNEQ(str, nbdkitCaps->path)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Expected caps for '%s' but saw '%s'"), + nbdkitCaps->path, str); + return -1; + } + + return 0; +} + + +static int +qemuNbdkitCapsParseFlags(qemuNbdkitCaps *nbdkitCaps, + xmlXPathContextPtr ctxt) +{ + g_autofree xmlNodePtr *nodes =3D NULL; + size_t i; + int n; + + if ((n =3D virXPathNodeSet("./flag", ctxt, &nodes)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to parse qemu capabilities flags")); + return -1; + } + + VIR_DEBUG("Got flags %d", n); + for (i =3D 0; i < n; i++) { + unsigned int flag; + + if (virXMLPropEnum(nodes[i], "name", qemuNbdkitCapsTypeFromString, + VIR_XML_PROP_REQUIRED, &flag) < 0) + return -1; + + qemuNbdkitCapsSet(nbdkitCaps, flag); + } + + return 0; +} + + +/* + * Parsing a doc that looks like + * + * + * /some/path + * 234235253 + * 234235253 + * 234235253 + * 234235253 + * 1002016 + * + * + * ... + * + * + * Returns 0 on success, 1 if outdated, -1 on error + */ +static int +qemuNbdkitCapsLoadCache(qemuNbdkitCaps *nbdkitCaps, + const char *filename) +{ + g_autoptr(xmlDoc) doc =3D NULL; + g_autoptr(xmlXPathContext) ctxt =3D NULL; + long long int l; + + if (!(doc =3D virXMLParse(filename, NULL, NULL, "nbdkitCaps", &ctxt, N= ULL, false))) + return -1; + + if (virXPathLongLong("string(./selfctime)", ctxt, &l) < 0) { + VIR_DEBUG("missing selfctime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->libvirtCtime =3D (time_t)l; + + nbdkitCaps->libvirtVersion =3D 0; + virXPathUInt("string(./selfvers)", ctxt, &nbdkitCaps->libvirtVersion); + + if (nbdkitCaps->libvirtCtime !=3D virGetSelfLastChanged() || + nbdkitCaps->libvirtVersion !=3D LIBVIR_VERSION_NUMBER) { + VIR_DEBUG("Outdated capabilities in %s: libvirt changed (%lld vs %= lld, %lu vs %lu), stopping load", + nbdkitCaps->path, + (long long)nbdkitCaps->libvirtCtime, + (long long)virGetSelfLastChanged(), + (unsigned long)nbdkitCaps->libvirtVersion, + (unsigned long)LIBVIR_VERSION_NUMBER); + return 1; + } + + if (qemuNbdkitCapsValidateBinary(nbdkitCaps, ctxt) < 0) + return -1; + + if (virXPathLongLong("string(./nbdkitctime)", ctxt, &l) < 0) { + VIR_DEBUG("missing nbdkitctime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->ctime =3D (time_t)l; + + if (virXPathLongLong("string(./plugindirmtime)", ctxt, &l) < 0) { + VIR_DEBUG("missing plugindirmtime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->pluginDirMtime =3D (time_t)l; + + if (virXPathLongLong("string(./filterdirmtime)", ctxt, &l) < 0) { + VIR_DEBUG("missing filterdirmtime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->filterDirMtime =3D (time_t)l; + + if (qemuNbdkitCapsParseFlags(nbdkitCaps, ctxt) < 0) + return -1; + + if ((nbdkitCaps->version =3D virXPathString("string(./version)", ctxt)= ) =3D=3D NULL) { + VIR_DEBUG("missing version in nbdkit capabilities cache"); + return -1; + } + + return 0; +} + + +static void* +virNbdkitCapsLoadFile(const char *filename, + const char *binary, + void *privData G_GNUC_UNUSED, + bool *outdated) +{ + g_autoptr(qemuNbdkitCaps) nbdkitCaps =3D qemuNbdkitCapsNew(binary); + int ret; + + if (!nbdkitCaps) + return NULL; + + ret =3D qemuNbdkitCapsLoadCache(nbdkitCaps, filename); + if (ret < 0) + return NULL; + if (ret =3D=3D 1) { + *outdated =3D true; + return NULL; + } + + return g_steal_pointer(&nbdkitCaps); +} + + +static char* +qemuNbdkitCapsFormatCache(qemuNbdkitCaps *nbdkitCaps) +{ + g_auto(virBuffer) buf =3D VIR_BUFFER_INITIALIZER; + size_t i; + + virBufferAddLit(&buf, "\n"); + virBufferAdjustIndent(&buf, 2); + + virBufferEscapeString(&buf, "%s\n", + nbdkitCaps->path); + virBufferAsprintf(&buf, "%lu\n", + nbdkitCaps->ctime); + virBufferAsprintf(&buf, "%lu\n", + nbdkitCaps->pluginDirMtime); + virBufferAsprintf(&buf, "%lu\n", + nbdkitCaps->filterDirMtime); + virBufferAsprintf(&buf, "%lu\n", + nbdkitCaps->libvirtCtime); + virBufferAsprintf(&buf, "%u\n", + nbdkitCaps->libvirtVersion); + + for (i =3D 0; i < QEMU_NBDKIT_CAPS_LAST; i++) { + if (qemuNbdkitCapsGet(nbdkitCaps, i)) { + virBufferAsprintf(&buf, "\n", + qemuNbdkitCapsTypeToString(i)); + } + } + + virBufferAsprintf(&buf, "%s\n", + nbdkitCaps->version); + + virBufferAdjustIndent(&buf, -2); + virBufferAddLit(&buf, "\n"); + + return virBufferContentAndReset(&buf); +} + + +static int +virNbdkitCapsSaveFile(void *data, + const char *filename, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *nbdkitCaps =3D data; + g_autofree char *xml =3D NULL; + + xml =3D qemuNbdkitCapsFormatCache(nbdkitCaps); + + if (virFileWriteStr(filename, xml, 0600) < 0) { + virReportSystemError(errno, + _("Failed to save '%s' for '%s'"), + filename, nbdkitCaps->path); + return -1; + } + + VIR_DEBUG("Saved caps '%s' for '%s' with (%lu, %lu)", + filename, nbdkitCaps->path, + nbdkitCaps->ctime, + nbdkitCaps->libvirtCtime); + + return 0; +} + + virFileCacheHandlers nbdkitCapsCacheHandlers =3D { .isValid =3D virNbdkitCapsIsValid, .newData =3D virNbdkitCapsNewData, - .loadFile =3D NULL, - .saveFile =3D NULL, + .loadFile =3D virNbdkitCapsLoadFile, + .saveFile =3D virNbdkitCapsSaveFile, .privFree =3D NULL, }; =20 --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252373; cv=none; d=zohomail.com; s=zohoarc; b=ICwkohuO9fnjRYFV6kT6rVxRqEDqjSCc0QhoQfhWuP7ytcZB84FaaZ1/THZON0J5wiaxA6Vx6fQjXb6IrVqozLNGe2xIMy5Oxs/Ngu5p3G9U0gZpfq7WnmPvifFVhEBjDgQlMOnLbA7NG0Tjh96zAjOAAP/XPKmmSgxQnBJOi3I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252373; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=th878AuQJ5CLbDi+0EUe4EYD2B7AloIN1nVQzTMZk9Y=; b=OzWIUZxSPI3QZWug034hIBo/AL41gXgZLZLgcMXx14iyDCaUgv5e/HKDXGPg0QGfLz278xGqk4N4/SHtBLKIsXTf64+1Ao5qgiSvDGVFGcx6fMTkJAEOuaYRDAjpuX16lhaRz16/6+gE6K39LXr8Se/9+NZlyuXT9kKf4e/KoMY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252373358775.2189551527713; Fri, 20 Jan 2023 14:06:13 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-633-uDaKNnUPPQ-_A6S1F13pxg-1; Fri, 20 Jan 2023 17:03:51 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2BACA88606F; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E4944228F; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 414481947057; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8CFD21947044 for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 812621415113; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5639D140EBF6; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252372; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=th878AuQJ5CLbDi+0EUe4EYD2B7AloIN1nVQzTMZk9Y=; b=RU1R2abIMARYzZumPCTcmRN8thu5o1QFfQ3fHQZfKVt2lw9d04TsmC5wZNnr8zzd8pyPo/ pRsopkeoq4Y9H438YV+F8ukIpn3ZJ6DYtlZa74viemhIG82miyFkaXt2JRG9H/OQqFJ3PQ SDkw5DyKPaSngLzVaZge4UoTc441h7w= X-MC-Unique: uDaKNnUPPQ-_A6S1F13pxg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 07/31] qemu: use file cache for nbdkit caps Date: Fri, 20 Jan 2023 16:03:01 -0600 Message-Id: <20230120220325.1015090-8-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252373834100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Add the virFileCache implementation for nbdkit capabilities to the qemu driver. This allows us to determine whether nbdkit is installed and which plugins are supported. it also has persistent caching and the capabilities are re-queried whenever something changes. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_conf.h | 3 +++ src/qemu/qemu_driver.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 4f610d86a1..a44985fb8b 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -318,6 +318,9 @@ struct _virQEMUDriver { =20 /* Immutable pointer, self-locking APIs */ virHashAtomic *migrationErrors; + + /* Immutable pointer, self-locking APIs */ + virFileCache *nbdkitCapsCache; }; =20 virQEMUDriverConfig *virQEMUDriverConfigNew(bool privileged, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index d6879175fe..df0f829196 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -845,6 +845,8 @@ qemuStateInitialize(bool privileged, defsecmodel))) goto error; =20 + qemu_driver->nbdkitCapsCache =3D qemuNbdkitCapsCacheNew(cfg->cacheDir); + /* If hugetlbfs is present, then we need to create a sub-directory wit= hin * it, since we can't assume the root mount point has permissions that * will let our spawned QEMU instances use it. */ @@ -1078,6 +1080,7 @@ qemuStateCleanup(void) VIR_FREE(qemu_driver->qemuImgBinary); virObjectUnref(qemu_driver->domains); virThreadPoolFree(qemu_driver->workerPool); + virObjectUnref(qemu_driver->nbdkitCapsCache); =20 if (qemu_driver->lockFD !=3D -1) virPidFileRelease(qemu_driver->config->stateDir, "driver", qemu_dr= iver->lockFD); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252238; cv=none; d=zohomail.com; s=zohoarc; b=Eq5O63W66bNyEKOMdoKO5DYnWag2VZiipIq2UefMU2mqjvrWetAz0OwHv7ts68ytgJ5CN9PSMeM9d0eWzueyVHYl8OHg3rtVq3WaVBQ+eM9sy+0WINOENMRNaQE94t/3gICjqIc4vBfmhopitbcDuaHps1PC76Tpd65n106JrOw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252238; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Txd3/y8bQHCS4bUJ+0j3oG1QBTH8Ji2W75BktRmOvn0=; b=I6diO9/NU/CBaBvyiIAOasuf2Guw4hfJo2YTD5rzti2SYSw1gAKJTNvxroIR0I5+41ZW1mJhp/sDu0lxfBaoUBgEuivDIYykx2pg5xjrJiWfhCHFan8wJ3APBPnzKRTL/n3cUS6XbS+4sUnmJ/n2G+xd1w9+fA6yvKQtKQDhzuw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252237998878.8379311684855; Fri, 20 Jan 2023 14:03:57 -0800 (PST) Received: from mimecast-mx02.redhat.com (66.187.233.88 [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-637-9v-5Mue2OJWE-uCCYqSYzw-1; Fri, 20 Jan 2023 17:03:53 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 88C7880A0C7; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6CE9B2026D2C; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 849481947063; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id CF8531946A6D for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B518F1415108; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 93345140EBF6 for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252237; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Txd3/y8bQHCS4bUJ+0j3oG1QBTH8Ji2W75BktRmOvn0=; b=UtzhUYe1oZ5hV3I6Z5b7LjhHE5tkJ79cICm32oy68oPNLCynv4fhjJ2C3FwACK901aEoI1 +VwdReJrsD0bPmSUW8lpaUj3agoMf2s6OacFzECs+EEnTbEH8IKaydtDHCOj2tIpqni3Q5 cSs3AtRTbpJnRWivVCqBYHCdm9sXRGw= X-MC-Unique: 9v-5Mue2OJWE-uCCYqSYzw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 08/31] qemu: Add qemuNbdkitProcess Date: Fri, 20 Jan 2023 16:03:02 -0600 Message-Id: <20230120220325.1015090-9-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252239926100013 Content-Type: text/plain; charset="utf-8"; x-default="true" An object for storing information about a nbdkit process that is serving a specific virStorageSource. At the moment, this information is just stored in the private data of virStorageSource and not used at all. Future commits will use this data to actually start a nbdkit process. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_conf.c | 22 ++++++++++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_domain.c | 31 ++++++++++++++++ src/qemu/qemu_domain.h | 4 +++ src/qemu/qemu_nbdkit.c | 82 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 26 ++++++++++++++ 6 files changed, 167 insertions(+) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 16d52cbbd4..f47c70985f 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -1669,3 +1669,25 @@ qemuHugepageMakeBasedir(virQEMUDriver *driver, =20 return 0; } + + +/* + * qemuGetNbdkitCaps: + * @driver: the qemu driver + * + * Gets the capabilities for Nbdkit for the specified driver. These can be= used + * to determine whether a particular disk source can be served by nbdkit or + * not. + * + * Returns: a reference to qemuNbdkitCaps or NULL + */ +qemuNbdkitCaps* +qemuGetNbdkitCaps(virQEMUDriver *driver) +{ + char *nbdkitBinary =3D virFindFileInPath("nbdkit"); + + if (!nbdkitBinary) + return NULL; + + return virFileCacheLookup(driver->nbdkitCapsCache, nbdkitBinary); +} diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index a44985fb8b..1a3ba3a0fb 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -377,3 +377,5 @@ int qemuGetMemoryBackingPath(virQEMUDriver *driver, =20 int qemuHugepageMakeBasedir(virQEMUDriver *driver, virHugeTLBFS *hugepage); + +qemuNbdkitCaps* qemuGetNbdkitCaps(virQEMUDriver *driver); diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 2eb5653254..bcf051d6b1 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -875,6 +875,7 @@ qemuDomainStorageSourcePrivateDispose(void *obj) g_clear_pointer(&priv->httpcookie, qemuDomainSecretInfoFree); g_clear_pointer(&priv->tlsKeySecret, qemuDomainSecretInfoFree); g_clear_pointer(&priv->fdpass, qemuFDPassFree); + g_clear_pointer(&priv->nbdkitProcess, qemuNbdkitProcessFree); } =20 =20 @@ -10097,6 +10098,34 @@ qemuDomainPrepareStorageSourceNFS(virStorageSource= *src) } =20 =20 +/* qemuPrepareStorageSourceNbdkit: + * @src: source for a disk + * + * If src is an network source that is managed by nbdkit, prepare data so = that + * nbdkit can be launched before the domain is started + * + * Returns true if nbdkit will be used for this source, + */ +static bool +qemuDomainPrepareStorageSourceNbdkit(virStorageSource *src, + virQEMUDriverConfig *cfg, + const char *alias, + qemuDomainObjPrivate *priv) +{ + g_autoptr(qemuNbdkitCaps) nbdkit =3D NULL; + + if (virStorageSourceGetActualType(src) !=3D VIR_STORAGE_TYPE_NETWORK) + return false; + + nbdkit =3D qemuGetNbdkitCaps(priv->driver); + if (!nbdkit) + return false; + + return qemuNbdkitInitStorageSource(nbdkit, src, priv->libDir, + alias, cfg->user, cfg->group); +} + + /* qemuProcessPrepareStorageSourceTLS: * @source: source for a disk * @cfg: driver configuration @@ -10929,6 +10958,8 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virD= omainDiskDef *disk, if (qemuDomainPrepareStorageSourceFDs(src, priv) < 0) return -1; =20 + qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, priv); + return 0; } =20 diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 08430b67b9..a3b9acab70 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -33,6 +33,7 @@ #include "qemu_conf.h" #include "qemu_capabilities.h" #include "qemu_migration_params.h" +#include "qemu_nbdkit.h" #include "qemu_slirp.h" #include "qemu_fd.h" #include "virchrdev.h" @@ -308,6 +309,9 @@ struct _qemuDomainStorageSourcePrivate { =20 /* file descriptors if user asks for FDs to be passed */ qemuFDPass *fdpass; + + /* an nbdkit process for serving network storage sources */ + qemuNbdkitProcess *nbdkitProcess; }; =20 virObject *qemuDomainStorageSourcePrivateNew(void); diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 82586379d0..231908b6b1 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -548,3 +548,85 @@ qemuNbdkitCapsCacheNew(const char *cachedir) g_autofree char *dir =3D g_build_filename(cachedir, "nbdkitcapabilitie= s", NULL); return virFileCacheNew(dir, "xml", &nbdkitCapsCacheHandlers); } + + +static qemuNbdkitProcess * +qemuNbdkitProcessNew(virStorageSource *source, + const char *pidfile, + const char *socketfile) +{ + qemuNbdkitProcess *nbdkit =3D g_new0(qemuNbdkitProcess, 1); + /* weak reference -- source owns this object, so it will always outliv= e us */ + nbdkit->source =3D source; + nbdkit->user =3D -1; + nbdkit->group =3D -1; + nbdkit->pid =3D -1; + nbdkit->pidfile =3D g_strdup(pidfile); + nbdkit->socketfile =3D g_strdup(socketfile); + + return nbdkit; +} + + +bool +qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, + virStorageSource *source, + char *statedir, + const char *alias, + uid_t user, + gid_t group) +{ + qemuDomainStorageSourcePrivate *srcPriv =3D qemuDomainStorageSourcePri= vateFetch(source); + g_autofree char *pidname =3D g_strdup_printf("nbdkit-%s.pid", alias); + g_autofree char *socketname =3D g_strdup_printf("nbdkit-%s.socket", al= ias); + g_autofree char *pidfile =3D g_build_filename(statedir, pidname, NULL); + g_autofree char *socketfile =3D g_build_filename(statedir, socketname,= NULL); + qemuNbdkitProcess *proc; + + if (srcPriv->nbdkitProcess) + return false; + + switch (source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (!virBitmapIsBitSet(caps->flags, QEMU_NBDKIT_CAPS_PLUGIN_CU= RL)) + return false; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (!virBitmapIsBitSet(caps->flags, QEMU_NBDKIT_CAPS_PLUGIN_SS= H)) + return false; + break; + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + return false; + } + + proc =3D qemuNbdkitProcessNew(source, pidfile, socketfile); + proc->caps =3D g_object_ref(caps); + proc->user =3D user; + proc->group =3D group; + + srcPriv->nbdkitProcess =3D proc; + + return true; +} + + +void +qemuNbdkitProcessFree(qemuNbdkitProcess *proc) +{ + g_clear_pointer(&proc->pidfile, g_free); + g_clear_pointer(&proc->socketfile, g_free); + g_clear_object(&proc->caps); + g_free(proc); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 4aba7c8455..8844bba13c 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -20,10 +20,12 @@ #pragma once =20 #include "internal.h" +#include "storage_source_conf.h" #include "virenum.h" #include "virfilecache.h" =20 typedef struct _qemuNbdkitCaps qemuNbdkitCaps; +typedef struct _qemuNbdkitProcess qemuNbdkitProcess; =20 typedef enum { /* 0 */ @@ -42,6 +44,14 @@ qemuNbdkitCapsNew(const char *path); virFileCache * qemuNbdkitCapsCacheNew(const char *cachedir); =20 +bool +qemuNbdkitInitStorageSource(qemuNbdkitCaps *nbdkitCaps, + virStorageSource *source, + char *statedir, + const char *alias, + uid_t user, + gid_t group); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); @@ -52,3 +62,19 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, =20 #define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, = GObject); + +struct _qemuNbdkitProcess { + qemuNbdkitCaps *caps; + virStorageSource *source; + + char *pidfile; + char *socketfile; + uid_t user; + gid_t group; + pid_t pid; +}; + +void +qemuNbdkitProcessFree(qemuNbdkitProcess *proc); + +G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuNbdkitProcess, qemuNbdkitProcessFree); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252232; cv=none; d=zohomail.com; s=zohoarc; b=jM7Eohs5zGp0GucJ0CgXacZRSlpSc9uEduCRWcHpU0YNn2w7kD79IiHwvzeRKl/vSTMmtpcbtXwVDefX498BJ/bYshigAxGU29ZlB3VcUoutiHE0JawQLDHUs57+fHKo6YDCv+QrHFpYjj1eg7OWLKd2lqvtPmAeFrg1480+1gE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252232; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HJmTXwNOiARjwmz0gdlZ3+ka8F0u80i2VE4mlsZMSZk=; b=Jkv/1XWuJcJjOFaVIVkcy4O4e3oB8E5FpKZePa7AQwEHx7SSJPOt8T5y8qBH4uIq11xlrswUDcmykqIe8OVismZAN/suDoxKBI23cJGlc4cvKgwWjrDlETAa6h6hgp49KmzI/dHBMJylasENTa/o2XbYudjYeosZHP5mqufMA8A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252232502237.6095178459766; Fri, 20 Jan 2023 14:03:52 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-474-4RczUv46N0-yMOCdsZEBlw-1; Fri, 20 Jan 2023 17:03:49 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ED4CE1818E58; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id D815339D92; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BF8131946A6D; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 122E619465B6 for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id EAB031415126; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C9010140EBF6 for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252231; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=HJmTXwNOiARjwmz0gdlZ3+ka8F0u80i2VE4mlsZMSZk=; b=Y3XxzHNjloUo0rg+kntDw7I39mxF8+j0FDSAcXVIm/HqCegOeuCBXUYOucstEUfEh8z7jg E7bfOpI4awpFV+xt+MMYBAiVJxdd9otpsrc5rJACE54JhMjlSUn2PQRPuIMgyrAb1fDUrD 7sxMN5agGvsl325QVkh0u8ZW/D7ouqI= X-MC-Unique: 4RczUv46N0-yMOCdsZEBlw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 09/31] qemu: query nbdkit module dir from binary Date: Fri, 20 Jan 2023 16:03:03 -0600 Message-Id: <20230120220325.1015090-10-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252233815100003 Content-Type: text/plain; charset="utf-8"; x-default="true" Rather than having NBDKIT_MODDIR as a build configuration option, query the nbdkit binary for the location to these directories. nbdkit provides a --dump-config optiont that outputs this information and can be easily parsed. We can also get the version from this output rather than executing `nbdkit --version` separately. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- meson.build | 6 ---- meson_options.txt | 1 - src/qemu/qemu_nbdkit.c | 75 +++++++++++++++++++++++++++++++++--------- 3 files changed, 60 insertions(+), 22 deletions(-) diff --git a/meson.build b/meson.build index ca02fde91b..e498b49be4 100644 --- a/meson.build +++ b/meson.build @@ -1664,12 +1664,6 @@ if not get_option('driver_qemu').disabled() qemu_dbus_daemon_path =3D '/usr/bin/dbus-daemon' endif conf.set_quoted('QEMU_DBUS_DAEMON', qemu_dbus_daemon_path) - - nbdkit_moddir =3D get_option('nbdkit_moddir') - if nbdkit_moddir =3D=3D '' - nbdkit_moddir =3D libdir / 'nbdkit' - endif - conf.set_quoted('NBDKIT_MODDIR', nbdkit_moddir) endif endif =20 diff --git a/meson_options.txt b/meson_options.txt index d5ea4376e0..861c5577d2 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -71,7 +71,6 @@ option('driver_vbox', type: 'feature', value: 'auto', des= cription: 'VirtualBox X option('vbox_xpcomc_dir', type: 'string', value: '', description: 'Locatio= n of directory containing VirtualBox XPCOMC library') option('driver_vmware', type: 'feature', value: 'auto', description: 'VMwa= re driver') option('driver_vz', type: 'feature', value: 'auto', description: 'Virtuozz= o driver') -option('nbdkit_moddir', type: 'string', value: '', description: 'set the d= irectory where nbdkit modules are located') =20 option('secdriver_apparmor', type: 'feature', value: 'auto', description: = 'use AppArmor security driver') option('apparmor_profiles', type: 'feature', value: 'auto', description: '= install apparmor profiles') diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 231908b6b1..840710862e 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -39,9 +39,6 @@ =20 VIR_LOG_INIT("qemu.nbdkit"); =20 -#define NBDKIT_PLUGINDIR NBDKIT_MODDIR "/plugins" -#define NBDKIT_FILTERDIR NBDKIT_MODDIR "/filters" - VIR_ENUM_IMPL(qemuNbdkitCaps, QEMU_NBDKIT_CAPS_LAST, /* 0 */ @@ -55,6 +52,9 @@ struct _qemuNbdkitCaps { =20 char *path; char *version; + char *filterDir; + char *pluginDir; + time_t ctime; time_t libvirtCtime; time_t pluginDirMtime; @@ -128,18 +128,47 @@ qemuNbdkitCapsQueryFilters(qemuNbdkitCaps *nbdkit) =20 =20 static int -qemuNbdkitCapsQueryVersion(qemuNbdkitCaps *nbdkit) +qemuNbdkitCapsQueryBuildConfig(qemuNbdkitCaps *nbdkit) { + size_t i; + g_autofree char *output =3D NULL; + g_auto(GStrv) lines =3D NULL; + const char *line; g_autoptr(virCommand) cmd =3D virCommandNewArgList(nbdkit->path, - "--version", + "--dump-config", NULL); =20 - virCommandSetOutputBuffer(cmd, &nbdkit->version); + virCommandSetOutputBuffer(cmd, &output); =20 if (virCommandRun(cmd, NULL) !=3D 0) return -1; =20 - VIR_DEBUG("Got nbdkit version %s", nbdkit->version); + lines =3D g_strsplit(output, "\n", 0); + if (!lines) + return -1; + + for (i =3D 0; (line =3D lines[i]); i++) { + const char *key; + const char *val; + char *p; + + p =3D strchr(line, '=3D'); + if (!p) + continue; + + *p =3D '\0'; + key =3D line; + val =3D p + 1; + + VIR_DEBUG("Got nbdkit config value %s=3D%s", key, val); + + if (STREQ(key, "version")) + nbdkit->version =3D g_strdup(val); + else if (STREQ(key, "filterdir")) + nbdkit->filterDir =3D g_strdup(val); + else if (STREQ(key, "plugindir")) + nbdkit->pluginDir =3D g_strdup(val); + } return 0; } =20 @@ -151,6 +180,8 @@ qemuNbdkitCapsFinalize(GObject *object) =20 g_clear_pointer(&nbdkit->path, g_free); g_clear_pointer(&nbdkit->version, g_free); + g_clear_pointer(&nbdkit->filterDir, g_free); + g_clear_pointer(&nbdkit->pluginDir, g_free); g_clear_pointer(&nbdkit->flags, virBitmapFree); =20 G_OBJECT_CLASS(qemu_nbdkit_caps_parent_class)->finalize(object); @@ -213,15 +244,15 @@ qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) return; } =20 + qemuNbdkitCapsQueryBuildConfig(caps); + qemuNbdkitCapsQueryPlugins(caps); + qemuNbdkitCapsQueryFilters(caps); + caps->ctime =3D st.st_ctime; - caps->filterDirMtime =3D qemuNbdkitGetDirMtime(NBDKIT_FILTERDIR); - caps->pluginDirMtime =3D qemuNbdkitGetDirMtime(NBDKIT_PLUGINDIR); + caps->filterDirMtime =3D qemuNbdkitGetDirMtime(caps->filterDir); + caps->pluginDirMtime =3D qemuNbdkitGetDirMtime(caps->pluginDir); caps->libvirtCtime =3D virGetSelfLastChanged(); caps->libvirtVersion =3D LIBVIR_VERSION_NUMBER; - - qemuNbdkitCapsQueryPlugins(caps); - qemuNbdkitCapsQueryFilters(caps); - qemuNbdkitCapsQueryVersion(caps); } =20 =20 @@ -266,9 +297,9 @@ virNbdkitCapsIsValid(void *data, if (!nbdkitCaps->path) return true; =20 - if (!virNbkditCapsCheckModdir(NBDKIT_PLUGINDIR, nbdkitCaps->pluginDirM= time)) + if (!virNbkditCapsCheckModdir(nbdkitCaps->pluginDir, nbdkitCaps->plugi= nDirMtime)) return false; - if (!virNbkditCapsCheckModdir(NBDKIT_FILTERDIR, nbdkitCaps->filterDirM= time)) + if (!virNbkditCapsCheckModdir(nbdkitCaps->filterDir, nbdkitCaps->filte= rDirMtime)) return false; =20 if (nbdkitCaps->libvirtCtime !=3D virGetSelfLastChanged() || @@ -420,12 +451,22 @@ qemuNbdkitCapsLoadCache(qemuNbdkitCaps *nbdkitCaps, } nbdkitCaps->ctime =3D (time_t)l; =20 + if ((nbdkitCaps->pluginDir =3D virXPathString("string(./plugindir)", c= txt)) =3D=3D NULL) { + VIR_DEBUG("missing plugindir in nbdkit capabilities cache"); + return -1; + } + if (virXPathLongLong("string(./plugindirmtime)", ctxt, &l) < 0) { VIR_DEBUG("missing plugindirmtime in nbdkit capabilities XML"); return -1; } nbdkitCaps->pluginDirMtime =3D (time_t)l; =20 + if ((nbdkitCaps->filterDir =3D virXPathString("string(./filterdir)", c= txt)) =3D=3D NULL) { + VIR_DEBUG("missing filterdir in nbdkit capabilities cache"); + return -1; + } + if (virXPathLongLong("string(./filterdirmtime)", ctxt, &l) < 0) { VIR_DEBUG("missing filterdirmtime in nbdkit capabilities XML"); return -1; @@ -481,8 +522,12 @@ qemuNbdkitCapsFormatCache(qemuNbdkitCaps *nbdkitCaps) nbdkitCaps->path); virBufferAsprintf(&buf, "%lu\n", nbdkitCaps->ctime); + virBufferEscapeString(&buf, "%s\n", + nbdkitCaps->pluginDir); virBufferAsprintf(&buf, "%lu\n", nbdkitCaps->pluginDirMtime); + virBufferEscapeString(&buf, "%s\n", + nbdkitCaps->filterDir); virBufferAsprintf(&buf, "%lu\n", nbdkitCaps->filterDirMtime); virBufferAsprintf(&buf, "%lu\n", --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252236; cv=none; d=zohomail.com; s=zohoarc; b=bwTdAo6r6AxtHA+bSKrjxgPniu0zSGkuzyLjbjRTgQ4UPA1oS609i7+uUpqOx2F4DY9RpbrneNeUFkjfrdE5/1nRXnlPLMysjkuyiw1uwWQu9pHGlwgf60WtdDrTL2ZtyWOXuBVoSpZuaWLMFe1WnalZDmH9EPUfioyLyZiQ7yc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252236; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zmP0ZRLTL6okfQd66Ui/V9mJCt9ALo3PDhz45zkaj+4=; b=YuyuZ0v7W4WmALYGWEHjX31GG9UFgNRdGndwj72ZGvM0EH9MtFLQUH3OW5lxSLYHvuGphekCPwSwhF5OsI2BXygUOEknxdrgcgH3IYd3FPVOKVSlbWxm9EoWipZ/6th6hfwzjqI/kuCbvbRKNXYtVRj8aNNaM2Ku8u8qUVz+eiE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 167425223655832.92848141562297; Fri, 20 Jan 2023 14:03:56 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-489-Tvw6bxRwPE2Jh8TVaII23w-1; Fri, 20 Jan 2023 17:03:52 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 221F52804850; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0585E2026D68; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E9FE71946A6D; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 531B519465B6 for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 287F41415108; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 07CEC140EBF6 for ; Fri, 20 Jan 2023 22:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252235; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=zmP0ZRLTL6okfQd66Ui/V9mJCt9ALo3PDhz45zkaj+4=; b=Yf/vQTvc0PseqTF6uqD/+8KqqFCySb2dXkA1DBickM5s24yJ2Bs3lNlHzBtwq2ivdSalNB K5MI3ttud6UqWx/brPtGz6qovQwcfxbUOA3YyylP6PgGYiCm+LYrEm1QGCR4SDqAO58dy2 TX+me6yhPcfhq8We6hkGCri44lQN+Bs= X-MC-Unique: Tvw6bxRwPE2Jh8TVaII23w-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 10/31] qemu: add functions to start and stop nbdkit Date: Fri, 20 Jan 2023 16:03:04 -0600 Message-Id: <20230120220325.1015090-11-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252237821100007 Content-Type: text/plain; charset="utf-8"; x-default="true" Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 255 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed, 265 insertions(+) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 840710862e..34462f784a 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -24,6 +24,7 @@ #include "virerror.h" #include "virlog.h" #include "virpidfile.h" +#include "virtime.h" #include "virutil.h" #include "qemu_block.h" #include "qemu_conf.h" @@ -667,6 +668,167 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } =20 =20 +static int +qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + g_autoptr(virURI) uri =3D qemuBlockStorageSourceGetURI(proc->source); + g_autofree char *uristring =3D virURIFormat(uri); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "curl"); + if (proc->source->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_HTTP) { + /* allow http to be upgraded to https via e.g. redirect */ + g_autofree char* protocols =3D g_strdup_printf("%s,%s", + virStorageNetProtocol= TypeToString(proc->source->protocol), + virStorageNetProtocol= TypeToString(VIR_STORAGE_NET_PROTOCOL_HTTPS)); + virCommandAddArgPair(cmd, "protocols", protocols); + } else { + virCommandAddArgPair(cmd, "protocols", + virStorageNetProtocolTypeToString(proc->sourc= e->protocol)); + } + virCommandAddArgPair(cmd, "url", uristring); + + if (proc->source->auth) { + g_autoptr(virConnect) conn =3D virGetConnectSecret(); + g_autofree uint8_t *secret =3D NULL; + size_t secretlen =3D 0; + g_autofree char *password =3D NULL; + int secrettype; + virStorageAuthDef *authdef =3D proc->source->auth; + + virCommandAddArgPair(cmd, "user", + proc->source->auth->username); + + if ((secrettype =3D virSecretUsageTypeFromString(proc->source->aut= h->secrettype)) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("invalid secret type %s"), + proc->source->auth->secrettype); + return -1; + } + + if (virSecretGetSecretString(conn, + &authdef->seclookupdef, + secrettype, + &secret, + &secretlen) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to get auth secret for storage")); + return -1; + } + + /* ensure that the secret is a NULL-terminated string */ + password =3D g_strndup((char*)secret, secretlen); + + /* for now, just report an error rather than passing the password = in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Password not yet supported for nbdkit sources")); + return -1; + } + + if (proc->source->ncookies > 0) { + /* for now, just report an error rather than passing cookies in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cookies not yet supported for nbdkit sources")); + return -1; + } + + if (proc->source->sslverify =3D=3D VIR_TRISTATE_BOOL_NO) { + virCommandAddArgPair(cmd, "sslverify", "false"); + } + + if (proc->source->timeout > 0) { + g_autofree char *timeout =3D g_strdup_printf("%llu", proc->source-= >timeout); + virCommandAddArgPair(cmd, "timeout", timeout); + } + + return 0; +} + + +static int +qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + const char *user =3D NULL; + virStorageNetHostDef *host =3D &proc->source->hosts[0]; + g_autofree char *portstr =3D g_strdup_printf("%u", host->port); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "ssh"); + + virCommandAddArgPair(cmd, "host", host->name); + virCommandAddArgPair(cmd, "port", portstr); + virCommandAddArgPair(cmd, "path", proc->source->path); + + if (proc->source->auth) + user =3D proc->source->auth->username; + else if (proc->source->ssh_user) + user =3D proc->source->ssh_user; + + if (user) + virCommandAddArgPair(cmd, "user", user); + + if (proc->source->ssh_host_key_check_disabled) + virCommandAddArgPair(cmd, "verify-remote-host", "false"); + + return 0; +} + + +static virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) +{ + g_autoptr(virCommand) cmd =3D virCommandNewArgList(proc->caps->path, + "--unix", + proc->socketfile, + "--foreground", + NULL); + + if (proc->source->readonly) + virCommandAddArg(cmd, "--readonly"); + + if (qemuNbdkitCapsGet(proc->caps, QEMU_NBDKIT_CAPS_FILTER_READAHEAD) && + proc->source->readahead > 0) + virCommandAddArgPair(cmd, "--filter", "readahead"); + + switch (proc->source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (qemuNbdkitProcessBuildCommandCurl(proc, cmd) < 0) + return NULL; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (qemuNbdkitProcessBuildCommandSSH(proc, cmd) < 0) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportError(VIR_ERR_NO_SUPPORT, + _("protocol '%s' is not supported by nbdkit"), + virStorageNetProtocolTypeToString(proc->source-= >protocol)); + return NULL; + } + + virCommandDaemonize(cmd); + + return g_steal_pointer(&cmd); +} + + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { @@ -675,3 +837,96 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) g_clear_object(&proc->caps); g_free(proc); } + + +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver) +{ + g_autoptr(virCommand) cmd =3D NULL; + int rc; + int exitstatus =3D 0; + int cmdret =3D 0; + VIR_AUTOCLOSE errfd =3D -1; + virTimeBackOffVar timebackoff; + bool socketCreated =3D false; + + if (!(cmd =3D qemuNbdkitProcessBuildCommand(proc))) + return -1; + + VIR_DEBUG("starting nbdkit process for %s", proc->source->nodestorage); + virCommandSetErrorFD(cmd, &errfd); + virCommandSetPidFile(cmd, proc->pidfile); + + if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) + goto error; + + if (qemuSecurityCommandRun(driver, vm, cmd, proc->user, proc->group, &= exitstatus, &cmdret) < 0) + goto error; + + if (cmdret < 0 || exitstatus !=3D 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not start 'nbdkit'. exitstatus: %d"), exit= status); + goto error; + } + + if ((rc =3D virPidFileReadPath(proc->pidfile, &proc->pid)) < 0) { + virReportSystemError(-rc, + _("Failed to read pidfile %s"), + proc->pidfile); + goto error; + } + + if (virTimeBackOffStart(&timebackoff, 1, 1000) < 0) + goto error; + + while (virTimeBackOffWait(&timebackoff)) { + if ((socketCreated =3D virFileExists(proc->socketfile))) + break; + + if (virProcessKill(proc->pid, 0) =3D=3D 0) + continue; + + goto error; + } + + if (!socketCreated) { + virReportError(VIR_ERR_OPERATION_TIMEOUT, "%s", + _("nbdkit socket did not show up")); + goto error; + } + + return 0; + + error: + if (errfd > 0) { + g_autofree char *errbuf =3D g_new0(char, 1024); + if (read(errfd, errbuf, 1024) > 0) + virReportError(VIR_ERR_OPERATION_FAILED, + _("nbdkit failed to start and reported: %s"), e= rrbuf); + } + qemuNbdkitProcessStop(proc); + return -1; +} + + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc) +{ + if (proc->pid < 0) + return 0; + + VIR_DEBUG("Stopping nbdkit process %i", proc->pid); + unlink(proc->pidfile); + unlink(proc->socketfile); + + if (virProcessKill(proc->pid, SIGTERM) < 0) { + virReportSystemError(errno, _("Failed to stop nbdkit process %i"),= proc->pid); + return -1; + } + + proc->pid =3D -1; + + return 0; +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 8844bba13c..ccd418b7d3 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -38,6 +38,8 @@ typedef enum { =20 VIR_ENUM_DECL(qemuNbdkitCaps); =20 +typedef struct _virQEMUDriver virQEMUDriver; + qemuNbdkitCaps * qemuNbdkitCapsNew(const char *path); =20 @@ -74,6 +76,14 @@ struct _qemuNbdkitProcess { pid_t pid; }; =20 +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver); + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc); + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); =20 --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252337; cv=none; d=zohomail.com; s=zohoarc; b=K245nqwnkFsPN6bq8Ko3BYOwt82m101eDtS5FOvSHtJohhVagmDpG5LwGXoUEfz6yV/yr9TR7i18t+gpwlUOZN/ovb3Zt7c9VTOPHoTDeBya8dBLijoDjDbUB4prRHrS5PihTrZQcqrdJra2RT2S9acsCQbKn1vnvcvpxUJ/B1c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252337; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9ENYsZJsyr4jtSLzoZCWkvQtvMvq2/bV/0AdTo75kFk=; b=ZjM1pveevFCuKn+5ZORZufegXeYVOZNqzcapHY0bY3UDjTK6xI0YqPriqcuj7SpDt90A9LeodogrmHDZyOxabSRTpcftn7k5GYgcAww1Sc5OF5SJY5w7QzZpiVbWbp4FjSrrA8CAYpNKixDuVrQqn+EErukdJOpzBEhDlDyckcI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252337413584.4927012008684; Fri, 20 Jan 2023 14:05:37 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-323-hxJBtkujNYCobCGzD0QjkQ-1; Fri, 20 Jan 2023 17:03:49 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2CD2086C176; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1566C14171C8; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0BD9C1946A7C; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6880A19465B6 for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 5BFEE140EBF6; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3AB5D1415113 for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252335; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=9ENYsZJsyr4jtSLzoZCWkvQtvMvq2/bV/0AdTo75kFk=; b=L/Y5YbOhYPFpwgTiGpWyukAYxs1mAdJ+/2jU2HMVBNG6M8EWCeA1gcqKCVCj+A7NvSTMRN w2975cs04U/rxhGJsDi5rnsTaDit1GzNWcr9oKqoYUrrkkSbDnAyWhRJN/i+FIhZKz/L/r 8q+uIbAsh1sTRZNT7aBFbFTeAYOMXjs= X-MC-Unique: hxJBtkujNYCobCGzD0QjkQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 11/31] qemu: remove unused 'mode' param from qemuDomainLogContextNew() Date: Fri, 20 Jan 2023 16:03:05 -0600 Message-Id: <20230120220325.1015090-12-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252338237100001 Content-Type: text/plain; charset="utf-8"; x-default="true" The only use of this function always passes QEMU_DOMAIN_LOG_CONTEXT_MODE_START. The other enum values are never used anywhere in the code. Remove them. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 26 +++++++++++--------------- src/qemu/qemu_domain.h | 9 +-------- src/qemu/qemu_process.c | 3 +-- 3 files changed, 13 insertions(+), 25 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index bcf051d6b1..b01f035718 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -6791,8 +6791,7 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, =20 =20 qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm, - qemuDomainLogContextMode m= ode) + virDomainObj *vm) { g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); qemuDomainLogContext *ctxt =3D QEMU_DOMAIN_LOG_CONTEXT(g_object_new(QE= MU_TYPE_DOMAIN_LOG_CONTEXT, NULL)); @@ -6833,25 +6832,22 @@ qemuDomainLogContext *qemuDomainLogContextNew(virQE= MUDriver *driver, /* For unprivileged startup we must truncate the file since * we can't rely on logrotate. We don't use O_TRUNC since * it is better for SELinux policy if we truncate afterwards */ - if (mode =3D=3D QEMU_DOMAIN_LOG_CONTEXT_MODE_START && - !driver->privileged && + if (!driver->privileged && ftruncate(ctxt->writefd, 0) < 0) { virReportSystemError(errno, _("failed to truncate %s"), ctxt->path); goto error; } =20 - if (mode =3D=3D QEMU_DOMAIN_LOG_CONTEXT_MODE_START) { - if ((ctxt->readfd =3D open(ctxt->path, O_RDONLY)) < 0) { - virReportSystemError(errno, _("failed to open logfile %s"), - ctxt->path); - goto error; - } - if (virSetCloseExec(ctxt->readfd) < 0) { - virReportSystemError(errno, _("failed to set close-on-exec= flag on %s"), - ctxt->path); - goto error; - } + if ((ctxt->readfd =3D open(ctxt->path, O_RDONLY)) < 0) { + virReportSystemError(errno, _("failed to open logfile %s"), + ctxt->path); + goto error; + } + if (virSetCloseExec(ctxt->readfd) < 0) { + virReportSystemError(errno, _("failed to set close-on-exec fla= g on %s"), + ctxt->path); + goto error; } =20 if ((ctxt->pos =3D lseek(ctxt->writefd, 0, SEEK_END)) < 0) { diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index a3b9acab70..37482a59f3 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -657,15 +657,8 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, virDomainNetDef *net, qemuDomainLogContext *logCtxt); =20 -typedef enum { - QEMU_DOMAIN_LOG_CONTEXT_MODE_START, - QEMU_DOMAIN_LOG_CONTEXT_MODE_ATTACH, - QEMU_DOMAIN_LOG_CONTEXT_MODE_STOP, -} qemuDomainLogContextMode; - qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm, - qemuDomainLogContextMode m= ode); + virDomainObj *vm); int qemuDomainLogContextWrite(qemuDomainLogContext *ctxt, const char *fmt, ...) G_GNUC_PRINTF(2, 3); ssize_t qemuDomainLogContextRead(qemuDomainLogContext *ctxt, diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index ee9f0784d3..fb3369b6a5 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7683,8 +7683,7 @@ qemuProcessLaunch(virConnectPtr conn, hookData.cfg =3D cfg; =20 VIR_DEBUG("Creating domain log file"); - if (!(logCtxt =3D qemuDomainLogContextNew(driver, vm, - QEMU_DOMAIN_LOG_CONTEXT_MODE_S= TART))) { + if (!(logCtxt =3D qemuDomainLogContextNew(driver, vm))) { virLastErrorPrefixMessage("%s", _("can't connect to virtlogd")); goto cleanup; } --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252275; cv=none; d=zohomail.com; s=zohoarc; b=Atb0PYCG47SReiSCGRH1Y6D3+VU7ac8Jm52AZs3FMU14ByKNOem2B9PZx81caGHd1zngGSBh1yAcQL/jnw4Dq4Gy8q6jFPS7i1nLYGjwUUQk6wzkdcSX1oKeiwFrECYRULGRH5iivJdi29CIJluYeZPY35ycSAd5XENlxhm3K2E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252275; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FmIUj1kKi3IqKR2EfCRUvifZ2RByyGDIsS5pFXytiYg=; b=dF2yR7CO0oNOjKbWUC3DcPDDfNccOgoiDKP0Xg5rpYVfqfFZMZ6F4Tbni93OT1UHxgxhC0yEdQe6T7tL+JJVAOcpPDWjhD+BlNI1/4NvQXZnIVQjxXNm2+QYfjpS7lt9tLWJTCyZ89+d9+E5LOJdMD7IQyEQT18IWr2VNi875hM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252275542386.55153899353047; Fri, 20 Jan 2023 14:04:35 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-587-K8J3m4GaOgKqBTb1eRmZRQ-1; Fri, 20 Jan 2023 17:03:52 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7ED1B2802E59; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 65E51492B01; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 497011946A7A; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A29401946A7C for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 974B414171C1; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6F98A14171C0 for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252274; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=FmIUj1kKi3IqKR2EfCRUvifZ2RByyGDIsS5pFXytiYg=; b=ZwL/p2sA4OirFWPG+HUsaZTor/lP+O7D3mFf4KFIlcX0Y1A3ggzr50XJyrjss7HOmyef25 swkV5SHNfrjR7jEeXggvHzNQUAwiL/5e1avrNYUZKzsp96DMEF5UDFe0m2nwgD133yHsax 8o1A4ZaPWNG3d2Gf19Y66v8zuv4RKoc= X-MC-Unique: K8J3m4GaOgKqBTb1eRmZRQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 12/31] Generalize qemuDomainLogContextNew() Date: Fri, 20 Jan 2023 16:03:06 -0600 Message-Id: <20230120220325.1015090-13-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252276004100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Allow to specify a basename for the log file so that qemuDomainLogContextNew() can be used to create log contexts for secondary loggers. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 5 +++-- src/qemu/qemu_domain.h | 3 ++- src/qemu/qemu_process.c | 2 +- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index b01f035718..8961efa804 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -6791,7 +6791,8 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, =20 =20 qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm) + virDomainObj *vm, + const char *basename) { g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); qemuDomainLogContext *ctxt =3D QEMU_DOMAIN_LOG_CONTEXT(g_object_new(QE= MU_TYPE_DOMAIN_LOG_CONTEXT, NULL)); @@ -6800,7 +6801,7 @@ qemuDomainLogContext *qemuDomainLogContextNew(virQEMU= Driver *driver, ctxt->writefd =3D -1; ctxt->readfd =3D -1; =20 - ctxt->path =3D g_strdup_printf("%s/%s.log", cfg->logDir, vm->def->name= ); + ctxt->path =3D g_strdup_printf("%s/%s.log", cfg->logDir, basename); =20 if (cfg->stdioLogD) { ctxt->manager =3D virLogManagerNew(driver->privileged); diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 37482a59f3..2a01c7e631 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -658,7 +658,8 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, qemuDomainLogContext *logCtxt); =20 qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm); + virDomainObj *vm, + const char *basename); int qemuDomainLogContextWrite(qemuDomainLogContext *ctxt, const char *fmt, ...) G_GNUC_PRINTF(2, 3); ssize_t qemuDomainLogContextRead(qemuDomainLogContext *ctxt, diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index fb3369b6a5..a2118c4f09 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7683,7 +7683,7 @@ qemuProcessLaunch(virConnectPtr conn, hookData.cfg =3D cfg; =20 VIR_DEBUG("Creating domain log file"); - if (!(logCtxt =3D qemuDomainLogContextNew(driver, vm))) { + if (!(logCtxt =3D qemuDomainLogContextNew(driver, vm, vm->def->name)))= { virLastErrorPrefixMessage("%s", _("can't connect to virtlogd")); goto cleanup; } --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252325; cv=none; d=zohomail.com; s=zohoarc; b=RMaChZpxc4XYDjXPDNjmL+ApwRhLRhL2FKrafEFzv7TBrUIXdHMtYwhKfUowm6asVXXPxJV4ZSURYL6F3iwuteDS4QrBauSqIhFuP9B55ZLeWqULVoPxXcSgZW76fsZgd1SzZU/BsQnacOLgybxU5xYaYJsLJt0p1uiBX1yHH4A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252325; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3mPDTn2+tiM/tVnUAn5cGTsryQNEns2XI0ILkFu7ReI=; b=EU5P09PFlQ6Qe38YS5ajGqcw51NdG/kxaIHbtUsqPbVWvIGLio7SNAn3fS5KwtZ93qkbuWS4IQ2YAVvLhh18XZ/j7J4ajsspfALGvJTg7hknFYNp8EB9Mq5KXc/DMVxJEIveJP12v2yxVCTd57ZSUL4XpVAc8L+m4xicuJ/UdaM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252325223104.57016723305912; Fri, 20 Jan 2023 14:05:25 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-179--UhwW6spN_OOU09QcW3rpQ-1; Fri, 20 Jan 2023 17:03:44 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 85CC838173E7; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 712892166B2B; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 626B71946588; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id DB74F194704F for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id CE8A614171C1; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A534C140EBF6 for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252322; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3mPDTn2+tiM/tVnUAn5cGTsryQNEns2XI0ILkFu7ReI=; b=fltBQhHVVIvU+p+xr6XHberyzx77LyObMcDAbqgViqQw0zB+IErvFSncdYybg3Qpy5eTsV jXcrJd3v6Y9fCSObE/l/ttgS1u2pRnzS0fvCikk1wZgzXfTMjTSw85n+/HQlc8uNMXXacD Bpq6KiCrvCpvkXNtMcTsoSyAS8+oCn8= X-MC-Unique: -UhwW6spN_OOU09QcW3rpQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 13/31] qemu: Extract qemuDomainLogContext into a new file Date: Fri, 20 Jan 2023 16:03:07 -0600 Message-Id: <20230120220325.1015090-14-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252326235100001 Content-Type: text/plain; charset="utf-8"; x-default="true" This will allow us to use it for nbdkit logging in upcoming commits. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- po/POTFILES | 1 + src/qemu/meson.build | 1 + src/qemu/qemu_domain.c | 247 ++-------------------------------- src/qemu/qemu_domain.h | 29 +--- src/qemu/qemu_logcontext.c | 264 +++++++++++++++++++++++++++++++++++++ src/qemu/qemu_logcontext.h | 38 ++++++ src/qemu/qemu_process.c | 44 +++---- 7 files changed, 347 insertions(+), 277 deletions(-) create mode 100644 src/qemu/qemu_logcontext.c create mode 100644 src/qemu/qemu_logcontext.h diff --git a/po/POTFILES b/po/POTFILES index a45259c0d8..024575a0d9 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -172,6 +172,7 @@ src/qemu/qemu_hostdev.c src/qemu/qemu_hotplug.c src/qemu/qemu_interface.c src/qemu/qemu_interop_config.c +src/qemu/qemu_logcontext.c src/qemu/qemu_migration.c src/qemu/qemu_migration_cookie.c src/qemu/qemu_migration_params.c diff --git a/src/qemu/meson.build b/src/qemu/meson.build index 9be6996195..6d7a1bfbb0 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -21,6 +21,7 @@ qemu_driver_sources =3D [ 'qemu_hotplug.c', 'qemu_interface.c', 'qemu_interop_config.c', + 'qemu_logcontext.c', 'qemu_migration.c', 'qemu_migration_cookie.c', 'qemu_migration_params.c', diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 8961efa804..63c2c602ec 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -454,21 +454,8 @@ qemuDomainObjFromDomain(virDomainPtr domain) } =20 =20 -struct _qemuDomainLogContext { - GObject parent; - - int writefd; - int readfd; /* Only used if manager =3D=3D NULL */ - off_t pos; - ino_t inode; /* Only used if manager !=3D NULL */ - char *path; - virLogManager *manager; -}; - -G_DEFINE_TYPE(qemuDomainLogContext, qemu_domain_log_context, G_TYPE_OBJECT= ); static virClass *qemuDomainSaveCookieClass; =20 -static void qemuDomainLogContextFinalize(GObject *obj); static void qemuDomainSaveCookieDispose(void *obj); =20 =20 @@ -481,32 +468,8 @@ qemuDomainOnceInit(void) return 0; } =20 -static void qemu_domain_log_context_init(qemuDomainLogContext *logctxt G_G= NUC_UNUSED) -{ -} - -static void qemu_domain_log_context_class_init(qemuDomainLogContextClass *= klass) -{ - GObjectClass *obj =3D G_OBJECT_CLASS(klass); - - obj->finalize =3D qemuDomainLogContextFinalize; -} - VIR_ONCE_GLOBAL_INIT(qemuDomain); =20 -static void -qemuDomainLogContextFinalize(GObject *object) -{ - qemuDomainLogContext *ctxt =3D QEMU_DOMAIN_LOG_CONTEXT(object); - VIR_DEBUG("ctxt=3D%p", ctxt); - - virLogManagerFree(ctxt->manager); - VIR_FREE(ctxt->path); - VIR_FORCE_CLOSE(ctxt->writefd); - VIR_FORCE_CLOSE(ctxt->readfd); - G_OBJECT_CLASS(qemu_domain_log_context_parent_class)->finalize(object); -} - /* qemuDomainGetMasterKeyFilePath: * @libDir: Directory path to domain lib files * @@ -6555,7 +6518,7 @@ qemuDomainDefFormatLive(virQEMUDriver *driver, void qemuDomainObjTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainTaintFlags taint, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { qemuDomainObjTaintMsg(driver, obj, taint, logCtxt, NULL); qemuDomainSaveStatus(obj); @@ -6564,7 +6527,7 @@ void qemuDomainObjTaint(virQEMUDriver *driver, void qemuDomainObjTaintMsg(virQEMUDriver *driver, virDomainObj *obj, virDomainTaintFlags taint, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, const char *fmt, ...) { virErrorPtr orig_err =3D NULL; @@ -6617,12 +6580,12 @@ void qemuDomainObjTaintMsg(virQEMUDriver *driver, goto cleanup; =20 if (logCtxt) { - rc =3D qemuDomainLogContextWrite(logCtxt, - "%s: Domain id=3D%d is tainted: %s%= s%s%s\n", - timestamp, - obj->def->id, - virDomainTaintTypeToString(taint), - extraprefix, extramsg, extrasuffix); + rc =3D qemuLogContextWrite(logCtxt, + "%s: Domain id=3D%d is tainted: %s%s%s%s\= n", + timestamp, + obj->def->id, + virDomainTaintTypeToString(taint), + extraprefix, extramsg, extrasuffix); } else { rc =3D qemuDomainLogAppendMessage(driver, obj, "%s: Domain id=3D%d is tainted: %s= %s%s%s\n", @@ -6642,7 +6605,7 @@ void qemuDomainObjTaintMsg(virQEMUDriver *driver, static void qemuDomainObjCheckMachineTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { qemuDomainObjPrivate *priv =3D obj->privateData; virQEMUCaps *qemuCaps =3D priv->qemuCaps; @@ -6660,7 +6623,7 @@ qemuDomainObjCheckMachineTaint(virQEMUDriver *driver, static void qemuDomainObjCheckCPUTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool incomingMigration) { qemuDomainObjPrivate *priv =3D obj->privateData; @@ -6692,7 +6655,7 @@ qemuDomainObjCheckCPUTaint(virQEMUDriver *driver, =20 void qemuDomainObjCheckTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool incomingMigration) { size_t i; @@ -6748,7 +6711,7 @@ void qemuDomainObjCheckTaint(virQEMUDriver *driver, void qemuDomainObjCheckDiskTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainDiskDef *disk, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { if (disk->rawio =3D=3D VIR_TRISTATE_BOOL_YES) qemuDomainObjTaint(driver, obj, VIR_DOMAIN_TAINT_HIGH_PRIVILEGES, @@ -6765,7 +6728,7 @@ void qemuDomainObjCheckDiskTaint(virQEMUDriver *drive= r, void qemuDomainObjCheckHostdevTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainHostdevDef *hostdev, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { if (!virHostdevIsSCSIDevice(hostdev)) return; @@ -6778,7 +6741,7 @@ void qemuDomainObjCheckHostdevTaint(virQEMUDriver *dr= iver, void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainNetDef *net, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { /* script is only useful for NET_TYPE_ETHERNET (qemu) and * NET_TYPE_BRIDGE (xen), but could be (incorrectly) specified for @@ -6790,163 +6753,6 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driv= er, } =20 =20 -qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm, - const char *basename) -{ - g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); - qemuDomainLogContext *ctxt =3D QEMU_DOMAIN_LOG_CONTEXT(g_object_new(QE= MU_TYPE_DOMAIN_LOG_CONTEXT, NULL)); - - VIR_DEBUG("Context new %p stdioLogD=3D%d", ctxt, cfg->stdioLogD); - ctxt->writefd =3D -1; - ctxt->readfd =3D -1; - - ctxt->path =3D g_strdup_printf("%s/%s.log", cfg->logDir, basename); - - if (cfg->stdioLogD) { - ctxt->manager =3D virLogManagerNew(driver->privileged); - if (!ctxt->manager) - goto error; - - ctxt->writefd =3D virLogManagerDomainOpenLogFile(ctxt->manager, - "qemu", - vm->def->uuid, - vm->def->name, - ctxt->path, - 0, - &ctxt->inode, - &ctxt->pos); - if (ctxt->writefd < 0) - goto error; - } else { - if ((ctxt->writefd =3D open(ctxt->path, O_WRONLY | O_CREAT | O_APP= END, S_IRUSR | S_IWUSR)) < 0) { - virReportSystemError(errno, _("failed to create logfile %s"), - ctxt->path); - goto error; - } - if (virSetCloseExec(ctxt->writefd) < 0) { - virReportSystemError(errno, _("failed to set close-on-exec fla= g on %s"), - ctxt->path); - goto error; - } - - /* For unprivileged startup we must truncate the file since - * we can't rely on logrotate. We don't use O_TRUNC since - * it is better for SELinux policy if we truncate afterwards */ - if (!driver->privileged && - ftruncate(ctxt->writefd, 0) < 0) { - virReportSystemError(errno, _("failed to truncate %s"), - ctxt->path); - goto error; - } - - if ((ctxt->readfd =3D open(ctxt->path, O_RDONLY)) < 0) { - virReportSystemError(errno, _("failed to open logfile %s"), - ctxt->path); - goto error; - } - if (virSetCloseExec(ctxt->readfd) < 0) { - virReportSystemError(errno, _("failed to set close-on-exec fla= g on %s"), - ctxt->path); - goto error; - } - - if ((ctxt->pos =3D lseek(ctxt->writefd, 0, SEEK_END)) < 0) { - virReportSystemError(errno, _("failed to seek in log file %s"), - ctxt->path); - goto error; - } - } - - return ctxt; - - error: - g_clear_object(&ctxt); - return NULL; -} - - -int qemuDomainLogContextWrite(qemuDomainLogContext *ctxt, - const char *fmt, ...) -{ - va_list argptr; - g_autofree char *message =3D NULL; - int ret =3D -1; - - va_start(argptr, fmt); - - message =3D g_strdup_vprintf(fmt, argptr); - if (!ctxt->manager && - lseek(ctxt->writefd, 0, SEEK_END) < 0) { - virReportSystemError(errno, "%s", - _("Unable to seek to end of domain logfile")); - goto cleanup; - } - if (safewrite(ctxt->writefd, message, strlen(message)) < 0) { - virReportSystemError(errno, "%s", - _("Unable to write to domain logfile")); - goto cleanup; - } - - ret =3D 0; - - cleanup: - va_end(argptr); - return ret; -} - - -ssize_t qemuDomainLogContextRead(qemuDomainLogContext *ctxt, - char **msg) -{ - char *buf; - size_t buflen; - - VIR_DEBUG("Context read %p manager=3D%p inode=3D%llu pos=3D%llu", - ctxt, ctxt->manager, - (unsigned long long)ctxt->inode, - (unsigned long long)ctxt->pos); - - if (ctxt->manager) { - buf =3D virLogManagerDomainReadLogFile(ctxt->manager, - ctxt->path, - ctxt->inode, - ctxt->pos, - 1024 * 128, - 0); - if (!buf) - return -1; - buflen =3D strlen(buf); - } else { - ssize_t got; - - buflen =3D 1024 * 128; - - /* Best effort jump to start of messages */ - ignore_value(lseek(ctxt->readfd, ctxt->pos, SEEK_SET)); - - buf =3D g_new0(char, buflen); - - got =3D saferead(ctxt->readfd, buf, buflen - 1); - if (got < 0) { - VIR_FREE(buf); - virReportSystemError(errno, "%s", - _("Unable to read from log file")); - return -1; - } - - buf[got] =3D '\0'; - - buf =3D g_renew(char, buf, got + 1); - buflen =3D got; - } - - *msg =3D buf; - - return buflen; -} - - /** * qemuDomainLogAppendMessage: * @@ -7004,31 +6810,6 @@ qemuDomainLogAppendMessage(virQEMUDriver *driver, } =20 =20 -int qemuDomainLogContextGetWriteFD(qemuDomainLogContext *ctxt) -{ - return ctxt->writefd; -} - - -void qemuDomainLogContextMarkPosition(qemuDomainLogContext *ctxt) -{ - if (ctxt->manager) - virLogManagerDomainGetLogFilePosition(ctxt->manager, - ctxt->path, - 0, - &ctxt->inode, - &ctxt->pos); - else - ctxt->pos =3D lseek(ctxt->writefd, 0, SEEK_END); -} - - -virLogManager *qemuDomainLogContextGetManager(qemuDomainLogContext *ctxt) -{ - return ctxt->manager; -} - - /* Locate an appropriate 'qemu-img' binary. */ const char * qemuFindQemuImgBinary(virQEMUDriver *driver) diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 2a01c7e631..318865f920 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -32,13 +32,13 @@ #include "qemu_domainjob.h" #include "qemu_conf.h" #include "qemu_capabilities.h" +#include "qemu_logcontext.h" #include "qemu_migration_params.h" #include "qemu_nbdkit.h" #include "qemu_slirp.h" #include "qemu_fd.h" #include "virchrdev.h" #include "virobject.h" -#include "logging/log_manager.h" #include "virdomainmomentobjlist.h" #include "virenum.h" #include "vireventthread.h" @@ -473,9 +473,6 @@ struct qemuProcessEvent { =20 void qemuProcessEventFree(struct qemuProcessEvent *event); =20 -#define QEMU_TYPE_DOMAIN_LOG_CONTEXT qemu_domain_log_context_get_type() -G_DECLARE_FINAL_TYPE(qemuDomainLogContext, qemu_domain_log_context, QEMU, = DOMAIN_LOG_CONTEXT, GObject); - typedef struct _qemuDomainSaveCookie qemuDomainSaveCookie; struct _qemuDomainSaveCookie { virObject parent; @@ -628,12 +625,12 @@ char *qemuDomainDefFormatLive(virQEMUDriver *driver, void qemuDomainObjTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainTaintFlags taint, - qemuDomainLogContext *logCtxt); + qemuLogContext *logCtxt); =20 void qemuDomainObjTaintMsg(virQEMUDriver *driver, virDomainObj *obj, virDomainTaintFlags taint, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, const char *msg, ...) G_GNUC_PRINTF(5, 6); =20 @@ -642,32 +639,20 @@ char **qemuDomainObjGetTainting(virQEMUDriver *driver, =20 void qemuDomainObjCheckTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool incomingMigration); void qemuDomainObjCheckDiskTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainDiskDef *disk, - qemuDomainLogContext *logCtxt); + qemuLogContext *logCtxt); void qemuDomainObjCheckHostdevTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainHostdevDef *disk, - qemuDomainLogContext *logCtxt); + qemuLogContext *logCtxt); void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainNetDef *net, - qemuDomainLogContext *logCtxt); - -qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm, - const char *basename); -int qemuDomainLogContextWrite(qemuDomainLogContext *ctxt, - const char *fmt, ...) G_GNUC_PRINTF(2, 3); -ssize_t qemuDomainLogContextRead(qemuDomainLogContext *ctxt, - char **msg); -int qemuDomainLogContextGetWriteFD(qemuDomainLogContext *ctxt); -void qemuDomainLogContextMarkPosition(qemuDomainLogContext *ctxt); - -virLogManager *qemuDomainLogContextGetManager(qemuDomainLogContext *ctxt); + qemuLogContext *logCtxt); =20 int qemuDomainLogAppendMessage(virQEMUDriver *driver, virDomainObj *vm, diff --git a/src/qemu/qemu_logcontext.c b/src/qemu/qemu_logcontext.c new file mode 100644 index 0000000000..6ad0beeeae --- /dev/null +++ b/src/qemu/qemu_logcontext.c @@ -0,0 +1,264 @@ +/* + * qemu_logcontext.c: QEMU log context + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + */ + +#include + +#include "qemu_logcontext.h" +#include "viralloc.h" +#include "virlog.h" +#include "virutil.h" + +#include + +#define VIR_FROM_THIS VIR_FROM_QEMU + +VIR_LOG_INIT("qemu.qemu_logcontext"); + + +struct _qemuLogContext { + GObject parent; + + int writefd; + int readfd; /* Only used if manager =3D=3D NULL */ + off_t pos; + ino_t inode; /* Only used if manager !=3D NULL */ + char *path; + virLogManager *manager; +}; + +G_DEFINE_TYPE(qemuLogContext, qemu_log_context, G_TYPE_OBJECT); + +static void +qemuLogContextFinalize(GObject *obj); + + +static void +qemu_log_context_init(qemuLogContext *logctxt G_GNUC_UNUSED) +{ +} + + +static void +qemu_log_context_class_init(qemuLogContextClass *klass) +{ + GObjectClass *obj =3D G_OBJECT_CLASS(klass); + + obj->finalize =3D qemuLogContextFinalize; +} + + +static void +qemuLogContextFinalize(GObject *object) +{ + qemuLogContext *ctxt =3D QEMU_LOG_CONTEXT(object); + VIR_DEBUG("ctxt=3D%p", ctxt); + + virLogManagerFree(ctxt->manager); + VIR_FREE(ctxt->path); + VIR_FORCE_CLOSE(ctxt->writefd); + VIR_FORCE_CLOSE(ctxt->readfd); + G_OBJECT_CLASS(qemu_log_context_parent_class)->finalize(object); +} + + +qemuLogContext * +qemuLogContextNew(virQEMUDriver *driver, + virDomainObj *vm, + const char *basename) +{ + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + qemuLogContext *ctxt =3D QEMU_LOG_CONTEXT(g_object_new(QEMU_TYPE_LOG_C= ONTEXT, NULL)); + + VIR_DEBUG("Context new %p stdioLogD=3D%d", ctxt, cfg->stdioLogD); + ctxt->writefd =3D -1; + ctxt->readfd =3D -1; + + ctxt->path =3D g_strdup_printf("%s/%s.log", cfg->logDir, basename); + + if (cfg->stdioLogD) { + ctxt->manager =3D virLogManagerNew(driver->privileged); + if (!ctxt->manager) + goto error; + + ctxt->writefd =3D virLogManagerDomainOpenLogFile(ctxt->manager, + "qemu", + vm->def->uuid, + vm->def->name, + ctxt->path, + 0, + &ctxt->inode, + &ctxt->pos); + if (ctxt->writefd < 0) + goto error; + } else { + if ((ctxt->writefd =3D open(ctxt->path, O_WRONLY | O_CREAT | O_APP= END, S_IRUSR | S_IWUSR)) < 0) { + virReportSystemError(errno, _("failed to create logfile %s"), + ctxt->path); + goto error; + } + if (virSetCloseExec(ctxt->writefd) < 0) { + virReportSystemError(errno, _("failed to set close-on-exec fla= g on %s"), + ctxt->path); + goto error; + } + + /* For unprivileged startup we must truncate the file since + * we can't rely on logrotate. We don't use O_TRUNC since + * it is better for SELinux policy if we truncate afterwards */ + if (!driver->privileged && + ftruncate(ctxt->writefd, 0) < 0) { + virReportSystemError(errno, _("failed to truncate %s"), + ctxt->path); + goto error; + } + + if ((ctxt->readfd =3D open(ctxt->path, O_RDONLY)) < 0) { + virReportSystemError(errno, _("failed to open logfile %s"), + ctxt->path); + goto error; + } + if (virSetCloseExec(ctxt->readfd) < 0) { + virReportSystemError(errno, _("failed to set close-on-exec fla= g on %s"), + ctxt->path); + goto error; + } + + if ((ctxt->pos =3D lseek(ctxt->writefd, 0, SEEK_END)) < 0) { + virReportSystemError(errno, _("failed to seek in log file %s"), + ctxt->path); + goto error; + } + } + + return ctxt; + + error: + g_clear_object(&ctxt); + return NULL; +} + + +int +qemuLogContextWrite(qemuLogContext *ctxt, + const char *fmt, ...) +{ + va_list argptr; + g_autofree char *message =3D NULL; + int ret =3D -1; + + va_start(argptr, fmt); + + message =3D g_strdup_vprintf(fmt, argptr); + if (!ctxt->manager && + lseek(ctxt->writefd, 0, SEEK_END) < 0) { + virReportSystemError(errno, "%s", + _("Unable to seek to end of domain logfile")); + goto cleanup; + } + if (safewrite(ctxt->writefd, message, strlen(message)) < 0) { + virReportSystemError(errno, "%s", + _("Unable to write to domain logfile")); + goto cleanup; + } + + ret =3D 0; + + cleanup: + va_end(argptr); + return ret; +} + + +ssize_t +qemuLogContextRead(qemuLogContext *ctxt, + char **msg) +{ + char *buf; + size_t buflen; + + VIR_DEBUG("Context read %p manager=3D%p inode=3D%llu pos=3D%llu", + ctxt, ctxt->manager, + (unsigned long long)ctxt->inode, + (unsigned long long)ctxt->pos); + + if (ctxt->manager) { + buf =3D virLogManagerDomainReadLogFile(ctxt->manager, + ctxt->path, + ctxt->inode, + ctxt->pos, + 1024 * 128, + 0); + if (!buf) + return -1; + buflen =3D strlen(buf); + } else { + ssize_t got; + + buflen =3D 1024 * 128; + + /* Best effort jump to start of messages */ + ignore_value(lseek(ctxt->readfd, ctxt->pos, SEEK_SET)); + + buf =3D g_new0(char, buflen); + + got =3D saferead(ctxt->readfd, buf, buflen - 1); + if (got < 0) { + VIR_FREE(buf); + virReportSystemError(errno, "%s", + _("Unable to read from log file")); + return -1; + } + + buf[got] =3D '\0'; + + buf =3D g_renew(char, buf, got + 1); + buflen =3D got; + } + + *msg =3D buf; + + return buflen; +} + + +int +qemuLogContextGetWriteFD(qemuLogContext *ctxt) +{ + return ctxt->writefd; +} + + +void +qemuLogContextMarkPosition(qemuLogContext *ctxt) +{ + if (ctxt->manager) + virLogManagerDomainGetLogFilePosition(ctxt->manager, + ctxt->path, + 0, + &ctxt->inode, + &ctxt->pos); + else + ctxt->pos =3D lseek(ctxt->writefd, 0, SEEK_END); +} + + +virLogManager * +qemuLogContextGetManager(qemuLogContext *ctxt) +{ + return ctxt->manager; +} diff --git a/src/qemu/qemu_logcontext.h b/src/qemu/qemu_logcontext.h new file mode 100644 index 0000000000..c6dbf3cb84 --- /dev/null +++ b/src/qemu/qemu_logcontext.h @@ -0,0 +1,38 @@ +/* + * qemu_logcontext.h: QEMU log context + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + */ + +#pragma once + +#include +#include "qemu_conf.h" +#include "logging/log_manager.h" + +#define QEMU_TYPE_LOG_CONTEXT qemu_log_context_get_type() +G_DECLARE_FINAL_TYPE(qemuLogContext, qemu_log_context, QEMU, LOG_CONTEXT, = GObject); + +qemuLogContext *qemuLogContextNew(virQEMUDriver *driver, + virDomainObj *vm, + const char *basename); +int qemuLogContextWrite(qemuLogContext *ctxt, + const char *fmt, ...) G_GNUC_PRINTF(2, 3); +ssize_t qemuLogContextRead(qemuLogContext *ctxt, + char **msg); +int qemuLogContextGetWriteFD(qemuLogContext *ctxt); +void qemuLogContextMarkPosition(qemuLogContext *ctxt); + +virLogManager *qemuLogContextGetManager(qemuLogContext *ctxt); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a2118c4f09..db25f26057 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1812,7 +1812,7 @@ qemuProcessMonitorReportLogError(qemuMonitor *mon, static void qemuProcessMonitorLogFree(void *opaque) { - qemuDomainLogContext *logCtxt =3D opaque; + qemuLogContext *logCtxt =3D opaque; g_clear_object(&logCtxt); } =20 @@ -1838,7 +1838,7 @@ static int qemuConnectMonitor(virQEMUDriver *driver, virDomainObj *vm, int asyncJob, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool reconnect) { qemuDomainObjPrivate *priv =3D vm->privateData; @@ -1902,7 +1902,7 @@ qemuConnectMonitor(virQEMUDriver *driver, * Returns 0 on success or -1 on error */ static int -qemuProcessReadLog(qemuDomainLogContext *logCtxt, +qemuProcessReadLog(qemuLogContext *logCtxt, char **msg, size_t max) { @@ -1912,7 +1912,7 @@ qemuProcessReadLog(qemuDomainLogContext *logCtxt, char *filter_next; size_t skip; =20 - if ((got =3D qemuDomainLogContextRead(logCtxt, &buf)) < 0) + if ((got =3D qemuLogContextRead(logCtxt, &buf)) < 0) return -1; =20 /* Filter out debug messages from intermediate libvirt process */ @@ -1955,7 +1955,7 @@ qemuProcessReadLog(qemuDomainLogContext *logCtxt, =20 =20 static int -qemuProcessReportLogError(qemuDomainLogContext *logCtxt, +qemuProcessReportLogError(qemuLogContext *logCtxt, const char *msgprefix) { g_autofree char *logmsg =3D NULL; @@ -1980,7 +1980,7 @@ qemuProcessMonitorReportLogError(qemuMonitor *mon G_G= NUC_UNUSED, const char *msg, void *opaque) { - qemuDomainLogContext *logCtxt =3D opaque; + qemuLogContext *logCtxt =3D opaque; qemuProcessReportLogError(logCtxt, msg); } =20 @@ -2282,7 +2282,7 @@ static int qemuProcessWaitForMonitor(virQEMUDriver *driver, virDomainObj *vm, int asyncJob, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { int ret =3D -1; g_autoptr(GHashTable) info =3D NULL; @@ -4657,7 +4657,7 @@ static void qemuLogOperation(virDomainObj *vm, const char *msg, virCommand *cmd, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { g_autofree char *timestamp =3D NULL; qemuDomainObjPrivate *priv =3D vm->privateData; @@ -4671,20 +4671,20 @@ qemuLogOperation(virDomainObj *vm, if ((timestamp =3D virTimeStringNow()) =3D=3D NULL) return; =20 - if (qemuDomainLogContextWrite(logCtxt, - "%s: %s %s, qemu version: %d.%d.%d%s, ke= rnel: %s, hostname: %s\n", - timestamp, msg, VIR_LOG_VERSION_STRING, - (qemuVersion / 1000000) % 1000, - (qemuVersion / 1000) % 1000, - qemuVersion % 1000, - NULLSTR_EMPTY(package), - uts.release, - NULLSTR_EMPTY(hostname)) < 0) + if (qemuLogContextWrite(logCtxt, + "%s: %s %s, qemu version: %d.%d.%d%s, kernel: = %s, hostname: %s\n", + timestamp, msg, VIR_LOG_VERSION_STRING, + (qemuVersion / 1000000) % 1000, + (qemuVersion / 1000) % 1000, + qemuVersion % 1000, + NULLSTR_EMPTY(package), + uts.release, + NULLSTR_EMPTY(hostname)) < 0) return; =20 if (cmd) { g_autofree char *args =3D virCommandToString(cmd, true); - qemuDomainLogContextWrite(logCtxt, "%s\n", args); + qemuLogContextWrite(logCtxt, "%s\n", args); } } =20 @@ -7634,7 +7634,7 @@ qemuProcessLaunch(virConnectPtr conn, int ret =3D -1; int rv; int logfile =3D -1; - g_autoptr(qemuDomainLogContext) logCtxt =3D NULL; + g_autoptr(qemuLogContext) logCtxt =3D NULL; qemuDomainObjPrivate *priv =3D vm->privateData; g_autoptr(virCommand) cmd =3D NULL; struct qemuProcessHookData hookData; @@ -7683,11 +7683,11 @@ qemuProcessLaunch(virConnectPtr conn, hookData.cfg =3D cfg; =20 VIR_DEBUG("Creating domain log file"); - if (!(logCtxt =3D qemuDomainLogContextNew(driver, vm, vm->def->name)))= { + if (!(logCtxt =3D qemuLogContextNew(driver, vm, vm->def->name))) { virLastErrorPrefixMessage("%s", _("can't connect to virtlogd")); goto cleanup; } - logfile =3D qemuDomainLogContextGetWriteFD(logCtxt); + logfile =3D qemuLogContextGetWriteFD(logCtxt); =20 if (qemuProcessGenID(vm, flags) < 0) goto cleanup; @@ -7717,7 +7717,7 @@ qemuProcessLaunch(virConnectPtr conn, =20 qemuDomainObjCheckTaint(driver, vm, logCtxt, incoming !=3D NULL); =20 - qemuDomainLogContextMarkPosition(logCtxt); + qemuLogContextMarkPosition(logCtxt); =20 VIR_DEBUG("Building mount namespace"); =20 --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252315; cv=none; d=zohomail.com; s=zohoarc; b=XvxzIdLBmatHj+YZNEOR512cg709psqe4XDkrbve+twcyKSokfnTM29Uu7Z2qJK/cWVDFn4NepIOuFgvOWWikT8khSADlVPbNo1aNzlhgexz7Fb2AL0SI14DAAfDJbsuLPjOlStFb9EIguiIFf98lz6BjJfurUI205fgHXEE7Jo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252315; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HK97PynrklkZixfC4oITkUxvRstHroTd3uC1+BPeww0=; b=X9iRkywhq5MTo0IavoJNTe5P0oK++wOqMSDv5KohMGfChHAYOHW6KsJaUOT5rU+k5DWa6D26rXMgZXMx1m2laJUbv4mXBXJZPKaYB44gqVsGjaprPZnSvZUAqnOgsZBvEZe1bR3G5Sddg8t6Q3Qycdmzz2jo9Setl+Apk0IPSlg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252315702887.4459966435135; Fri, 20 Jan 2023 14:05:15 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-505-oMaUcNULMCO7n63ZlzLuRw-1; Fri, 20 Jan 2023 17:03:46 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E155380556A; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id CAB55C15BAE; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A631A1946A7E; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1E49A1947057 for ; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 0EDEE14171C5; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E1210140EBF6 for ; Fri, 20 Jan 2023 22:03:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252314; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=HK97PynrklkZixfC4oITkUxvRstHroTd3uC1+BPeww0=; b=etyfFmLpfrT1JsQHQUTZgkSzYLT+hc2gywQcQpxiSDNIj9Yyr6i756Hap5Bf+e54BqRgSx MWAVu1hdUCOFKn6ib8lL64JVTM5fmcJPdOwvz3rmqAzs4C1PHtv84a7AxLhac4Vsq3Cwil J0wLqN3Gsg5bqyA8g+qCSt8TGPmCfmU= X-MC-Unique: oMaUcNULMCO7n63ZlzLuRw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 14/31] qemu: move qemuProcessReadLog() to qemuLogContext Date: Fri, 20 Jan 2023 16:03:08 -0600 Message-Id: <20230120220325.1015090-15-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252316174100007 Content-Type: text/plain; charset="utf-8"; x-default="true" This code can be used by the nbdkit implementation for reading back filtered log data for error reporting. Move it to qemuLogContext so that it can be shared. Renamed to qemuLogContextReadFiltered(). Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_logcontext.c | 65 ++++++++++++++++++++++++++++++++++++ src/qemu/qemu_logcontext.h | 3 ++ src/qemu/qemu_process.c | 67 +------------------------------------- 3 files changed, 69 insertions(+), 66 deletions(-) diff --git a/src/qemu/qemu_logcontext.c b/src/qemu/qemu_logcontext.c index 6ad0beeeae..1d19700eab 100644 --- a/src/qemu/qemu_logcontext.c +++ b/src/qemu/qemu_logcontext.c @@ -21,6 +21,7 @@ #include "qemu_logcontext.h" #include "viralloc.h" #include "virlog.h" +#include "virstring.h" #include "virutil.h" =20 #include @@ -236,6 +237,70 @@ qemuLogContextRead(qemuLogContext *ctxt, } =20 =20 +/** + * qemuLogContextFilter: Read and filter log for relevant messages + * @ctxt: the domain log context + * @msg: pointer to buffer to store the read messages in + * @max: maximum length of the message returned in @msg after filtering + * + * Reads log output from @ctxt and filters it. Skips messages not produced= by + * the target executable or irrelevant messages. If @max is not zero, @buf= will + * contain at most @max characters from the end of the log and @buf will s= tart + * after a new line if possible. + */ +int +qemuLogContextReadFiltered(qemuLogContext *ctxt, + char **msg, + size_t max) +{ + char *buf; + char *eol; + char *filter_next; + size_t skip; + ssize_t got; + + if ((got =3D qemuLogContextRead(ctxt, &buf) < 0)) + return -1; + + /* Filter out debug messages from intermediate libvirt process */ + filter_next =3D buf; + while ((eol =3D strchr(filter_next, '\n'))) { + *eol =3D '\0'; + if (virLogProbablyLogMessage(filter_next) || + strstr(filter_next, "char device redirected to")) { + skip =3D (eol + 1) - filter_next; + memmove(filter_next, eol + 1, buf + got - eol); + got -=3D skip; + } else { + filter_next =3D eol + 1; + *eol =3D '\n'; + } + } + + if (got > 0 && + buf[got - 1] =3D=3D '\n') { + buf[got - 1] =3D '\0'; + got--; + } + + if (max > 0 && got > max) { + skip =3D got - max; + + if (buf[skip - 1] !=3D '\n' && + (eol =3D strchr(buf + skip, '\n')) && + !virStringIsEmpty(eol + 1)) + skip =3D eol + 1 - buf; + + memmove(buf, buf + skip, got - skip + 1); + got -=3D skip; + } + + buf =3D g_renew(char, buf, got + 1); + *msg =3D buf; + return 0; +} + + int qemuLogContextGetWriteFD(qemuLogContext *ctxt) { diff --git a/src/qemu/qemu_logcontext.h b/src/qemu/qemu_logcontext.h index c6dbf3cb84..669a05ae97 100644 --- a/src/qemu/qemu_logcontext.h +++ b/src/qemu/qemu_logcontext.h @@ -32,6 +32,9 @@ int qemuLogContextWrite(qemuLogContext *ctxt, const char *fmt, ...) G_GNUC_PRINTF(2, 3); ssize_t qemuLogContextRead(qemuLogContext *ctxt, char **msg); +int qemuLogContextReadFiltered(qemuLogContext *ctxt, + char **msg, + size_t max); int qemuLogContextGetWriteFD(qemuLogContext *ctxt); void qemuLogContextMarkPosition(qemuLogContext *ctxt); =20 diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index db25f26057..6d1751b5d7 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1889,71 +1889,6 @@ qemuConnectMonitor(virQEMUDriver *driver, } =20 =20 -/** - * qemuProcessReadLog: Read log file of a qemu VM - * @logCtxt: the domain log context - * @msg: pointer to buffer to store the read messages in - * @max: maximum length of the message returned in @msg - * - * Reads log of a qemu VM. Skips messages not produced by qemu or irreleva= nt - * messages. If @max is not zero, @msg will contain at most @max characters - * from the end of the log and @msg will start after a new line if possibl= e. - * - * Returns 0 on success or -1 on error - */ -static int -qemuProcessReadLog(qemuLogContext *logCtxt, - char **msg, - size_t max) -{ - char *buf; - ssize_t got; - char *eol; - char *filter_next; - size_t skip; - - if ((got =3D qemuLogContextRead(logCtxt, &buf)) < 0) - return -1; - - /* Filter out debug messages from intermediate libvirt process */ - filter_next =3D buf; - while ((eol =3D strchr(filter_next, '\n'))) { - *eol =3D '\0'; - if (virLogProbablyLogMessage(filter_next) || - strstr(filter_next, "char device redirected to")) { - skip =3D (eol + 1) - filter_next; - memmove(filter_next, eol + 1, buf + got - eol); - got -=3D skip; - } else { - filter_next =3D eol + 1; - *eol =3D '\n'; - } - } - - if (got > 0 && - buf[got - 1] =3D=3D '\n') { - buf[got - 1] =3D '\0'; - got--; - } - - if (max > 0 && got > max) { - skip =3D got - max; - - if (buf[skip - 1] !=3D '\n' && - (eol =3D strchr(buf + skip, '\n')) && - !virStringIsEmpty(eol + 1)) - skip =3D eol + 1 - buf; - - memmove(buf, buf + skip, got - skip + 1); - got -=3D skip; - } - - buf =3D g_renew(char, buf, got + 1); - *msg =3D buf; - return 0; -} - - static int qemuProcessReportLogError(qemuLogContext *logCtxt, const char *msgprefix) @@ -1961,7 +1896,7 @@ qemuProcessReportLogError(qemuLogContext *logCtxt, g_autofree char *logmsg =3D NULL; =20 /* assume that 1024 chars of qemu log is the right balance */ - if (qemuProcessReadLog(logCtxt, &logmsg, 1024) < 0) + if (qemuLogContextReadFiltered(logCtxt, &logmsg, 1024) < 0) return -1; =20 virResetLastError(); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252250; cv=none; d=zohomail.com; s=zohoarc; b=cW4VePos8oS7GTyn7llS+74mhfvl7HnIVRIY4tE/5sH04789K8wHODclH++CVLqdCcrK3RY1+lYWb7tVPiz3KsC+k5elgO5/aSb/Cmm0Kt07TFE+rN9ei6+BLA+qYCtV2Q6Eh9aBvL7aeKFtzKBjkO2Rikpa7fD86wzZcBqmrP4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252250; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5cBSk54gfYWSGDspoauhdsRoJHPV2mt68SWsFH++0Ak=; b=iBwTdWC5lNDl4MFqG1RZeseOU6sh+mmXAG9KDq3wxlwbbFBnitrhVWlOhiy1K6WzbNjsg30eY/I8wghFcRpzp5e1TKsfJEbmC8CWNpKMMG/VycDm+d7lSnVedZk8nqlHWIH2fqfP8CtRc4rIUUkDTe/IMNeKa0UhAiascEiZHaM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252250291347.9230237286945; Fri, 20 Jan 2023 14:04:10 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-335-sJHJC7evN5ChJtl3P0Ibhg-1; Fri, 20 Jan 2023 17:03:55 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 37DE780B3C8; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F515483EC3; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E9F7C1946588; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 52CA61947063 for ; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 4684B14171C2; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 246BA14171C1 for ; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252249; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=5cBSk54gfYWSGDspoauhdsRoJHPV2mt68SWsFH++0Ak=; b=GHUS+YrARf0bMUPqfTasMWHFvY+NAw6D8EYn/Loug+rEjzeJU9mjPARYKtMIUqNyyPYqxP Sr7WGkUs1PC0zFim55mmr1llE7r+A+YnE3vtVn3gM9IbTfDPuskRvmN+lp1j7y2yuWN0V6 /S68rfB6AtIrGzDvvPSGUjivdaMQT1Y= X-MC-Unique: sJHJC7evN5ChJtl3P0Ibhg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 15/31] qemu: log error output from nbdkit Date: Fri, 20 Jan 2023 16:03:09 -0600 Message-Id: <20230120220325.1015090-16-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252251857100001 Content-Type: text/plain; charset="utf-8"; x-default="true" log stderr and stdout from nbdkit into its own log so that nbdkit-related issues can be debugged more easily. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 34462f784a..7a7b9a6a54 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -848,15 +848,24 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, int rc; int exitstatus =3D 0; int cmdret =3D 0; - VIR_AUTOCLOSE errfd =3D -1; virTimeBackOffVar timebackoff; bool socketCreated =3D false; + g_autofree char *basename =3D g_strdup_printf("%s-nbdkit-%i", vm->def-= >name, proc->source->id); + int logfd =3D -1; + g_autoptr(qemuLogContext) logContext =3D NULL; + g_autofree char *errmsg =3D NULL; + g_autoptr(virURI) uri =3D NULL; + g_autofree char *uristring =3D NULL; =20 if (!(cmd =3D qemuNbdkitProcessBuildCommand(proc))) return -1; =20 + logContext =3D qemuLogContextNew(driver, vm, basename); + logfd =3D qemuLogContextGetWriteFD(logContext); + VIR_DEBUG("starting nbdkit process for %s", proc->source->nodestorage); - virCommandSetErrorFD(cmd, &errfd); + virCommandSetErrorFD(cmd, &logfd); + virCommandSetOutputFD(cmd, &logfd); virCommandSetPidFile(cmd, proc->pidfile); =20 if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) @@ -900,12 +909,15 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, return 0; =20 error: - if (errfd > 0) { - g_autofree char *errbuf =3D g_new0(char, 1024); - if (read(errfd, errbuf, 1024) > 0) - virReportError(VIR_ERR_OPERATION_FAILED, - _("nbdkit failed to start and reported: %s"), e= rrbuf); - } + if (qemuLogContextReadFiltered(logContext, &errmsg, 1024) < 0) + VIR_WARN("Unable to read from nbdkit log"); + + uri =3D qemuBlockStorageSourceGetURI(proc->source); + uristring =3D virURIFormat(uri); + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Failed to connect to nbdkit for '%s': %s"), + uristring, errmsg); + qemuNbdkitProcessStop(proc); return -1; } --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252324; cv=none; d=zohomail.com; s=zohoarc; b=KhxyCS4vftbSVv8a7rHvaLQ4lYc0+ySaeRcyfG1iW2OH2/XstN506003pJGEG5nBP0mMN/EPnrBGZ1Q5l5DCMTNN7SuVvaOR5BtktUyed7guzgTHLm3ijQnMipVP824alCJbf3ZOP+BE0n+PzhpghWERw49coz7D1thCOV2iliU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252324; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Dlzloq3SJXwwX+ZVI0cQymHDATOll/SBEnSKnvLqiVw=; b=S+X5GHCBk4gf0f6ygAoN9uoOO9NNRbayE5vQhfCwnPRAk5iXZm65cfjl2ijJ02byTNoi/jlOQA6AGkzY3hjaf6yy8G9axFXYoIEfi4j8OmS+M6lCGXs5JEXmPn1ZVA0SmJz3CY4X7jAnjGs9lgo31oRj1cgSsrNM243vQb8Virk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 167425232462270.78633775545222; Fri, 20 Jan 2023 14:05:24 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-219-XOICJHR1MWqAErCBc7i6Jg-1; Fri, 20 Jan 2023 17:03:51 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7941C80B3DE; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D4AE202700B; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 336BA1947043; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8EF751947069 for ; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 843C514171C2; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5894D14171C1; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252323; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Dlzloq3SJXwwX+ZVI0cQymHDATOll/SBEnSKnvLqiVw=; b=VVNjrG8BSxFeMZ4kwLJjrrJuYibIAPXf3QgJxRrkh1KSABIFEep1dVS+tVjFHuztws1sUx KjvmeP0laK8TsnRlYROMquMadVhQWGYcMMEly0JSWa3V8NyH1Pk0YKmSssytp67jOQ4l97 augi4YYKS18DOgSkYV1JSy4UK1aAVYg= X-MC-Unique: XOICJHR1MWqAErCBc7i6Jg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 16/31] tests: add ability to test various nbdkit capabilities Date: Fri, 20 Jan 2023 16:03:10 -0600 Message-Id: <20230120220325.1015090-17-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252326246100002 Content-Type: text/plain; charset="utf-8"; x-default="true" Add new DO_TEST_CAPS_LATEST_NBDKIT macro to test xml2argv for various nbdkit capability scenarios. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 20 +++++++++++++++++--- tests/qemuxml2argvtest.c | 11 +++++++++++ tests/testutilsqemu.c | 27 +++++++++++++++++++++++++++ tests/testutilsqemu.h | 5 +++++ 4 files changed, 60 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 7a7b9a6a54..00ca945904 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -290,10 +290,16 @@ virNbkditCapsCheckModdir(const char *moddir, =20 static bool virNbdkitCapsIsValid(void *data, - void *privData G_GNUC_UNUSED) + void *privData) { qemuNbdkitCaps *nbdkitCaps =3D data; struct stat st; + /* when run under test, we will use privData as a signal to indicate t= hat + * we shouldn't touch the filesystem */ + bool skipValidation =3D (privData !=3D NULL); + + if (skipValidation) + return true; =20 if (!nbdkitCaps->path) return true; @@ -334,9 +340,17 @@ virNbdkitCapsIsValid(void *data, =20 static void* virNbdkitCapsNewData(const char *binary, - void *privData G_GNUC_UNUSED) + void *privData) { - qemuNbdkitCaps *caps =3D qemuNbdkitCapsNew(binary); + /* when run under test, we will use privData as a signal to indicate t= hat + * we shouldn't touch the filesystem */ + bool skipNewData =3D (privData !=3D NULL); + qemuNbdkitCaps *caps =3D NULL; + + if (skipNewData) + return NULL; + + caps =3D qemuNbdkitCapsNew(binary); qemuNbdkitCapsQuery(caps); =20 return caps; diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index f3f57c44d0..61267abf4a 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -671,6 +671,14 @@ testCompareXMLToArgv(const void *data) if (rc < 0) goto cleanup; =20 + if (info->nbdkitCaps) { + if (virFileCacheInsertData(driver.nbdkitCapsCache, TEST_NBDKIT_PAT= H, + g_object_ref(info->nbdkitCaps)) < 0) { + g_object_unref(info->nbdkitCaps); + goto cleanup; + } + } + if (info->migrateFrom && !(migrateURI =3D qemuMigrationDstGetURI(info->migrateFrom, info->migrateFd))) @@ -933,6 +941,9 @@ mymain(void) # define DO_TEST_CAPS_ARCH_VER(name, arch, ver) \ DO_TEST_CAPS_ARCH_VER_FULL(name, arch, ver, ARG_END) =20 +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ + DO_TEST_CAPS_ARCH_LATEST_FULL(name, "x86_64", ARG_NBDKIT_CAPS, __VA_AR= GS__, QEMU_NBDKIT_CAPS_LAST, ARG_END) + # define DO_TEST_CAPS_LATEST(name) \ DO_TEST_CAPS_ARCH_LATEST(name, "x86_64") =20 diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index 396803c40b..24fc068058 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -131,6 +131,10 @@ virFindFileInPath(const char *file) return g_strdup_printf("/usr/bin/%s", file); } =20 + if (g_str_equal(file, "nbdkit")) { + return g_strdup(TEST_NBDKIT_PATH); + } + /* Nothing in tests should be relying on real files * in host OS, so we return NULL to try to force * an error in such a case @@ -422,6 +426,7 @@ void qemuTestDriverFree(virQEMUDriver *driver) virObjectUnref(driver->caps); virObjectUnref(driver->config); virObjectUnref(driver->securityManager); + g_clear_object(&driver->nbdkitCapsCache); =20 virCPUDefFree(cpuDefault); virCPUDefFree(cpuHaswell); @@ -665,6 +670,12 @@ int qemuTestDriverInit(virQEMUDriver *driver) if (!driver->qemuCapsCache) goto error; =20 + driver->nbdkitCapsCache =3D qemuNbdkitCapsCacheNew("/dev/null"); + /* the nbdkitCapsCache just interprets the presence of a non-null priv= ate + * data pointer as a signal to skip cache validation. This prevents the + * cache from trying to validate the plugindir mtime, etc during test = */ + virFileCacheSetPriv(driver->nbdkitCapsCache, GUINT_TO_POINTER(1)); + driver->xmlopt =3D virQEMUDriverCreateXMLConf(driver, "none"); if (!driver->xmlopt) goto error; @@ -885,6 +896,7 @@ testQemuInfoSetArgs(struct testQemuInfo *info, =20 info->conf =3D conf; info->args.newargs =3D true; + info->args.fakeNbdkitCaps =3D qemuNbdkitCapsNew(TEST_NBDKIT_PATH); =20 va_start(argptr, conf); while ((argname =3D va_arg(argptr, testQemuInfoArgName)) !=3D ARG_END)= { @@ -896,6 +908,13 @@ testQemuInfoSetArgs(struct testQemuInfo *info, virQEMUCapsSet(info->args.fakeCaps, flag); break; =20 + case ARG_NBDKIT_CAPS: + info->args.fakeNbdkitCapsUsed =3D true; + + while ((flag =3D va_arg(argptr, int)) < QEMU_NBDKIT_CAPS_LAST) + qemuNbdkitCapsSet(info->args.fakeNbdkitCaps, flag); + break; + case ARG_GIC: info->args.gic =3D va_arg(argptr, int); break; @@ -1052,6 +1071,12 @@ testQemuInfoInitArgs(struct testQemuInfo *info) info->qemuCaps =3D g_steal_pointer(&info->args.fakeCaps); } =20 + if (info->args.fakeNbdkitCapsUsed) + info->nbdkitCaps =3D g_steal_pointer(&info->args.fakeNbdkitCaps); + else + /* empty caps */ + info->nbdkitCaps =3D qemuNbdkitCapsNew(TEST_NBDKIT_PATH); + if (info->args.gic !=3D GIC_NONE && testQemuCapsSetGIC(info->qemuCaps, info->args.gic) < 0) return -1; @@ -1070,6 +1095,8 @@ testQemuInfoClear(struct testQemuInfo *info) virObjectUnref(info->qemuCaps); g_clear_pointer(&info->args.fakeCaps, virObjectUnref); g_clear_pointer(&info->args.fds, g_hash_table_unref); + g_clear_object(&info->nbdkitCaps); + g_clear_object(&info->args.fakeNbdkitCaps); } =20 =20 diff --git a/tests/testutilsqemu.h b/tests/testutilsqemu.h index 51c072cb13..8d246ec4ac 100644 --- a/tests/testutilsqemu.h +++ b/tests/testutilsqemu.h @@ -28,6 +28,7 @@ # define TEST_TPM_ENV_VAR "VIR_TEST_MOCK_FAKE_TPM_VERSION" # define TPM_VER_1_2 "1.2" # define TPM_VER_2_0 "2.0" +# define TEST_NBDKIT_PATH "/fakebindir/nbdkit" =20 enum { GIC_NONE =3D 0, @@ -53,6 +54,7 @@ typedef enum { ARG_CAPS_HOST_CPU_MODEL, ARG_HOST_OS, ARG_FD_GROUP, /* name, nfds, fd[0], ... fd[n-1] */ + ARG_NBDKIT_CAPS, ARG_END, } testQemuInfoArgName; =20 @@ -83,6 +85,8 @@ struct testQemuArgs { bool newargs; virQEMUCaps *fakeCaps; bool fakeCapsUsed; + qemuNbdkitCaps *fakeNbdkitCaps; + bool fakeNbdkitCapsUsed; char *capsver; char *capsarch; qemuTestCPUDef capsHostCPUModel; @@ -98,6 +102,7 @@ struct testQemuInfo { char *outfile; char *errfile; virQEMUCaps *qemuCaps; + qemuNbdkitCaps *nbdkitCaps; const char *migrateFrom; int migrateFd; unsigned int flags; --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252311; cv=none; d=zohomail.com; s=zohoarc; b=VO1QmSZVHMfF6c2BE/R+glJI3KNZYC9ZpRxAWmaJ0VbAedyq9JrI+rMP+ashkISB7kz14D99sXh6SvP88IHnpFCjXhiBx5sF2xaGWpcSlercWjqDc7N3sg0+dbvTCmKsvE1ISr6W52GC7O2AuvnOHS0M8KbpMKaim+DEIlC8oiE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252311; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7WG3xnG99YOyYJwpp5+trKnS25eFEKzxFwP7pUzRKi0=; b=XzFLY8UYLEq60AzKBC2JC6VucEd2mCJMGrkgqbSEfm4sTUN2U67GXX4q7cYjORde5nO6YaUMre4i+8AzHrTUAOUhxx0F0dtKuH1nIGVgN80LSdrD/f4yozB0LLa+R6Ev+YaV7gkaxHqBU843qz31ngtsRI+ECJUA3B1PnnuiCtY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252311652204.75151243148434; Fri, 20 Jan 2023 14:05:11 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-590--3wSCosGMIeuA6N2V6MNtA-1; Fri, 20 Jan 2023 17:03:51 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6E2403C11C6C; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 503942026D2C; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3688A1946A7C; Fri, 20 Jan 2023 22:03:33 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id DBAC61946A72 for ; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id C03DD14171C0; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 96171140EBF6; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252310; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=7WG3xnG99YOyYJwpp5+trKnS25eFEKzxFwP7pUzRKi0=; b=XGgKBR15nVxsYN/MUTC6CpSeEQulu0tbxseHbvUILtUw+3l7JF3Km5mzHOki6UEBwDNmD9 GDuEIR8qkJLwghXtT+Mz+VWefCw46t76UaHZvFLhZpMCdjZoa0iNmRhhPX7sJJvtNjHdp7 J5NaV2HkQVSoU7EsBbt8q89GcjurBm4= X-MC-Unique: -3wSCosGMIeuA6N2V6MNtA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 17/31] qemu: split qemuDomainSecretStorageSourcePrepare Date: Fri, 20 Jan 2023 16:03:11 -0600 Message-Id: <20230120220325.1015090-18-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252312151100003 Content-Type: text/plain; charset="utf-8"; x-default="true" This prepares encryption secrets and authentication secrets. When we add nbdkit-backed network storage sources, we will not need to send authentication secrets to qemu, since they will be sent to nbdkit instead. So split this into two different functions. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 83 ++++++++++++++++++++++++++---------------- 1 file changed, 52 insertions(+), 31 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 63c2c602ec..0e3eaf49f8 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1358,24 +1358,19 @@ qemuDomainSecretStorageSourcePrepareCookies(qemuDom= ainObjPrivate *priv, =20 =20 /** - * qemuDomainSecretStorageSourcePrepare: + * qemuDomainSecretStorageSourcePrepareEncryption: * @priv: domain private object * @src: storage source struct to setup - * @authalias: prefix of the alias for secret holding authentication data - * @encalias: prefix of the alias for secret holding encryption password + * @alias: prefix of the alias for secret holding encryption password * - * Prepares data necessary for encryption and authentication of @src. The = two - * alias prefixes are provided since in the backing chain authentication b= elongs - * to the storage protocol data whereas encryption is relevant to the form= at - * driver in qemu. The two will have different node names. + * Prepares data necessary for encryption of @src. * * Returns 0 on success; -1 on error while reporting an libvirt error. */ static int -qemuDomainSecretStorageSourcePrepare(qemuDomainObjPrivate *priv, - virStorageSource *src, - const char *aliasprotocol, - const char *aliasformat) +qemuDomainSecretStorageSourcePrepareEncryption(qemuDomainObjPrivate *priv, + virStorageSource *src, + const char *alias) { qemuDomainStorageSourcePrivate *srcPriv; bool hasEnc =3D src->encryption && src->encryption->nsecrets > 0; @@ -1383,13 +1378,43 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjP= rivate *priv, if (virStorageSourceIsEmpty(src)) return 0; =20 - if (!src->auth && !hasEnc && src->ncookies =3D=3D 0) + if (!hasEnc) return 0; =20 - if (!(src->privateData =3D qemuDomainStorageSourcePrivateNew())) + srcPriv =3D qemuDomainStorageSourcePrivateFetch(src); + + if (!(srcPriv->encinfo =3D qemuDomainSecretInfoSetupFromSecret(priv, a= lias, + "encrypti= on", + VIR_SECRE= T_USAGE_TYPE_VOLUME, + NULL, + &src->enc= ryption->secrets[0]->seclookupdef))) return -1; =20 - srcPriv =3D QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); + return 0; +} + + +/** + * qemuDomainSecretStorageSourcePrepareAuth: + * @priv: domain private object + * @src: storage source struct to setup + * @alias: prefix of the alias for secret holding authentication data + * + * Prepares data necessary for authentication of @src. + * + * Returns 0 on success; -1 on error while reporting an libvirt error. + */ +static int +qemuDomainSecretStorageSourcePrepareAuth(qemuDomainObjPrivate *priv, + virStorageSource *src, + const char *alias) +{ + qemuDomainStorageSourcePrivate *srcPriv; + + if (virStorageSourceIsEmpty(src)) + return 0; + + srcPriv =3D qemuDomainStorageSourcePrivateFetch(src); =20 if (src->auth) { virSecretUsageType usageType =3D VIR_SECRET_USAGE_TYPE_ISCSI; @@ -1397,7 +1422,7 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjPri= vate *priv, if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_RBD) usageType =3D VIR_SECRET_USAGE_TYPE_CEPH; =20 - if (!(srcPriv->secinfo =3D qemuDomainSecretInfoSetupFromSecret(pri= v, aliasprotocol, + if (!(srcPriv->secinfo =3D qemuDomainSecretInfoSetupFromSecret(pri= v, alias, "auth= ", usage= Type, src->= auth->username, @@ -1405,19 +1430,10 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjP= rivate *priv, return -1; } =20 - if (hasEnc) { - if (!(srcPriv->encinfo =3D qemuDomainSecretInfoSetupFromSecret(pri= v, aliasformat, - "encr= yption", - VIR_S= ECRET_USAGE_TYPE_VOLUME, - NULL, - &src-= >encryption->secrets[0]->seclookupdef))) - return -1; - } - if (src->ncookies && !(srcPriv->httpcookie =3D qemuDomainSecretStorageSourcePrepareCook= ies(priv, = src, - = aliasprotocol))) + = alias))) return -1; =20 return 0; @@ -10625,9 +10641,12 @@ qemuDomainPrepareDiskSourceLegacy(virDomainDiskDef= *disk, qemuDomainPrepareStorageSourceConfig(disk->src, cfg); qemuDomainPrepareDiskSourceData(disk, disk->src); =20 - if (qemuDomainSecretStorageSourcePrepare(priv, disk->src, - disk->info.alias, - disk->info.alias) < 0) + if (qemuDomainSecretStorageSourcePrepareEncryption(priv, disk->src, + disk->info.alias) <= 0) + return -1; + + if (qemuDomainSecretStorageSourcePrepareAuth(priv, disk->src, + disk->info.alias) < 0) return -1; =20 if (qemuDomainPrepareStorageSourcePR(disk->src, priv, disk->info.alias= ) < 0) @@ -10718,9 +10737,11 @@ qemuDomainPrepareStorageSourceBlockdevNodename(vir= DomainDiskDef *disk, qemuDomainPrepareStorageSourceConfig(src, cfg); qemuDomainPrepareDiskSourceData(disk, src); =20 - if (qemuDomainSecretStorageSourcePrepare(priv, src, - src->nodestorage, - src->nodeformat) < 0) + if (qemuDomainSecretStorageSourcePrepareEncryption(priv, src, + src->nodeformat) < = 0) + return -1; + if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, + src->nodestorage) < 0) return -1; =20 if (qemuDomainPrepareStorageSourcePR(src, priv, src->nodestorage) < 0) --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252284; cv=none; d=zohomail.com; s=zohoarc; b=CMt4j/mcwkqYldlaQ3h/rLcm69nWDJEMMkYF/vpvVSa9FGdhA3XOm/TQN6YT099+fiQTV9H7fjbJfSJ66H/hpGu1FLTT6ERR44FMoqKT5v8vkJd/R/vGDtjpJRlxBpQZ8Z5Da1V4TQk0SoK2lQ8d8c63AN9AHL/2w+IghHqtCbQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252284; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Gire2gEybJL7fm7wOIJodipCyzwlDP2f9F4j/DpqdVU=; b=iDS9xGuPCPmjZUIGC7fQhrXMs1GdGaWKfGId7+qcqbRzbQcFurnnToAj468XgH3LiXFMnv7mpD4MAWAqI/rmmkLC8Ywn+Pf9vododga1xzNhX2072u/L1xzwEv/xUrvSMSdsYEYpSerPT6s87G16g48e02XUwG9YJKtJPbxuZzQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252283888339.6333698446816; Fri, 20 Jan 2023 14:04:43 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-241-KkAx2G9EOZCLx6kJIwRSrA-1; Fri, 20 Jan 2023 17:04:40 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C7BC281B551; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id B065440AE1EF; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7F3131946A76; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1D02E1946A6D for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 0074914171C8; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D1AEF140EBF6 for ; Fri, 20 Jan 2023 22:03:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252282; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Gire2gEybJL7fm7wOIJodipCyzwlDP2f9F4j/DpqdVU=; b=ZxjYS6aRQ5ORHJQo9nfT+n0RoRrnqUcq96rYPlzIXTEQGu+LmvO4TyErE/U4Zf/CollcyO av+YHDnyt/RgYr9B/UaE+GZDSEJBFBtYlNTyEzP+22GVsM5EYVZqv4jpdSYAw28R0tVPTi Z7HsqO57SfP7SfWGoIqRCTdg42Aa1CY= X-MC-Unique: KkAx2G9EOZCLx6kJIwRSrA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 18/31] qemu: include nbdkit state in private xml Date: Fri, 20 Jan 2023 16:03:12 -0600 Message-Id: <20230120220325.1015090-19-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252286040100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Add xml to the private data for a disk source to represent the nbdkit process so that the state can be re-created if the libvirt daemon is restarted. Format: /path/to/nbdkit.pid /path/to/nbdkit.socket Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_domain.c | 48 ++++++++++++++ src/qemu/qemu_nbdkit.c | 77 +++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 8 +++ src/qemu/qemu_process.c | 11 ++++ tests/qemustatusxml2xmldata/modern-in.xml | 4 ++ 5 files changed, 148 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 0e3eaf49f8..28d4bddf14 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1911,6 +1911,29 @@ qemuStorageSourcePrivateDataAssignSecinfo(qemuDomain= SecretInfo **secinfo, } =20 =20 +static int +qemuStorageSourcePrivateDataParseNbdkit(xmlNodePtr node, + xmlXPathContextPtr ctxt, + virStorageSource *src) +{ + g_autofree char *pidfile =3D NULL; + g_autofree char *socketfile =3D NULL; + VIR_XPATH_NODE_AUTORESTORE(ctxt); + + ctxt->node =3D node; + + if (!(pidfile =3D virXPathString("string(./pidfile)", ctxt))) + return -1; + + if (!(socketfile =3D virXPathString("string(./socketfile)", ctxt))) + return -1; + + qemuNbdkitReconnectStorageSource(src, pidfile, socketfile); + + return 0; +} + + static int qemuStorageSourcePrivateDataParse(xmlXPathContextPtr ctxt, virStorageSource *src) @@ -1921,6 +1944,7 @@ qemuStorageSourcePrivateDataParse(xmlXPathContextPtr = ctxt, g_autofree char *httpcookiealias =3D NULL; g_autofree char *tlskeyalias =3D NULL; g_autofree char *thresholdEventWithIndex =3D NULL; + xmlNodePtr nbdkitnode =3D NULL; =20 src->nodestorage =3D virXPathString("string(./nodenames/nodename[@type= =3D'storage']/@name)", ctxt); src->nodeformat =3D virXPathString("string(./nodenames/nodename[@type= =3D'format']/@name)", ctxt); @@ -1964,6 +1988,10 @@ qemuStorageSourcePrivateDataParse(xmlXPathContextPtr= ctxt, virTristateBoolTypeFromString(thresholdEventWithIndex) =3D=3D VIR_= TRISTATE_BOOL_YES) src->thresholdEventWithIndex =3D true; =20 + if ((nbdkitnode =3D virXPathNode("nbdkit", ctxt))) { + if (qemuStorageSourcePrivateDataParseNbdkit(nbdkitnode, ctxt, src)= < 0) + return -1; + } return 0; } =20 @@ -1981,6 +2009,23 @@ qemuStorageSourcePrivateDataFormatSecinfo(virBuffer = *buf, } =20 =20 +static void +qemuStorageSourcePrivateDataFormatNbdkit(qemuNbdkitProcess *nbdkit, + virBuffer *buf) +{ + g_auto(virBuffer) childBuf =3D VIR_BUFFER_INIT_CHILD(buf); + + if (!nbdkit) + return; + + virBufferEscapeString(&childBuf, "%s\n", + nbdkit->pidfile); + virBufferEscapeString(&childBuf, "%s\n", + nbdkit->socketfile); + virXMLFormatElement(buf, "nbdkit", NULL, &childBuf); +} + + static int qemuStorageSourcePrivateDataFormat(virStorageSource *src, virBuffer *buf) @@ -2019,6 +2064,9 @@ qemuStorageSourcePrivateDataFormat(virStorageSource *= src, if (src->thresholdEventWithIndex) virBufferAddLit(buf, "\n"); =20 + if (srcPriv) + qemuStorageSourcePrivateDataFormatNbdkit(srcPriv->nbdkitProcess, b= uf); + return 0; } =20 diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 00ca945904..2b26e9bc08 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -627,6 +627,83 @@ qemuNbdkitProcessNew(virStorageSource *source, return nbdkit; } =20 +/** + * qemuNbdkitReconnectStorageSource: + * @source: a storage source + * @pidfile: a pidfile for an nbdkit process + * @socketfile: the socket file associated with the nbdkit process + * + * This function constructs a new qemuNbdkitProcess object with the given = values for @pidfile and + * @socketfile and stores it in @source. This is intended to be called whe= n the libvirt daemon is + * restarted and tries to reconnect to all currently-running domains. Sinc= e this function is called + * from the code that parses the current daemon state, it should not perfo= rm any filesystem + * operations, or anything else that might fail. Additional initialization= will be done later by + * calling qemuNbdkitStorageSourceManageProcess(). + */ +void +qemuNbdkitReconnectStorageSource(virStorageSource *source, + const char *pidfile, + const char *socketfile) +{ + qemuDomainStorageSourcePrivate *srcpriv =3D qemuDomainStorageSourcePri= vateFetch(source); + + if (srcpriv->nbdkitProcess) { + VIR_WARN("source already has an nbdkit process"); + return; + } + + srcpriv->nbdkitProcess =3D qemuNbdkitProcessNew(source, pidfile, socke= tfile); +} + + +static int +qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source) +{ + qemuDomainStorageSourcePrivate *srcpriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(source); + qemuNbdkitProcess *proc; + + if (!srcpriv) + return 0; + + proc =3D srcpriv->nbdkitProcess; + + if (proc) { + if (proc->pid <=3D 0) { + if (virPidFileReadPath(proc->pidfile, &proc->pid) < 0) + return -1; + } + + if (virProcessKill(proc->pid, 0) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("nbdkit process %i is not alive"), proc->pid); + return -1; + } + } + + return 0; +} + +/** + * qemuNbdkitStorageSourceManageProcess: + * @source: a storage source + * @vm: the vm that owns this storage source + * + * This function re-enables monitoring of any nbdkit processes associated = with the backing chain of + * @source. It is intended to be called after libvirt restarts and has loa= ded its current state from + * disk and is attempting to re-connect to active domains. + */ +int +qemuNbdkitStorageSourceManageProcess(virStorageSource *source) +{ + virStorageSource *backing; + for (backing =3D source->backingStore; backing !=3D NULL; backing =3D = backing->backingStore) { + if (qemuNbdkitStorageSourceManageProcessOne(backing) < 0) + return -1; + } + + return qemuNbdkitStorageSourceManageProcessOne(source); +} + =20 bool qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index ccd418b7d3..2be46b6002 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -54,6 +54,14 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *nbdkitCaps, uid_t user, gid_t group); =20 +void +qemuNbdkitReconnectStorageSource(virStorageSource *source, + const char *pidfile, + const char *socketfile); + +int +qemuNbdkitStorageSourceManageProcess(virStorageSource *src); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 6d1751b5d7..7ec31ef6ac 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -9047,6 +9047,17 @@ qemuProcessReconnect(void *opaque) } } =20 + for (i =3D 0; i < obj->def->ndisks; i++) { + virDomainDiskDef *disk =3D obj->def->disks[i]; + if (qemuNbdkitStorageSourceManageProcess(disk->src) < 0) + goto error; + } + + if (obj->def->os.loader && obj->def->os.loader->nvram) { + if (qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvra= m) < 0) + goto error; + } + /* update domain state XML with possibly updated state in virDomainObj= */ if (virDomainObjSave(obj, driver->xmlopt, cfg->stateDir) < 0) goto error; diff --git a/tests/qemustatusxml2xmldata/modern-in.xml b/tests/qemustatusxm= l2xmldata/modern-in.xml index 7759034f7a..71b3eb4736 100644 --- a/tests/qemustatusxml2xmldata/modern-in.xml +++ b/tests/qemustatusxml2xmldata/modern-in.xml @@ -342,6 +342,10 @@ + + /path/to/nbdkit.pid + /path/to/nbdkit.socket + --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252348; cv=none; d=zohomail.com; s=zohoarc; b=ZDhAdwpUIrm6EsCm8xMcQkZNUxiKeZVWpm+InaZ9FkyRUQq/RJrpHoMlBqti3Sf7EVSbn6eCWpHxehswLB84WoLW8/mVzdoYFXGUbjHzOMRpMy4VXfYD4Yet8HEllYh3zrZ2GDfW0l7A/Wt7ZI9MSC68x/myTF7VuAvZCuQ7Icw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252348; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3ldjDjgo3sUgxdEoto5V4vQzfmcqNb+kfEXGjoPKILY=; b=b9DHXyVcqorIDjix4ATT9TQ6qsFkCTlWeLZM2ygxbRy2r9R9fRZFRWY6fr9P2hmlGHhgUXEMvjc6oEQ0bVGp2l8rZ/MTqCue/uGUc61/k1k/deuDYSlWF5bYQpqU6GbBvWGM9kzLL86fQN5BGXeZh0gUhuyS0hs0VJY/yB+GqE0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252348489546.802183317497; Fri, 20 Jan 2023 14:05:48 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-599-IJ12LBdoPzehdzO3dtDJug-1; Fri, 20 Jan 2023 17:03:52 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C9E381816ED0; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id B345840C2064; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 90EF01947043; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 643891946588 for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 37E7E14171C7; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1108314171CA for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252347; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3ldjDjgo3sUgxdEoto5V4vQzfmcqNb+kfEXGjoPKILY=; b=f4WzjMTcr12uCKjU+A0+sfTMoCACC5zo3XU99mNAU8mdJYuSRVUduycuMz3qj/5+pIx+GP u/Z87wHF+AZ2K7zDgHTNldAxfB4KmBETYeemJrUbLwWCg3ajD3qnsALNB+8wsH3ApBB/JY tM59r/toz9GZfPIKVdmB4nDG3y2264Q= X-MC-Unique: IJ12LBdoPzehdzO3dtDJug-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 19/31] qemu: pass sensitive data to nbdkit via pipe Date: Fri, 20 Jan 2023 16:03:13 -0600 Message-Id: <20230120220325.1015090-20-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252349819100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Rather than passing passwords and cookies (which could contain passwords) to nbdkit via commandline arguments, use the alternate format that nbdkit supports where we can specify a file descriptor which nbdkit will read to get the password or cookies. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 55 ++++++++++++++++++++++++++++++------------ 1 file changed, 40 insertions(+), 15 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 2b26e9bc08..ba84958e8d 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -67,6 +67,12 @@ struct _qemuNbdkitCaps { G_DEFINE_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, G_TYPE_OBJECT); =20 =20 +enum { + PIPE_FD_READ =3D 0, + PIPE_FD_WRITE =3D 1 +}; + + static void qemuNbdkitCheckCommandCap(qemuNbdkitCaps *nbdkit, virCommand *cmd, @@ -759,6 +765,29 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } =20 =20 +static int +qemuNbdkitCommandPassDataByPipe(virCommand *cmd, + const char *argName, + unsigned char **buf, + size_t buflen) +{ + g_autofree char *fdfmt =3D NULL; + int fd =3D virCommandSetSendBuffer(cmd, buf, buflen); + + if (fd < 0) + return -1; + + /* some nbdkit arguments accept a variation where nbdkit will read the= data + * from a file descriptor, e.g. password=3D-FD */ + fdfmt =3D g_strdup_printf("-%i", fd); + virCommandAddArgPair(cmd, argName, fdfmt); + + virCommandDoAsyncIO(cmd); + + return 0; +} + + static int qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, virCommand *cmd) @@ -784,7 +813,6 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *pr= oc, g_autoptr(virConnect) conn =3D virGetConnectSecret(); g_autofree uint8_t *secret =3D NULL; size_t secretlen =3D 0; - g_autofree char *password =3D NULL; int secrettype; virStorageAuthDef *authdef =3D proc->source->auth; =20 @@ -808,22 +836,19 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *= proc, return -1; } =20 - /* ensure that the secret is a NULL-terminated string */ - password =3D g_strndup((char*)secret, secretlen); - - /* for now, just report an error rather than passing the password = in - * cleartext on the commandline */ - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Password not yet supported for nbdkit sources")); - return -1; + if (qemuNbdkitCommandPassDataByPipe(cmd, "password", + &secret, secretlen) < 0) + return -1; } =20 - if (proc->source->ncookies > 0) { - /* for now, just report an error rather than passing cookies in - * cleartext on the commandline */ - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Cookies not yet supported for nbdkit sources")); - return -1; + /* Create a pipe to send the cookies to the nbdkit process. */ + if (proc->source->ncookies) { + g_autofree char *cookies =3D qemuBlockStorageSourceGetCookieString= (proc->source); + + if (qemuNbdkitCommandPassDataByPipe(cmd, "cookie", + (unsigned char**)&cookies, + strlen(cookies)) < 0) + return -1; } =20 if (proc->source->sslverify =3D=3D VIR_TRISTATE_BOOL_NO) { --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252344; cv=none; d=zohomail.com; s=zohoarc; b=G3+rBv3iLcGH+MPkftlQ82Lt8/kH9I1/LXr/EtiTR3lLPeDplrJo/WLJ3+jloPM9g7CCkp4hF9MLd1ENXarWXT6dkWEu3M0mD8JstHJO+R2iPuhsRtc1MkyalP2z27L94+LtqlBhtGIoM6yet6tQWsngpha+pOWuoYpIQ51+lv0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252344; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mRSgglfCciKAqrBz+UwfI82OVl3fm6xnjiTheaQNyAI=; b=D1Z1kdx8v8L7VHp+a9VAHwWXO0qoqSKgUGFPUOrqImbtJ7/1jY9Faw4lm9aFtvyBedjn+4mAMMxheI9+GD9vM2EQC6Qibyw8r8fzQTZmlXp6IiN2faG6qCl4U4n3jAo7NnXGp23eBG2zHfeWJzCO8wLwEs1g8z16XfkwY9YMnV8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252344616852.1044509338006; Fri, 20 Jan 2023 14:05:44 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-153-JR5_q-8dMtakUShvi0ixNA-1; Fri, 20 Jan 2023 17:03:59 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E767E38123D0; Fri, 20 Jan 2023 22:03:51 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD5122026D2B; Fri, 20 Jan 2023 22:03:51 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id EBB0C19465B6; Fri, 20 Jan 2023 22:03:44 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8C0221946588 for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 70790140EBF6; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 461EC14171C8 for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252343; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=mRSgglfCciKAqrBz+UwfI82OVl3fm6xnjiTheaQNyAI=; b=ixtZ+mPJt8yMmmxPVKJk5B2ADIEZ28f2x83YrWe8X7wM9+ashYQVR8ZJiEU0Oapv5hHlt1 9mdDtvnzaoOaDZrgO/XYqeCnR7E/lg0o8L9a8I4tQraPUMMSxdgDu7iHaRbkwNzJPPTzRj vVq8SAnSYIvV82kBvVPg049xQHGtWoM= X-MC-Unique: JR5_q-8dMtakUShvi0ixNA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 20/31] qemu: use nbdkit to serve network disks if available Date: Fri, 20 Jan 2023 16:03:14 -0600 Message-Id: <20230120220325.1015090-21-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252345955100001 Content-Type: text/plain; charset="utf-8"; x-default="true" For virStorageSource objects that contain an nbdkitProcess, start that nbdkit process to serve that network drive and then pass the nbdkit socket to qemu rather than sending the network url to qemu directly. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_block.c | 162 +++++++++++------- src/qemu/qemu_domain.c | 13 +- src/qemu/qemu_extdevice.c | 56 ++++++ src/qemu/qemu_hotplug.c | 7 + src/qemu/qemu_nbdkit.c | 63 +++++++ src/qemu/qemu_nbdkit.h | 13 ++ ...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 +++++ .../disk-cdrom-network-nbdkit.xml | 1 + ...isk-network-http-nbdkit.x86_64-latest.args | 45 +++++ .../disk-network-http-nbdkit.xml | 1 + ...rce-curl-nbdkit-backing.x86_64-latest.args | 38 ++++ ...isk-network-source-curl-nbdkit-backing.xml | 45 +++++ ...work-source-curl-nbdkit.x86_64-latest.args | 50 ++++++ .../disk-network-source-curl-nbdkit.xml | 1 + ...isk-network-source-curl.x86_64-latest.args | 53 ++++++ .../disk-network-source-curl.xml | 71 ++++++++ ...disk-network-ssh-nbdkit.x86_64-latest.args | 36 ++++ .../disk-network-ssh-nbdkit.xml | 1 + tests/qemuxml2argvtest.c | 6 + 19 files changed, 638 insertions(+), 66 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64= -latest.args create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-= latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-= backing.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-= backing.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.= x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.= xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-= latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-l= atest.args create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index e865aa17f9..7b5ac20a3c 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -439,6 +439,32 @@ qemuBlockStorageSourceGetCURLProps(virStorageSource *s= rc, } =20 =20 +static virJSONValue * +qemuBlockStorageSourceGetNbdkitProps(virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *srcPriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(src); + virJSONValue *ret =3D NULL; + g_autoptr(virJSONValue) serverprops =3D NULL; + virStorageNetHostDef host =3D { .transport =3D VIR_STORAGE_NET_HOST_TR= ANS_UNIX }; + + /* srcPriv->nbdkitProcess will already be initialized if we can use nb= dkit + * to proxy this storage source */ + if (!(srcPriv && srcPriv->nbdkitProcess)) + return NULL; + + host.socket =3D srcPriv->nbdkitProcess->socketfile; + serverprops =3D qemuBlockStorageSourceBuildJSONSocketAddress(&host); + + if (!serverprops) + return NULL; + + if (virJSONValueObjectAdd(&ret, "a:server", &serverprops, NULL) < 0) + return NULL; + + return ret; +} + + static virJSONValue * qemuBlockStorageSourceGetISCSIProps(virStorageSource *src, bool onlytarget) @@ -876,69 +902,75 @@ qemuBlockStorageSourceGetBackendProps(virStorageSourc= e *src, return NULL; =20 case VIR_STORAGE_TYPE_NETWORK: - switch ((virStorageNetProtocol) src->protocol) { - case VIR_STORAGE_NET_PROTOCOL_GLUSTER: - driver =3D "gluster"; - if (!(fileprops =3D qemuBlockStorageSourceGetGlusterProps(src,= onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_VXHS: - driver =3D "vxhs"; - if (!(fileprops =3D qemuBlockStorageSourceGetVxHSProps(src, on= lytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_HTTP: - case VIR_STORAGE_NET_PROTOCOL_HTTPS: - case VIR_STORAGE_NET_PROTOCOL_FTP: - case VIR_STORAGE_NET_PROTOCOL_FTPS: - case VIR_STORAGE_NET_PROTOCOL_TFTP: - driver =3D virStorageNetProtocolTypeToString(src->protocol); - if (!(fileprops =3D qemuBlockStorageSourceGetCURLProps(src, on= lytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_ISCSI: - driver =3D "iscsi"; - if (!(fileprops =3D qemuBlockStorageSourceGetISCSIProps(src, o= nlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_NBD: + /* prefer using nbdkit for sources that are supported */ + if ((fileprops =3D qemuBlockStorageSourceGetNbdkitProps(src))) { driver =3D "nbd"; - if (!(fileprops =3D qemuBlockStorageSourceGetNBDProps(src, onl= ytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_RBD: - driver =3D "rbd"; - if (!(fileprops =3D qemuBlockStorageSourceGetRBDProps(src, onl= ytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: - driver =3D "sheepdog"; - if (!(fileprops =3D qemuBlockStorageSourceGetSheepdogProps(src= ))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_SSH: - driver =3D "ssh"; - if (!(fileprops =3D qemuBlockStorageSourceGetSshProps(src))) - return NULL; break; - - case VIR_STORAGE_NET_PROTOCOL_NFS: - driver =3D "nfs"; - if (!(fileprops =3D qemuBlockStorageSourceGetNFSProps(src))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_NONE: - case VIR_STORAGE_NET_PROTOCOL_LAST: - virReportEnumRangeError(virStorageNetProtocol, src->protocol); - return NULL; + } else { + switch ((virStorageNetProtocol) src->protocol) { + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + driver =3D "gluster"; + if (!(fileprops =3D qemuBlockStorageSourceGetGlusterPr= ops(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_VXHS: + driver =3D "vxhs"; + if (!(fileprops =3D qemuBlockStorageSourceGetVxHSProps= (src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + driver =3D virStorageNetProtocolTypeToString(src->prot= ocol); + if (!(fileprops =3D qemuBlockStorageSourceGetCURLProps= (src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + driver =3D "iscsi"; + if (!(fileprops =3D qemuBlockStorageSourceGetISCSIProp= s(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NBD: + driver =3D "nbd"; + if (!(fileprops =3D qemuBlockStorageSourceGetNBDProps(= src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_RBD: + driver =3D "rbd"; + if (!(fileprops =3D qemuBlockStorageSourceGetRBDProps(= src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + driver =3D "sheepdog"; + if (!(fileprops =3D qemuBlockStorageSourceGetSheepdogP= rops(src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_SSH: + driver =3D "ssh"; + if (!(fileprops =3D qemuBlockStorageSourceGetSshProps(= src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NFS: + driver =3D "nfs"; + if (!(fileprops =3D qemuBlockStorageSourceGetNFSProps(= src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportEnumRangeError(virStorageNetProtocol, src->pr= otocol); + return NULL; + } } break; } @@ -2221,6 +2253,7 @@ qemuBlockStorageSourceCreateGetStorageProps(virStorag= eSource *src, g_autoptr(virJSONValue) location =3D NULL; const char *driver =3D NULL; const char *filename =3D NULL; + qemuDomainStorageSourcePrivate *srcPriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(src); =20 switch (actualType) { case VIR_STORAGE_TYPE_FILE: @@ -2249,6 +2282,13 @@ qemuBlockStorageSourceCreateGetStorageProps(virStora= geSource *src, break; =20 case VIR_STORAGE_NET_PROTOCOL_SSH: + if (srcPriv->nbdkitProcess) { + /* disk creation not yet supported with nbdkit, and even i= f it + * was supported, it would not be done with blockdev-create + * props */ + return 0; + } + driver =3D "ssh"; if (!(location =3D qemuBlockStorageSourceGetSshProps(src))) return -1; diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 28d4bddf14..3938f733d3 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -10788,9 +10788,14 @@ qemuDomainPrepareStorageSourceBlockdevNodename(vir= DomainDiskDef *disk, if (qemuDomainSecretStorageSourcePrepareEncryption(priv, src, src->nodeformat) < = 0) return -1; - if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, - src->nodestorage) < 0) - return -1; + + if (!qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, = priv)) { + /* If we're using nbdkit to serve the storage source, we don't pass + * authentication secrets to qemu, but will pass them to nbdkit in= stead */ + if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, + src->nodestorage) < 0) + return -1; + } =20 if (qemuDomainPrepareStorageSourcePR(src, priv, src->nodestorage) < 0) return -1; @@ -10805,8 +10810,6 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virD= omainDiskDef *disk, if (qemuDomainPrepareStorageSourceFDs(src, priv) < 0) return -1; =20 - qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, priv); - return 0; } =20 diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c index f7b2e2e653..444cdb9d61 100644 --- a/src/qemu/qemu_extdevice.c +++ b/src/qemu/qemu_extdevice.c @@ -229,6 +229,17 @@ qemuExtDevicesStart(virQEMUDriver *driver, return -1; } =20 + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + if (qemuNbdkitStartStorageSource(driver, vm, disk->src) < 0) + return -1; + } + + if (def->os.loader && def->os.loader->nvram) { + if (qemuNbdkitStartStorageSource(driver, vm, def->os.loader->nvram= ) < 0) + return -1; + } + return 0; } =20 @@ -280,6 +291,14 @@ qemuExtDevicesStop(virQEMUDriver *driver, fs->fsdriver =3D=3D VIR_DOMAIN_FS_DRIVER_TYPE_VIRTIOFS) qemuVirtioFSStop(driver, vm, fs); } + + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + qemuNbdkitStopStorageSource(disk->src); + } + + if (def->os.loader && def->os.loader->nvram) + qemuNbdkitStopStorageSource(def->os.loader->nvram); } =20 =20 @@ -305,10 +324,36 @@ qemuExtDevicesHasDevice(virDomainDef *def) return true; } =20 + for (i =3D 0; i < def->ndisks; i++) { + qemuDomainStorageSourcePrivate *priv =3D + QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(def->disks[i]->src); + if (priv->nbdkitProcess) + return true; + } + + return false; } =20 =20 +/* recursively setup nbdkit cgroups for backing chain of src */ +static int qemuExtDevicesSetupCgroupNbdkit(virStorageSource *src, + virCgroup *cgroup) +{ + qemuDomainStorageSourcePrivate *priv =3D QEMU_DOMAIN_STORAGE_SOURC= E_PRIVATE(src); + + if (src->backingStore) + if (qemuExtDevicesSetupCgroupNbdkit(src->backingStore, cgroup)= < 0) + return -1; + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessSetupCgroup(priv->nbdkitProcess, cgroup) < 0) + return -1; + + return 0; +} + + int qemuExtDevicesSetupCgroup(virQEMUDriver *driver, virDomainObj *vm, @@ -348,6 +393,17 @@ qemuExtDevicesSetupCgroup(virQEMUDriver *driver, return -1; } =20 + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + if (qemuExtDevicesSetupCgroupNbdkit(disk->src, cgroup) < 0) + return -1; + } + + if (def->os.loader && def->os.loader->nvram) { + if (qemuExtDevicesSetupCgroupNbdkit(def->os.loader->nvram, cgroup)= < 0) + return -1; + } + for (i =3D 0; i < def->nfss; i++) { virDomainFSDef *fs =3D def->fss[i]; =20 diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 026e1ee5ad..7ec16e702d 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -993,6 +993,9 @@ qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriver *d= river, if (qemuHotplugAttachManagedPR(vm, disk->src, VIR_ASYNC_JOB_NONE) < 0) goto cleanup; =20 + if (qemuNbdkitStartStorageSource(driver, vm, disk->src) < 0) + goto cleanup; + ret =3D qemuDomainAttachDiskGeneric(vm, disk, VIR_ASYNC_JOB_NONE); =20 virDomainAuditDisk(vm, NULL, disk->src, "attach", ret =3D=3D 0); @@ -1015,6 +1018,8 @@ qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriver = *driver, =20 if (virStorageSourceChainHasManagedPR(disk->src)) ignore_value(qemuHotplugRemoveManagedPR(vm, VIR_ASYNC_JOB_NONE= )); + + qemuNbdkitStopStorageSource(disk->src); } qemuDomainSecretDiskDestroy(disk); qemuDomainCleanupStorageSourceFD(disk->src); @@ -4338,6 +4343,8 @@ qemuDomainRemoveDiskDevice(virQEMUDriver *driver, qemuHotplugRemoveManagedPR(vm, VIR_ASYNC_JOB_NONE) < 0) goto cleanup; =20 + qemuNbdkitStopStorageSource(disk->src); + if (disk->transient) { VIR_DEBUG("Removing transient overlay '%s' of disk '%s'", disk->src->path, disk->dst); diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index ba84958e8d..85501b8373 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -765,6 +765,61 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } =20 =20 +static int +qemuNbdkitStartStorageSourceOne(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *priv =3D QEMU_DOMAIN_STORAGE_SOURCE_PR= IVATE(src); + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessStart(priv->nbdkitProcess, vm, driver) < 0) + return -1; + + return 0; +} + + +/* recursively start nbdkit for backing chain of src */ +int +qemuNbdkitStartStorageSource(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src) +{ + virStorageSource *backing; + + for (backing =3D src->backingStore; backing !=3D NULL; backing =3D bac= king->backingStore) + if (qemuNbdkitStartStorageSourceOne(driver, vm, backing) < 0) + return -1; + + return qemuNbdkitStartStorageSourceOne(driver, vm, src); +} + + +static void +qemuNbdkitStopStorageSourceOne(virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *priv =3D QEMU_DOMAIN_STORAGE_SOURCE_PR= IVATE(src); + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessStop(priv->nbdkitProcess) < 0) + VIR_WARN("Unable to stop nbdkit for storage source '%s'", src->nod= estorage); +} + + +/* recursively stop nbdkit processes for backing chain of src */ +void +qemuNbdkitStopStorageSource(virStorageSource *src) +{ + virStorageSource *backing; + + qemuNbdkitStopStorageSourceOne(src); + + for (backing =3D src->backingStore; backing !=3D NULL; backing =3D bac= king->backingStore) + qemuNbdkitStopStorageSourceOne(backing); +} + + static int qemuNbdkitCommandPassDataByPipe(virCommand *cmd, const char *argName, @@ -955,6 +1010,14 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) } =20 =20 +int +qemuNbdkitProcessSetupCgroup(qemuNbdkitProcess *proc, + virCgroup *cgroup) +{ + return virCgroupAddProcess(cgroup, proc->pid); +} + + int qemuNbdkitProcessStart(qemuNbdkitProcess *proc, virDomainObj *vm, diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 2be46b6002..8191ace522 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -21,6 +21,7 @@ =20 #include "internal.h" #include "storage_source_conf.h" +#include "vircgroup.h" #include "virenum.h" #include "virfilecache.h" =20 @@ -59,6 +60,14 @@ qemuNbdkitReconnectStorageSource(virStorageSource *sourc= e, const char *pidfile, const char *socketfile); =20 +int +qemuNbdkitStartStorageSource(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src); + +void +qemuNbdkitStopStorageSource(virStorageSource *src); + int qemuNbdkitStorageSourceManageProcess(virStorageSource *src); =20 @@ -95,4 +104,8 @@ qemuNbdkitProcessStop(qemuNbdkitProcess *proc); void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); =20 +int +qemuNbdkitProcessSetupCgroup(qemuNbdkitProcess *proc, + virCgroup *cgroup); + G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuNbdkitProcess, qemuNbdkitProcessFree); diff --git a/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest= .args b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..eec7ef2af7 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args @@ -0,0 +1,42 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 1024 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":1073741824}= ' \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw"= ,"file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-3-form= at","id":"ide0-0-0","bootindex":1}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw"= ,"file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-2-form= at","id":"ide0-0-1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw"= ,"file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.1","unit":0,"drive":"libvirt-1-form= at","id":"ide0-1-0"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","add= r":"0x2"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml b/tests/q= emuxml2argvdata/disk-cdrom-network-nbdkit.xml new file mode 120000 index 0000000000..55f677546f --- /dev/null +++ b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml @@ -0,0 +1 @@ +disk-cdrom-network.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.= args b/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..25d476d3ce --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args @@ -0,0 +1,45 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-4-storage.socket"},"node-name":"libvirt-4-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"raw= ","file":"libvirt-4-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-4-format","id":"virtio-disk0","bootindex":1}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":false,"driver":"raw= ","file":"libvirt-3-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-3-format","id":"virtio-disk1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"raw= ","file":"libvirt-2-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-2-format","id":"virtio-disk2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x5","drive":"li= bvirt-1-format","id":"virtio-disk3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-http-nbdkit.xml b/tests/qe= muxml2argvdata/disk-network-http-nbdkit.xml new file mode 120000 index 0000000000..6a05204e8a --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-http-nbdkit.xml @@ -0,0 +1 @@ +disk-network-http.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing= .x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-source-curl-nbdki= t-backing.x86_64-latest.args new file mode 100644 index 0000000000..98cfcd219a --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64= -latest.args @@ -0,0 +1,38 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"qcow= 2","file":"libvirt-2-storage"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"qcow= 2","file":"libvirt-1-storage","backing":"libvirt-2-format"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing= .xml b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml new file mode 100644 index 0000000000..37a30fcbd6 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml @@ -0,0 +1,45 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-x86_64 + + + + + + cookievalue1 + cookievalue2 + + + + + + + + cookievalue3 + cookievalue4 + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-= latest.args b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64= -latest.args new file mode 100644 index 0000000000..ec193bb10a --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.= args @@ -0,0 +1,50 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-device '{"driver":"ahci","id":"sata0","bus":"pci.0","addr":"0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-5-storage.socket"},"node-name":"libvirt-5-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw"= ,"file":"libvirt-5-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","= data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","k= eyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-4-storage.socket"},"node-name":"libvirt-4-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luk= s","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-sto= rage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-4-format","id":"virtio-disk4"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw"= ,"file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.1","drive":"libvirt-3-format","id= ":"sata0-0-1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw"= ,"file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.2","drive":"libvirt-2-format","id= ":"sata0-0-2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw"= ,"file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.3","drive":"libvirt-1-format","id= ":"sata0-0-3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml b/t= ests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml new file mode 120000 index 0000000000..4a1e40bd70 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml @@ -0,0 +1 @@ +disk-network-source-curl.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.= args b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args new file mode 100644 index 0000000000..ec6dd13f6c --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args @@ -0,0 +1,53 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-device '{"driver":"ahci","id":"sata0","bus":"pci.0","addr":"0x2"}' \ +-object '{"qom-type":"secret","id":"libvirt-5-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","= keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}'= \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk1.iso","cookie-secret":"libvirt-5-storage-httpcookie-secret0","node-na= me":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw"= ,"file":"libvirt-5-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","= data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","k= eyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ +-object '{"qom-type":"secret","id":"libvirt-4-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","= keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}'= \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk5.iso?foo=3Dbar","sslverify":false,"cookie-secret":"libvirt-4-storage-= httpcookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"= discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luk= s","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-sto= rage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-4-format","id":"virtio-disk4"}' \ +-object '{"qom-type":"secret","id":"libvirt-3-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBBv7TuTgTkyAyOPpC2P5qLbOIy= pLoHpppjz+u5O+X8oT+jA1m7q/OJQ8dk2EFD5c0A=3D","keyid":"masterKey0","iv":"AAE= CAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ +-blockdev '{"driver":"http","url":"http://http.example.org:8080/path/to/di= sk2.iso","cookie-secret":"libvirt-3-storage-httpcookie-secret0","node-name"= :"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw"= ,"file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.1","drive":"libvirt-3-format","id= ":"sata0-0-1"}' \ +-blockdev '{"driver":"ftp","url":"ftp://ftp.example.org:20/path/to/disk3.i= so","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"= }' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw"= ,"file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.2","drive":"libvirt-2-format","id= ":"sata0-0-2"}' \ +-blockdev '{"driver":"ftps","url":"ftps://ftps.example.org:22/path/to/disk= 4.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unm= ap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw"= ,"file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.3","drive":"libvirt-1-format","id= ":"sata0-0-3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.xml b/tests/qe= muxml2argvdata/disk-network-source-curl.xml new file mode 100644 index 0000000000..1e50314abe --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl.xml @@ -0,0 +1,71 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-x86_64 + + + + + cookievalue1 + cookievalue2 + + + + + + + + + + cookievalue1 + cookievalue2 + cookievalue3 + + + + + + + + + + + + + + + + + + + + + + cookievalue1 + cookievalue2 + + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.a= rgs b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..e22ba095b1 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args @@ -0,0 +1,36 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml b/tests/qem= uxml2argvdata/disk-network-ssh-nbdkit.xml new file mode 120000 index 0000000000..b0589bdfb5 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml @@ -0,0 +1 @@ +disk-network-ssh.xml \ No newline at end of file diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 61267abf4a..59f416ef72 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1262,6 +1262,7 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-cdrom-empty-network-invalid"); DO_TEST_CAPS_LATEST("disk-cdrom-bus-other"); DO_TEST_CAPS_LATEST("disk-cdrom-network"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-cdrom-network-nbdkit", QEMU_NBDKIT_CA= PS_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-cdrom-tray"); DO_TEST_CAPS_LATEST("disk-floppy"); DO_TEST_CAPS_LATEST("disk-floppy-q35"); @@ -1302,6 +1303,9 @@ mymain(void) /* qemu-6.0 is the last qemu version supporting sheepdog */ DO_TEST_CAPS_VER("disk-network-sheepdog", "6.0.0"); DO_TEST_CAPS_LATEST("disk-network-source-auth"); + DO_TEST_CAPS_LATEST("disk-network-source-curl"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-source-curl-nbdkit", QEMU_NBD= KIT_CAPS_PLUGIN_CURL); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-source-curl-nbdkit-backing", = QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-network-nfs"); driver.config->vxhsTLS =3D 1; driver.config->nbdTLSx509secretUUID =3D g_strdup("6fd3f62d-9fe7-4a4e-a= 869-7acd6376d8ea"); @@ -1312,7 +1316,9 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-network-tlsx509-nbd-hostname"); DO_TEST_CAPS_VER("disk-network-tlsx509-vxhs", "5.0.0"); DO_TEST_CAPS_LATEST("disk-network-http"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-http-nbdkit", QEMU_NBDKIT_CAP= S_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-network-ssh"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-ssh-nbdkit", QEMU_NBDKIT_CAPS= _PLUGIN_SSH); driver.config->vxhsTLS =3D 0; VIR_FREE(driver.config->vxhsTLSx509certdir); DO_TEST_CAPS_LATEST("disk-no-boot"); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252318; cv=none; d=zohomail.com; s=zohoarc; b=C46W8/2k9F8Iwm/ojwv1r43QZjtKjdAvmW1OB8ArYLnlDzhZZ+yKcz7V8s6QN0C+pJZ3cWTGpNT98718lb3RAQSzQ6EzSlR+1CGgF0NS5e7F9P7329uBHaAYTOvGMXYdGPLSCOzHXAfov71BNKyKxd9108jvg6p67TQm+wWffeA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252318; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OboLTqXKZoat/aWlqwDRbPt2HJVRMYTvlUTN0JLR/w4=; b=R6WgcYAm77PqumzCoqrNA0QIgw6fdgkQZr9SF7SCugCTlqP2tGhJE/+Dl3f9t8jezoin/1GOXoxTmzhRy7f0f8vNXJ5hCOqqel69hcqSX3/FdhMfqGlKmYDetop/txb6vzOTlxFy7AcVJ9NryfKzyTlpXAr9ULWp/lptYAbYUP0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252318603343.2094724805212; Fri, 20 Jan 2023 14:05:18 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-619-HmCU1_4SPzOEGJtFxlnHDQ-1; Fri, 20 Jan 2023 17:03:56 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7F7C0299E753; Fri, 20 Jan 2023 22:03:40 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 68F202026D68; Fri, 20 Jan 2023 22:03:40 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 58D6F1947040; Fri, 20 Jan 2023 22:03:40 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BFC9A1946588 for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B4357140EBF6; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8A15914171C1; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252314; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=OboLTqXKZoat/aWlqwDRbPt2HJVRMYTvlUTN0JLR/w4=; b=El+q8x3roWHFScxGi1Hxcw/Nnx3rdN1UVJwRgSfCKH/w1QY+xkvw2Ln8O2NNUs0tjNaMaw DZdFjvfAj/8Dd29RqcUIfZARqSV7Eqkb5NWrmizrDT+VktFHZUwEE5curHQPJFCMF8LRxP F22uW0CLA3G2/xyZv/YMpHmd4c9n6s8= X-MC-Unique: HmCU1_4SPzOEGJtFxlnHDQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 21/31] util: make virCommandSetSendBuffer testable Date: Fri, 20 Jan 2023 16:03:15 -0600 Message-Id: <20230120220325.1015090-22-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252320159100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Add a private function to peek at the list of send buffers in virCommand so that it is testable Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/libvirt_private.syms | 1 + src/util/vircommand.c | 17 +++++++++-------- src/util/vircommand.h | 8 ++++++++ src/util/vircommandpriv.h | 4 ++++ 4 files changed, 22 insertions(+), 8 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 576ec8f95f..94a1fb6f33 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2086,6 +2086,7 @@ virCommandNewArgs; virCommandNewVAList; virCommandNonblockingFDs; virCommandPassFD; +virCommandPeekSendBuffers; virCommandRawStatus; virCommandRequireHandshake; virCommandRun; diff --git a/src/util/vircommand.c b/src/util/vircommand.c index 0917bc9cfb..e81b29669a 100644 --- a/src/util/vircommand.c +++ b/src/util/vircommand.c @@ -77,14 +77,6 @@ struct _virCommandFD { unsigned int flags; }; =20 -typedef struct _virCommandSendBuffer virCommandSendBuffer; -struct _virCommandSendBuffer { - int fd; - unsigned char *buffer; - size_t buflen; - off_t offset; -}; - struct _virCommand { int has_error; /* 0 on success, -1 on error */ =20 @@ -3452,3 +3444,12 @@ virCommandSetRunAmong(virCommand *cmd, =20 cmd->schedCore =3D pid; } + +void +virCommandPeekSendBuffers(virCommand *cmd, + virCommandSendBuffer **buffers, + int *nbuffers) +{ + *buffers =3D cmd->sendBuffers; + *nbuffers =3D cmd->numSendBuffers; +} diff --git a/src/util/vircommand.h b/src/util/vircommand.h index e0002103b6..0f14d14e49 100644 --- a/src/util/vircommand.h +++ b/src/util/vircommand.h @@ -24,6 +24,14 @@ #include "internal.h" #include "virbuffer.h" =20 +typedef struct _virCommandSendBuffer virCommandSendBuffer; +struct _virCommandSendBuffer { + int fd; + unsigned char *buffer; + size_t buflen; + off_t offset; +}; + typedef struct _virCommand virCommand; =20 /* This will execute in the context of the first child diff --git a/src/util/vircommandpriv.h b/src/util/vircommandpriv.h index ff17fa5ded..d579810bb5 100644 --- a/src/util/vircommandpriv.h +++ b/src/util/vircommandpriv.h @@ -47,3 +47,7 @@ void virCommandSetDryRun(virCommandDryRunToken *tok, bool bufCommandStripPath, virCommandDryRunCallback cb, void *opaque); + +void virCommandPeekSendBuffers(virCommand *cmd, + virCommandSendBuffer **buffers, + int *nbuffers); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252238; cv=none; d=zohomail.com; s=zohoarc; b=mzyk85M+LWFFPSIdcyTWS8381B20qyZh4lGGhUfXcIW/htp4tbAliaAnhDcvqavARViY/2ElzEsYa9n0YKD0AvXcDocKab+kyY8oDDc1f1GfY6bJnyvkUOC/9eTl3ze3jDd4ew/tb/v0TfnhTBXrTzkYJDDMYF6bfXJ6WZUJwto= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252238; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=B58i+vuLSDZRBX0uRGkofchj/Xilb/F81rNm+6AxzyI=; b=Uv8ZHa5EfSiNy7FDFG2uj+Q63mghzM68ZItAnFg88COvuNO+ZgNZqPRASSdSbb7sIByI2XeUEUz5v60QxAeWI6GuHBB2KvBXN8PtxDviNXs7upvw8+CqvCVJfamqjqGxfD+ECVI/hpVU0UGRHKkTFh+7ZJvZs4rqZPTD304l2eI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252238068638.6622144266606; Fri, 20 Jan 2023 14:03:58 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-634-WLdcU6vrO9aa2pCoXZWREg-1; Fri, 20 Jan 2023 17:03:55 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 426511066149; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1FC2E2166B2F; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 719951946A7F; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id F3A571946588 for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id E6E5214171C0; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C617F140EBF6 for ; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252237; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=B58i+vuLSDZRBX0uRGkofchj/Xilb/F81rNm+6AxzyI=; b=UhXfoP6Epkb+q8DenLy77tGwT4+Nl2aFFO98MWXdjisKN22+j56HDuAFqqjtvHhnnlj9NF aBdAdNJWrgRPrb/L2IojyzLqccXHt94T9+1NRzQT1cYVKnE4T2ZMtrs/yIIYm4yn/PJEot HIvtVWPaPsXLXeMe9IazBnooVqvvtf0= X-MC-Unique: WLdcU6vrO9aa2pCoXZWREg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 22/31] tests: add tests for nbdkit invocation Date: Fri, 20 Jan 2023 16:03:16 -0600 Message-Id: <20230120220325.1015090-23-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252239835100010 Content-Type: text/plain; charset="utf-8"; x-default="true" We were testing the arguments that were being passed to qemu when a disk was being served by nbdkit, but the arguments used to start nbdkit itself were not testable. This adds a test to ensure that we're invoking nbdkit correctly for various disk source definitions. Signed-off-by: Jonathon Jongsma --- build-aux/syntax-check.mk | 2 +- src/qemu/qemu_nbdkit.c | 4 +- src/qemu/qemu_nbdkitpriv.h | 31 ++ tests/meson.build | 1 + .../disk-cdrom-network.args.disk0 | 6 + .../disk-cdrom-network.args.disk1 | 8 + .../disk-cdrom-network.args.disk1.pipe.778 | 1 + .../disk-cdrom-network.args.disk2 | 8 + .../disk-cdrom-network.args.disk2.pipe.780 | 1 + .../disk-network-http.args.disk0 | 6 + .../disk-network-http.args.disk1 | 5 + .../disk-network-http.args.disk2 | 6 + .../disk-network-http.args.disk2.pipe.778 | 1 + .../disk-network-http.args.disk3 | 7 + .../disk-network-http.args.disk3.pipe.780 | 1 + ...work-source-curl-nbdkit-backing.args.disk0 | 7 + ...ce-curl-nbdkit-backing.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk0 | 7 + ...sk-network-source-curl.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk1 | 7 + ...sk-network-source-curl.args.disk1.pipe.780 | 1 + .../disk-network-source-curl.args.disk2 | 7 + ...sk-network-source-curl.args.disk2.pipe.782 | 1 + .../disk-network-source-curl.args.disk3 | 6 + .../disk-network-source-curl.args.disk4 | 6 + .../disk-network-ssh.args.disk0 | 6 + tests/qemunbdkittest.c | 300 ++++++++++++++++++ 27 files changed, 436 insertions(+), 2 deletions(-) create mode 100644 src/qemu/qemu_nbdkitpriv.h create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe= .778 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe= .780 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.= 778 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.= 780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-ba= cking.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-ba= cking.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 1.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 2.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0 create mode 100644 tests/qemunbdkittest.c diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index e1d80bd536..a73de07db1 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -1359,7 +1359,7 @@ exclude_file_name_regexp--sc_prohibit_close =3D \ (\.p[yl]$$|\.spec\.in$$|^docs/|^(src/util/vir(file|event)\.c|src/libvirt= -stream\.c|tests/(vir.+mock\.c|commandhelper\.c|qemusecuritymock\.c)|tools/= nss/libvirt_nss_(leases|macs)\.c)|tools/virt-qemu-qmp-proxy$$) =20 exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF =3D \ - (^tests/(nodedevmdevctl|virhostcpu|virpcitest|virstoragetest)data/|docs/= js/.*\.js|docs/fonts/.*\.woff|\.diff|tests/virconfdata/no-newline\.conf$$) + (^tests/(nodedevmdevctl|virhostcpu|virpcitest|virstoragetest|qemunbdkit)= data/|docs/js/.*\.js|docs/fonts/.*\.woff|\.diff|tests/virconfdata/no-newlin= e\.conf$$) =20 exclude_file_name_regexp--sc_prohibit_fork_wrappers =3D \ (^(src/(util/(vircommand|virdaemon)|lxc/lxc_controller)|tests/testutils)= \.c$$) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 85501b8373..5848710dc2 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -32,6 +32,8 @@ #include "qemu_driver.h" #include "qemu_extdevice.h" #include "qemu_nbdkit.h" +#define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +#include "qemu_nbdkitpriv.h" #include "qemu_security.h" =20 #include @@ -949,7 +951,7 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *pro= c, } =20 =20 -static virCommand * +virCommand * qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) { g_autoptr(virCommand) cmd =3D virCommandNewArgList(proc->caps->path, diff --git a/src/qemu/qemu_nbdkitpriv.h b/src/qemu/qemu_nbdkitpriv.h new file mode 100644 index 0000000000..64f9bb99d8 --- /dev/null +++ b/src/qemu/qemu_nbdkitpriv.h @@ -0,0 +1,31 @@ +/* + * qemu_nbdkitpriv.h: exposing some functions for testing + * + * Copyright (C) 2021 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + */ + +#ifndef LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +# error "qemu_nbdkitpriv.h may only be included by qemu_nbdkit.c or test s= uites" +#endif /* LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW */ + +#pragma once + +#include "qemu_nbdkit.h" + +virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc); diff --git a/tests/meson.build b/tests/meson.build index 3365dce307..079fc0eb41 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -452,6 +452,7 @@ if conf.has('WITH_QEMU') { 'name': 'qemuvhostusertest', 'link_with': [ test_qemu_driver_lib ], = 'link_whole': [ test_file_wrapper_lib ] }, { 'name': 'qemuxml2argvtest', 'link_with': [ test_qemu_driver_lib, tes= t_utils_qemu_monitor_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_= wrapper_lib ] }, { 'name': 'qemuxml2xmltest', 'link_with': [ test_qemu_driver_lib ], 'l= ink_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, + { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'li= nk_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, ] endif =20 diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk0 b/tests/qem= unbdkitdata/disk-cdrom-network.args.disk0 new file mode 100644 index 0000000000..b2f3be4cba --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk0 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftp \ +url=3Dftp://host.name:21/url/path/file.iso diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk1 b/tests/qem= unbdkitdata/disk-cdrom-network.args.disk1 new file mode 100644 index 0000000000..a23f6573d6 --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftps \ +url=3Dftps://host.name:990/url/path/file.iso \ +user=3Dtestuser \ +password=3D-777 diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 b/= tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk2 b/tests/qem= unbdkitdata/disk-cdrom-network.args.disk2 new file mode 100644 index 0000000000..04e918609a --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttps \ +'url=3Dhttps://host.name:443/url/path/file.iso?test=3Dval' \ +user=3Dtestuser \ +password=3D-779 diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 b/= tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk0 b/tests/qemu= nbdkitdata/disk-network-http.args.disk0 new file mode 100644 index 0000000000..8316f353cb --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk0 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground curl \ +protocols=3Dhttp,https \ +url=3Dhttp://example.org:80/test.img \ +timeout=3D1234 diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk1 b/tests/qemu= nbdkitdata/disk-network-http.args.disk1 new file mode 100644 index 0000000000..a546a68b27 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk1 @@ -0,0 +1,5 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground curl \ +protocols=3Dhttps \ +url=3Dhttps://example.org:443/test2.img diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk2 b/tests/qemu= nbdkitdata/disk-network-http.args.disk2 new file mode 100644 index 0000000000..1004547b3a --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk2 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground curl \ +protocols=3Dhttp,https \ +url=3Dhttp://example.org:1234/test3.img \ +cookie=3D-777 diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 b/t= ests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 new file mode 100644 index 0000000000..2c42c95930 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 @@ -0,0 +1 @@ +test=3Dtestcookievalue; test2=3D"blurb" \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk3 b/tests/qemu= nbdkitdata/disk-network-http.args.disk3 new file mode 100644 index 0000000000..e3c357b89a --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk3 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-3/nbdkit-test-disk-3.socket \ +--foreground curl \ +protocols=3Dhttps \ +'url=3Dhttps://example.org:1234/test4.img?par=3Dval&other=3Dble' \ +cookie=3D-779 \ +sslverify=3Dfalse diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 b/t= ests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 new file mode 100644 index 0000000000..2c42c95930 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 @@ -0,0 +1 @@ +test=3Dtestcookievalue; test2=3D"blurb" \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.a= rgs.disk0 b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.ar= gs.disk0 new file mode 100644 index 0000000000..605354433b --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.dis= k0 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttps \ +url=3Dhttps://https.example.org:8443/path/to/disk1.qcow2 \ +cookie=3D-777 diff --git a/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.a= rgs.disk0.pipe.778 b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-b= acking.args.disk0.pipe.778 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.dis= k0.pipe.778 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk0 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk0 new file mode 100644 index 0000000000..948dbfbe5a --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttps \ +url=3Dhttps://https.example.org:8443/path/to/disk1.iso \ +cookie=3D-777 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.= 778 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk1 new file mode 100644 index 0000000000..fde6a4f533 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground curl \ +protocols=3Dhttps \ +'url=3Dhttps://https.example.org:8443/path/to/disk5.iso?foo=3Dbar' \ +cookie=3D-779 \ +sslverify=3Dfalse diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.= 780 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk2 new file mode 100644 index 0000000000..88c9fa35a1 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground \ +--readonly curl \ +protocols=3Dhttp,https \ +url=3Dhttp://http.example.org:8080/path/to/disk2.iso \ +cookie=3D-781 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.= 782 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 new file mode 100644 index 0000000000..5c035e84c5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2; cookie3=3Dcookievalue3 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk3 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk3 new file mode 100644 index 0000000000..f517baa948 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk3 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-3/nbdkit-test-disk-3.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftp \ +url=3Dftp://ftp.example.org:20/path/to/disk3.iso diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk4 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk4 new file mode 100644 index 0000000000..1df47a9d54 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk4 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-4/nbdkit-test-disk-4.socket \ +--foreground \ +--readonly curl \ +protocols=3Dftps \ +url=3Dftps://ftps.example.org:22/path/to/disk4.iso diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk0 b/tests/qemun= bdkitdata/disk-network-ssh.args.disk0 new file mode 100644 index 0000000000..c04dc8bb03 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk0 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground ssh \ +host=3Dexample.org \ +port=3D2222 \ +path=3Dtest.img diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c new file mode 100644 index 0000000000..5606e155eb --- /dev/null +++ b/tests/qemunbdkittest.c @@ -0,0 +1,300 @@ +#include + +#include +#include "internal.h" +#include "testutils.h" +#include "testutilsqemu.h" +#include "qemu/qemu_domain.h" +#include "qemu/qemu_nbdkit.h" +#define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +#include "qemu/qemu_nbdkitpriv.h" +#include "vircommand.h" +#define LIBVIRT_VIRCOMMANDPRIV_H_ALLOW +#include "vircommandpriv.h" +#include "virutil.h" +#include "virsecret.h" +#include "datatypes.h" +#include "virmock.h" + +#define VIR_FROM_THIS VIR_FROM_QEMU + +static virQEMUDriver driver; + + +/* Some mock implementations for testing */ +#define PIPE_FD_START 777 +static int mockpipefd =3D PIPE_FD_START; + +static int (*real_virPipeQuiet)(int fds[2]); +static void +init_syms(void) +{ + VIR_MOCK_REAL_INIT(virPipeQuiet); +} + +static int +moveToStableFd(int fd) +{ + int newfd; + + /* don't overwrite an existing fd */ + if (fcntl(mockpipefd, F_GETFD) !=3D -1) + abort(); + + newfd =3D dup2(fd, mockpipefd++); + + VIR_FORCE_CLOSE(fd); + + return newfd; +} + + +int +virPipeQuiet(int fds[2]) +{ + int tempfds[2]; + + init_syms(); + + if (real_virPipeQuiet(tempfds) < 0) + return -1; + + if ((fds[0] =3D moveToStableFd(tempfds[0])) < 0 || + (fds[1] =3D moveToStableFd(tempfds[1])) < 0) + return -1; + + return 0; +} + + +int +virSecretGetSecretString(virConnectPtr conn G_GNUC_UNUSED, + virSecretLookupTypeDef *seclookupdef, + virSecretUsageType secretUsageType, + uint8_t **secret, + size_t *secret_size) +{ + char uuidstr[VIR_UUID_BUFLEN]; + const char *secretname =3D NULL; + char *tmp =3D NULL; + + switch (seclookupdef->type) { + case VIR_SECRET_LOOKUP_TYPE_UUID: + virUUIDFormat(seclookupdef->u.uuid, uuidstr); + secretname =3D uuidstr; + break; + case VIR_SECRET_LOOKUP_TYPE_USAGE: + secretname =3D seclookupdef->u.usage; + break; + case VIR_SECRET_LOOKUP_TYPE_NONE: + case VIR_SECRET_LOOKUP_TYPE_LAST: + default: + virReportEnumRangeError(virSecretLookupType, seclookupdef->typ= e); + return -1; + }; + + /* For testing, just generate a value for the secret that includes the= type + * and the id of the secret */ + tmp =3D g_strdup_printf("%s-%s-secret", virSecretUsageTypeToString(sec= retUsageType), secretname); + *secret =3D (uint8_t*)tmp; + *secret_size =3D strlen(tmp) + 1; + + return 0; +} + +virConnectPtr virGetConnectSecret(void) +{ + return virGetConnect(); +} + +/* end of mock implementations */ + + +typedef struct { + const char *name; + char* infile; + char* outtemplate; + qemuNbdkitCaps *nbdkitcaps; + bool expectFail; +} TestInfo; + + +typedef enum { + NBDKIT_ARG_CAPS, + NBDKIT_ARG_EXPECT_FAIL, + NBDKIT_ARG_END +} NbdkitArgName; + + +static void +testInfoSetPaths(TestInfo *info) +{ + info->infile =3D g_strdup_printf("%s/qemuxml2argvdata/%s.xml", + abs_srcdir, info->name); + info->outtemplate =3D g_strdup_printf("%s/qemunbdkitdata/%s", + abs_srcdir, info->name); +} + +static void +testInfoClear(TestInfo *info) +{ + g_free(info->infile); + g_free(info->outtemplate); + g_clear_object(&info->nbdkitcaps); +} + +static void +testInfoSetArgs(TestInfo *info, ...) +{ + va_list argptr; + NbdkitArgName argname; + unsigned int cap; + + va_start(argptr, info); + while ((argname =3D va_arg(argptr, NbdkitArgName)) !=3D NBDKIT_ARG_END= ) { + switch (argname) { + case NBDKIT_ARG_CAPS: + while ((cap =3D va_arg(argptr, unsigned int)) < QEMU_NBDKI= T_CAPS_LAST) + qemuNbdkitCapsSet(info->nbdkitcaps, cap); + break; + case NBDKIT_ARG_EXPECT_FAIL: + info->expectFail =3D va_arg(argptr, unsigned int); + break; + case NBDKIT_ARG_END: + default: + break; + } + } +} + + +static int +testNbdkit(const void *data) +{ + const TestInfo *info =3D data; + g_autoptr(virDomainDef) def =3D NULL; + size_t i; + int ret =3D 0; + + /* restart mock pipe fds so tests are consistent */ + mockpipefd =3D PIPE_FD_START; + + if (!virFileExists(info->infile)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "Test input file '%s' is missing", info->infile); + return -1; + } + + if (!(def =3D virDomainDefParseFile(info->infile, driver.xmlopt, NULL, + VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE))) + return -1; + + for (i =3D 0; i < def->ndisks; i++) { + virDomainDiskDef *disk =3D def->disks[i]; + g_autofree char *statedir =3D g_strdup_printf("/tmp/statedir-%zi",= i); + g_autofree char *alias =3D g_strdup_printf("test-disk-%zi", i); + g_autofree char *cmdfile =3D g_strdup_printf("%s.args.disk%zi", + info->outtemplate, i); + + if (qemuNbdkitInitStorageSource(info->nbdkitcaps, disk->src, state= dir, + alias, 101, 101)) { + qemuDomainStorageSourcePrivate *srcPriv =3D + qemuDomainStorageSourcePrivateFetch(disk->src); + g_autoptr(virCommand) cmd =3D NULL; + g_autoptr(virCommandDryRunToken) dryRunToken =3D virCommandDry= RunTokenNew(); + g_auto(virBuffer) buf =3D VIR_BUFFER_INITIALIZER; + g_autofree char *actualCmdline =3D NULL; + virCommandSendBuffer *sendbuffers; + int nsendbuffers; + size_t j; + + virCommandSetDryRun(dryRunToken, &buf, true, true, NULL, NULL); + cmd =3D qemuNbdkitProcessBuildCommand(srcPriv->nbdkitProcess); + + if (virCommandRun(cmd, NULL) < 0) { + ret =3D -1; + continue; + } + virCommandPeekSendBuffers(cmd, &sendbuffers, &nsendbuffers); + + if (!(actualCmdline =3D virBufferContentAndReset(&buf))) { + ret =3D -1; + continue; + } + + if (virTestCompareToFileFull(actualCmdline, cmdfile, false) < = 0) + ret =3D -1; + + for (j =3D 0; j < nsendbuffers; j++) { + virCommandSendBuffer *buffer =3D &sendbuffers[j]; + g_autofree char *pipefile =3D g_strdup_printf("%s.pipe.%i", + cmdfile, + buffer->fd); + + if (virTestCompareToFile((const char*)buffer->buffer, pipe= file) < 0) + ret =3D -1; + } + } else { + if (virFileExists(cmdfile)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + "qemuNbdkitInitStorageSource() was not expe= cted to fail"); + ret =3D -1; + } + } + } + + if (info->expectFail) { + if (ret =3D=3D 0) { + ret =3D -1; + VIR_TEST_DEBUG("Error expected but there wasn't any."); + } else { + ret =3D 0; + } + } + return ret; +} + +static int +mymain(void) +{ + int ret =3D 0; + + if (qemuTestDriverInit(&driver) < 0) + return EXIT_FAILURE; + + +#define DO_TEST_FULL(_name, ...) \ + do { \ + TestInfo info =3D { \ + .name =3D _name, \ + .nbdkitcaps =3D qemuNbdkitCapsNew(TEST_NBDKIT_PATH), \ + }; \ + testInfoSetPaths(&info); \ + testInfoSetArgs(&info, __VA_ARGS__); \ + virTestRunLog(&ret, "nbdkit " _name, testNbdkit, &info); \ + testInfoClear(&info); \ + } while (0) + +#define DO_TEST(_name, ...) \ + DO_TEST_FULL(_name, NBDKIT_ARG_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAS= T, NBDKIT_ARG_END) + +#define DO_TEST_FAILURE(_name, ...) \ + DO_TEST_FULL(_name, \ + NBDKIT_ARG_EXPECT_FAIL, 1, \ + NBDKIT_ARG_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, NBDK= IT_ARG_END) + +#define DO_TEST_NOCAPS(_name) \ + DO_TEST_FULL(_name, NBDKIT_ARG_END) + + DO_TEST("disk-cdrom-network", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-http", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT_CAPS_PL= UGIN_CURL); + DO_TEST("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + + qemuTestDriverFree(&driver); + + return ret =3D=3D 0 ? EXIT_SUCCESS : EXIT_FAILURE; +} + +VIR_TEST_MAIN(mymain) --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252234; cv=none; d=zohomail.com; s=zohoarc; b=MB1onBy3rgm4yYZREzZ0FNFp+INvhpZQg4jTeHUHPeYVw+dlUG1R1iLGCnVZi6Mx9cX/lGhl1/KxkSLcyjRKr+6GI/oSVAB7uMtkQBCgR+jFScZLkcTCAmZDWhrBbKB5qX1R7YQkduSMUL2bVxcQ0dIwa8F4wCanfW0fu8cj6nQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252234; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PDVeTWAdURgluXpFM+ryXxn0iRjDu1i3lpJGlJCn4Kc=; b=ZmsL7GwTjHXVjyIg/g9tt4KHrKeqOJ0dsomLpVYbiVg+JqfRyZi7IwqTnrG4ZmJxyrxVtapdLb92t1WA8K2eNXepLsdTkheq2S+vIQf9pdqJVHLONjM6FUdbCf5nUYDA1GV3ILEZgq6mlcOpuEbOThVTwpARLJIthuyLWkvCsiY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252234873508.7516113541104; Fri, 20 Jan 2023 14:03:54 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-558-f839GOqpN028ia5547U2yg-1; Fri, 20 Jan 2023 17:03:51 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DE625889062; Fri, 20 Jan 2023 22:03:40 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id BE45D2027009; Fri, 20 Jan 2023 22:03:40 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 9A51C1946A78; Fri, 20 Jan 2023 22:03:40 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 4ADD81946588 for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 3094114171C1; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 06123140EBF6; Fri, 20 Jan 2023 22:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252233; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=PDVeTWAdURgluXpFM+ryXxn0iRjDu1i3lpJGlJCn4Kc=; b=awX8tbA73fScX5QwfBYKrteLpvNYmWRZbOMGnNfHlR3FcaLLqjG58eS2RDB0DyLaNWjpAu LXI1w4Iji12l/xBSqUNemdEdptTmUKk32kZ2nZSPvK30HyOK+y9VUikyZrUp2IZjtX/PJA B8mNOskfKWZjgeRgkq6avHAKOlfWFp4= X-MC-Unique: f839GOqpN028ia5547U2yg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 23/31] qemu: add test for authenticating a https network disk Date: Fri, 20 Jan 2023 16:03:17 -0600 Message-Id: <20230120220325.1015090-24-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Krempa Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252235788100005 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- tests/qemunbdkitdata/disk-network-source-curl.args.disk1 | 4 +++- .../disk-network-source-curl.args.disk1.pipe.780 | 2 +- .../disk-network-source-curl.args.disk1.pipe.782 | 1 + tests/qemunbdkitdata/disk-network-source-curl.args.disk2 | 2 +- .../disk-network-source-curl.args.disk2.pipe.784 | 1 + .../disk-network-source-curl.x86_64-latest.args | 3 ++- tests/qemuxml2argvdata/disk-network-source-curl.xml | 3 +++ 7 files changed, 12 insertions(+), 4 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 1.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk= 2.pipe.784 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk1 index fde6a4f533..d1288dd242 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 @@ -3,5 +3,7 @@ nbdkit \ --foreground curl \ protocols=3Dhttps \ 'url=3Dhttps://https.example.org:8443/path/to/disk5.iso?foo=3Dbar' \ -cookie=3D-779 \ +user=3Dmyname \ +password=3D-779 \ +cookie=3D-781 \ sslverify=3Dfalse diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.= 780 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 index 20af4ae383..ccdd4033fc 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 @@ -1 +1 @@ -cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.= 782 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 b/tes= ts/qemunbdkitdata/disk-network-source-curl.args.disk2 index 88c9fa35a1..f1d0e1929e 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 @@ -4,4 +4,4 @@ nbdkit \ --readonly curl \ protocols=3Dhttp,https \ url=3Dhttp://http.example.org:8080/path/to/disk2.iso \ -cookie=3D-781 +cookie=3D-783 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.= 784 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 new file mode 100644 index 0000000000..5c035e84c5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 @@ -0,0 +1 @@ +cookie1=3Dcookievalue1; cookie2=3Dcookievalue2; cookie3=3Dcookievalue3 \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.= args b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args index ec6dd13f6c..7f09e84227 100644 --- a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args +++ b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args @@ -33,9 +33,10 @@ XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ -blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk1.iso","cookie-secret":"libvirt-5-storage-httpcookie-secret0","node-na= me":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw"= ,"file":"libvirt-5-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-storage-auth-secret0","data"= :"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","keyid"= :"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ -object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","= data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","k= eyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ -object '{"qom-type":"secret","id":"libvirt-4-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","= keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}'= \ --blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk5.iso?foo=3Dbar","sslverify":false,"cookie-secret":"libvirt-4-storage-= httpcookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"= discard":"unmap"}' \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to= /disk5.iso?foo=3Dbar","username":"myname","password-secret":"libvirt-4-stor= age-auth-secret0","sslverify":false,"cookie-secret":"libvirt-4-storage-http= cookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"disc= ard":"unmap"}' \ -blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luk= s","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-sto= rage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"li= bvirt-4-format","id":"virtio-disk4"}' \ -object '{"qom-type":"secret","id":"libvirt-3-storage-httpcookie-secret0",= "data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBBv7TuTgTkyAyOPpC2P5qLbOIy= pLoHpppjz+u5O+X8oT+jA1m7q/OJQ8dk2EFD5c0A=3D","keyid":"masterKey0","iv":"AAE= CAwQFBgcICQoLDA0ODw=3D=3D","format":"base64"}' \ diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.xml b/tests/qe= muxml2argvdata/disk-network-source-curl.xml index 1e50314abe..8c3982cd73 100644 --- a/tests/qemuxml2argvdata/disk-network-source-curl.xml +++ b/tests/qemuxml2argvdata/disk-network-source-curl.xml @@ -59,6 +59,9 @@ + + + --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252259; cv=none; d=zohomail.com; s=zohoarc; b=D11UndEbff0gu+uGxz6/bIgd/Q0XaTTTQdtAtN8nNzAs0NNN4TcLwkNb0PnU3jfzWM43VV40jeDFI0CjCbZfcJEDbSgfQAPFrPtuQNljUlv5jsKJTI5HPOY0zfWRIigutKywWP5jM+Gk4/lxyBrhAzEWt3ETJ/Kq6sOEmEcuI90= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252259; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3Rjw8FJPxsNUa/usQR+EV+d7cVX+EALNU8facErSi1k=; b=mhmB3G13Cpj4NCFHcucPlAfXBiwPP8XR7rSNj4+VzxFZ4fqMUYUI2RGGm8F91JU/GfPtMxyPFH4bNZrLnRXOhunqve2BLYmnK4wlj8JAdP6MEmpK+WfFkPx94U7ZpOSzWK2YrtQX3rXaPiKzcRGQt44OdMJEnsdZj7eBDYn+teU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252259567397.61011485306676; Fri, 20 Jan 2023 14:04:19 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-611-68jeME0iO7ODACZf0iA8dA-1; Fri, 20 Jan 2023 17:03:59 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 96407810BD8; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 820C8C16021; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 75F791946A7C; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6DFE41946A76 for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 63B7C14171C0; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 42C72140EBF6 for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252258; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3Rjw8FJPxsNUa/usQR+EV+d7cVX+EALNU8facErSi1k=; b=VrJA+OexfnDV7hKceKXN7Cpe8Lg8se8nfa6d3ZVYVhvEo0K7w4T7xP4m6Sn3ahW8Ba7npI eZrMW5IQSNeWIegnI8bZ0PqdMdnVKm4BbVLJuZbGBsNCzwVmcgVIk8COs0DjyyA2aVbY4R fwahO0R+465Nk5KKK7VQD2PEs4h/O7g= X-MC-Unique: 68jeME0iO7ODACZf0iA8dA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 24/31] qemu: Monitor nbdkit process for exit Date: Fri, 20 Jan 2023 16:03:18 -0600 Message-Id: <20230120220325.1015090-25-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252259931100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Adds the ability to monitor the nbdkit process so that we can take action in case the child exits unexpectedly. When the nbdkit process exits, we pause the vm, restart nbdkit, and then resume the vm. This allows the vm to continue working in the event of a nbdkit failure. Eventually we may want to generalize this functionality since we may need something similar for e.g. qemu-storage-daemon, etc. The process is monitored with the pidfd_open() syscall if it exists (since linux 5.3). Otherwise it resorts to checking whether the process is alive once a second. The one-second time period was chosen somewhat arbitrarily. Signed-off-by: Jonathon Jongsma --- meson.build | 7 ++ src/qemu/qemu_nbdkit.c | 166 +++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_nbdkit.h | 7 +- src/qemu/qemu_process.c | 4 +- 4 files changed, 177 insertions(+), 7 deletions(-) diff --git a/meson.build b/meson.build index e498b49be4..048b15ff71 100644 --- a/meson.build +++ b/meson.build @@ -645,6 +645,13 @@ symbols =3D [ [ 'sched.h', 'cpu_set_t' ], ] =20 +if host_machine.system() =3D=3D 'linux' + symbols +=3D [ + # process management + [ 'sys/syscall.h', 'SYS_pidfd_open' ], + ] +endif + foreach symbol : symbols if cc.has_header_symbol(symbol[0], symbol[1], args: '-D_GNU_SOURCE', pre= fix: symbol.get(2, '')) conf.set('WITH_DECL_@0@'.format(symbol[1].to_upper()), 1) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 5848710dc2..934970e68c 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -19,9 +19,11 @@ =20 #include #include +#include =20 #include "vircommand.h" #include "virerror.h" +#include "virevent.h" #include "virlog.h" #include "virpidfile.h" #include "virtime.h" @@ -34,6 +36,7 @@ #include "qemu_nbdkit.h" #define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW #include "qemu_nbdkitpriv.h" +#include "qemu_process.h" #include "qemu_security.h" =20 #include @@ -69,6 +72,12 @@ struct _qemuNbdkitCaps { G_DEFINE_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, G_TYPE_OBJECT); =20 =20 +struct _qemuNbdkitProcessPrivate { + int pidfdwatch; + virDomainObj *vm; +}; + + enum { PIPE_FD_READ =3D 0, PIPE_FD_WRITE =3D 1 @@ -618,6 +627,137 @@ qemuNbdkitCapsCacheNew(const char *cachedir) } =20 =20 +static int +qemuNbdkitProcessStartMonitor(qemuNbdkitProcess *proc, + virDomainObj *vm); + + +static void +qemuNbdkitProcessHandleExit(qemuNbdkitProcess *proc) +{ + qemuNbdkitProcessPrivate *priv =3D proc->priv; + qemuDomainObjPrivate *vmpriv =3D priv->vm->privateData; + virQEMUDriver *driver =3D vmpriv->driver; + + VIR_DEBUG("nbdkit process %i died", proc->pid); + + /* clean up resources associated with process */ + qemuNbdkitProcessStop(proc); + + if (!priv->vm) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot restart nbdkit process without an associa= ted domain")); + return; + } + + if (qemuNbdkitProcessStart(proc, priv->vm, driver) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to restart nbkdit process")); + return; + } + + qemuNbdkitProcessStartMonitor(proc, NULL); +} + + +#if WITH_DECL_SYS_PIDFD_OPEN +static void +qemuNbdkitProcessPidfdCb(int watch G_GNUC_UNUSED, + int fd, + int events G_GNUC_UNUSED, + void *opaque) +{ + qemuNbdkitProcess *proc =3D opaque; + + VIR_FORCE_CLOSE(fd); + qemuNbdkitProcessHandleExit(proc); +} +#else +static void +qemuNbdkitProcessTimeoutCb(int timer G_GNUC_UNUSED, + void *opaque) +{ + qemuNbdkitProcess *proc =3D opaque; + + if (virProcessKill(proc->pid, 0) < 0) + qemuNbdkitProcessHandleExit(proc); +} +#endif /* WITH_DECL_SYS_PIDFD_OPEN */ + + +static int +qemuNbdkitProcessStartMonitor(qemuNbdkitProcess *proc, + virDomainObj *vm) +{ + qemuNbdkitProcessPrivate *priv =3D proc->priv; +#if WITH_DECL_SYS_PIDFD_OPEN + int pidfd; +#endif + + if (vm) { + virObjectRef(vm); + + if (priv->vm) + virObjectUnref(priv->vm); + + priv->vm =3D vm; + } + + if (!priv->vm) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot monitor nbdkit process without an associa= ted domain")); + return -1; + } + +#if WITH_DECL_SYS_PIDFD_OPEN + pidfd =3D syscall(SYS_pidfd_open, proc->pid, 0); + if (pidfd < 0) + return -1; + + priv->pidfdwatch =3D virEventAddHandle(pidfd, + VIR_EVENT_HANDLE_READABLE, + qemuNbdkitProcessPidfdCb, + proc, NULL); +#else + /* fall back to checking once a second */ + priv->pidfdwatch =3D virEventAddTimeout(1000, + qemuNbdkitProcessTimeoutCb, + proc, NULL); +#endif /* WITH_DECL_SYS_PIDFD_OPEN */ + + if (priv->pidfdwatch < 0) + return -1; + + VIR_DEBUG("Monitoring nbdkit process %i for exit", proc->pid); + + return 0; +} + + +static void +qemuNbdkitProcessStopMonitor(qemuNbdkitProcess *proc) +{ + qemuNbdkitProcessPrivate *priv =3D proc->priv; + + if (priv->pidfdwatch > 0) { +#if WITH_DECL_SYS_PIDFD_OPEN + virEventRemoveHandle(priv->pidfdwatch); +#else + virEventRemoveTimeout(priv->pidfdwatch); +#endif /* WITH_DECL_SYS_PIDFD_OPEN */ + priv->pidfdwatch =3D 0; + } +} + + +static void +qemuNbdkitProcessPrivateFree(qemuNbdkitProcessPrivate *priv) +{ + virObjectUnref(priv->vm); + g_free(priv); +} + + static qemuNbdkitProcess * qemuNbdkitProcessNew(virStorageSource *source, const char *pidfile, @@ -631,6 +771,7 @@ qemuNbdkitProcessNew(virStorageSource *source, nbdkit->pid =3D -1; nbdkit->pidfile =3D g_strdup(pidfile); nbdkit->socketfile =3D g_strdup(socketfile); + nbdkit->priv =3D g_new0(qemuNbdkitProcessPrivate, 1); =20 return nbdkit; } @@ -665,9 +806,11 @@ qemuNbdkitReconnectStorageSource(virStorageSource *sou= rce, =20 =20 static int -qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source) +qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source, + virDomainObj *vm) { qemuDomainStorageSourcePrivate *srcpriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(source); + qemuDomainObjPrivate *vmpriv =3D vm->privateData; qemuNbdkitProcess *proc; =20 if (!srcpriv) @@ -676,6 +819,9 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSourc= e *source) proc =3D srcpriv->nbdkitProcess; =20 if (proc) { + if (!proc->caps) + proc->caps =3D qemuGetNbdkitCaps(vmpriv->driver); + if (proc->pid <=3D 0) { if (virPidFileReadPath(proc->pidfile, &proc->pid) < 0) return -1; @@ -686,6 +832,9 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSourc= e *source) _("nbdkit process %i is not alive"), proc->pid); return -1; } + + if (qemuNbdkitProcessStartMonitor(proc, vm) < 0) + return -1; } =20 return 0; @@ -701,15 +850,16 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSou= rce *source) * disk and is attempting to re-connect to active domains. */ int -qemuNbdkitStorageSourceManageProcess(virStorageSource *source) +qemuNbdkitStorageSourceManageProcess(virStorageSource *source, + virDomainObj *vm) { virStorageSource *backing; for (backing =3D source->backingStore; backing !=3D NULL; backing =3D = backing->backingStore) { - if (qemuNbdkitStorageSourceManageProcessOne(backing) < 0) + if (qemuNbdkitStorageSourceManageProcessOne(backing, vm) < 0) return -1; } =20 - return qemuNbdkitStorageSourceManageProcessOne(source); + return qemuNbdkitStorageSourceManageProcessOne(source, vm); } =20 =20 @@ -1005,9 +1155,12 @@ qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *pro= c) void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { + qemuNbdkitProcessStopMonitor(proc); + g_clear_pointer(&proc->pidfile, g_free); g_clear_pointer(&proc->socketfile, g_free); g_clear_object(&proc->caps); + g_clear_pointer(&proc->priv, qemuNbdkitProcessPrivateFree); g_free(proc); } =20 @@ -1087,6 +1240,9 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, goto error; } =20 + if (qemuNbdkitProcessStartMonitor(proc, vm) < 0) + goto error; + return 0; =20 error: @@ -1107,6 +1263,8 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, int qemuNbdkitProcessStop(qemuNbdkitProcess *proc) { + qemuNbdkitProcessStopMonitor(proc); + if (proc->pid < 0) return 0; =20 diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 8191ace522..df45f409c0 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -69,7 +69,8 @@ void qemuNbdkitStopStorageSource(virStorageSource *src); =20 int -qemuNbdkitStorageSourceManageProcess(virStorageSource *src); +qemuNbdkitStorageSourceManageProcess(virStorageSource *src, + virDomainObj *vm); =20 bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, @@ -82,6 +83,8 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, #define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, = GObject); =20 +typedef struct _qemuNbdkitProcessPrivate qemuNbdkitProcessPrivate; + struct _qemuNbdkitProcess { qemuNbdkitCaps *caps; virStorageSource *source; @@ -91,6 +94,8 @@ struct _qemuNbdkitProcess { uid_t user; gid_t group; pid_t pid; + + qemuNbdkitProcessPrivate *priv; }; =20 int diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 7ec31ef6ac..54fd44fb40 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -9049,12 +9049,12 @@ qemuProcessReconnect(void *opaque) =20 for (i =3D 0; i < obj->def->ndisks; i++) { virDomainDiskDef *disk =3D obj->def->disks[i]; - if (qemuNbdkitStorageSourceManageProcess(disk->src) < 0) + if (qemuNbdkitStorageSourceManageProcess(disk->src, obj) < 0) goto error; } =20 if (obj->def->os.loader && obj->def->os.loader->nvram) { - if (qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvra= m) < 0) + if (qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvra= m, obj) < 0) goto error; } =20 --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252377; cv=none; d=zohomail.com; s=zohoarc; b=VDMleNKC0rl/3UhNQQf4cTTlFYLr89mqgqBgvGB8N07euasQy0nz9R7r/DLWx59wHC/jzjVMGjPJxACiRCCINq6tzT+b9pyRDsKuKoBEEk/IPHvzH799bejMn9/4KvIQwj2QhQz0ANFCxXBHmdQwETMulrrw+iMHNZeprE2FVJ8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252377; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5eaXoU6gWsMARdrTQwimbct6v9jYFv+PqeaAD588CfU=; b=PjlKufebvG1guutB4VLUn29dBJI+J4eWGYXgIiWYMn7FFLfeBYyDWs2Hb8rbpYIa+QWXNqbMnuvX0ZntPnAVYb/Jl7e+5A8jH4fC/OzRV8vRm1RTVLScqcdqIS+dS3zbMjC10hkTaieU8hBEo55L7kbX18etz9E5nsZ6N6PGh3g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 167425237756232.62297660029628; Fri, 20 Jan 2023 14:06:17 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-55-hO5jyOA4Nzu002xRD9R9CQ-1; Fri, 20 Jan 2023 17:03:50 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5D1D01C270C6; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45AF82166B2C; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D9C85194704E; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A07101946A6D for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 9654414171C2; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 755B514171C0 for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252376; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=5eaXoU6gWsMARdrTQwimbct6v9jYFv+PqeaAD588CfU=; b=XKZfCQFVON46Cj9BmYwSalUJNJrh0HQQTcfcrOxfIRw1hDwrthSVjXT565NVWX99TCXoux wA8MZuiu5yfFCooYw5FBS6xX1Rro5p4A/D98kLBMWKNFeZwmN+fXyzKMAvkE0sTiQRATfk JsOg35hix1cxD/L/b0MTENmmNcjM1Wg= X-MC-Unique: hO5jyOA4Nzu002xRD9R9CQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 25/31] qemu: try to connect to nbdkit early to detect errors Date: Fri, 20 Jan 2023 16:03:19 -0600 Message-Id: <20230120220325.1015090-26-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252377895100001 Content-Type: text/plain; charset="utf-8"; x-default="true" When using nbdkit to serve a network disk source, the nbdkit process will start and wait for an nbd connection before actually attempting to connect to the (remote) disk location. Because of this, nbdkit will not report an error until after qemu is launched and tries to read from the disk. This results in a fairly user-unfriendly error saying that qemu was unable to start because "Requested export not available". Ideally we'd like to be able to tell the user *why* the export is not available, but this sort of information is only available to nbdkit, not qemu. It could be because the url was incorrect, or because of an authentication failure, or one of many other possibilities. To make this friendlier for users and easier to detect misconfigurations, try to connect to nbdkit immediately after starting nbdkit and before we try to start qemu. This requires adding a dependency on libnbd. If an error occurs when connecting to nbdkit, read back from the nbdkit error log and provide that information in the error report from qemuNbdkitProcessStart(). User-visible change demonstrated below: Previous error: $ virsh start nbdkit-test 2023-01-18 19:47:45.778+0000: 30895: error : virNetClientProgramDispatc= hError:172 : internal error: process exited while connecting to monitor: 2023-01-18T19:47:45.= 704658Z qemu-system-x86_64: -blockdev {"driver":"nbd","server":{"type":"unix", "path":"/var/lib/libvirt/qemu/domain-1-nbdkit-test/nbdkit-libvirt-1-sto= rage.socket"}, "node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"= }: Requested export not available error: Failed to start domain 'nbdkit-test' error: internal error: process exited while connecting to monitor: 2023= -01-18T19:47:45.704658Z qemu-system-x86_64: -blockdev {"driver":"nbd","server":{"type":"unix", "path":"/var/lib/libvirt/qemu/domain-1-nbdkit-test/nbdkit-libvirt-1-sto= rage.socket"}, "node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"= }: Requested export not available After this change: $ virsh start nbdkit-test 2023-01-18 19:44:36.242+0000: 30895: error : virNetClientProgramDispatc= hError:172 : internal error: Failed to connect to nbdkit for 'http://localhost:8888/nonexiste= nt.iso': nbdkit: curl[1]: error: problem doing HEAD request to fetch size of URL [http://localhos= t:8888/nonexistent.iso]: HTTP response code said error: The requested URL returned error: 404 error: Failed to start domain 'nbdkit-test' error: internal error: Failed to connect to nbdkit for 'http://localhos= t:8888/nonexistent.iso]: error: problem doing HEAD request to fetch size of URL [http://localhos= t:8888/nonexistent.iso]: HTTP response code said error: The requested URL returned error: 404 Signed-off-by: Jonathon Jongsma --- meson.build | 7 +++++++ meson_options.txt | 1 + src/qemu/meson.build | 1 + src/qemu/qemu_nbdkit.c | 24 ++++++++++++++++++++++++ 4 files changed, 33 insertions(+) diff --git a/meson.build b/meson.build index 048b15ff71..83692bef22 100644 --- a/meson.build +++ b/meson.build @@ -959,6 +959,12 @@ endif libiscsi_version =3D '1.18.0' libiscsi_dep =3D dependency('libiscsi', version: '>=3D' + libiscsi_version= , required: get_option('libiscsi')) =20 +libnbd_version =3D '1.0' +libnbd_dep =3D dependency('libnbd', version: '>=3D' + libnbd_version, requ= ired: get_option('libnbd')) +if libnbd_dep.found() + conf.set('WITH_LIBNBD', 1) +endif + libnl_version =3D '3.0' if not get_option('libnl').disabled() and host_machine.system() =3D=3D 'li= nux' libnl_dep =3D dependency('libnl-3.0', version: '>=3D' + libnl_version, r= equired: get_option('libnl')) @@ -2200,6 +2206,7 @@ libs_summary =3D { 'glusterfs': glusterfs_dep.found(), 'libiscsi': libiscsi_dep.found(), 'libkvm': libkvm_dep.found(), + 'libnbd': libnbd_dep.found(), 'libnl': libnl_dep.found(), 'libparted': libparted_dep.found(), 'libpcap': libpcap_dep.found(), diff --git a/meson_options.txt b/meson_options.txt index 861c5577d2..2d68bd7349 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -24,6 +24,7 @@ option('curl', type: 'feature', value: 'auto', descriptio= n: 'curl support') option('fuse', type: 'feature', value: 'auto', description: 'fuse support') option('glusterfs', type: 'feature', value: 'auto', description: 'glusterf= s support') option('libiscsi', type: 'feature', value: 'auto', description: 'libiscsi = support') +option('libnbd', type: 'feature', value: 'auto', description: 'libnbd supp= ort') option('libnl', type: 'feature', value: 'auto', description: 'libnl suppor= t') option('libpcap', type: 'feature', value: 'auto', description: 'libpcap su= pport') option('libssh', type: 'feature', value: 'auto', description: 'libssh supp= ort') diff --git a/src/qemu/meson.build b/src/qemu/meson.build index 6d7a1bfbb0..607b597c8c 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -99,6 +99,7 @@ if conf.has('WITH_QEMU') access_dep, capng_dep, gnutls_dep, + libnbd_dep, libnl_dep, log_dep, selinux_dep, diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 934970e68c..98f8b70391 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -19,6 +19,9 @@ =20 #include #include +#if WITH_LIBNBD +# include +#endif #include =20 #include "vircommand.h" @@ -26,6 +29,7 @@ #include "virevent.h" #include "virlog.h" #include "virpidfile.h" +#include "virstring.h" #include "virtime.h" #include "virutil.h" #include "qemu_block.h" @@ -1190,6 +1194,9 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, g_autofree char *errmsg =3D NULL; g_autoptr(virURI) uri =3D NULL; g_autofree char *uristring =3D NULL; +#if WITH_LIBNBD + struct nbd_handle *nbd =3D NULL; +#endif =20 if (!(cmd =3D qemuNbdkitProcessBuildCommand(proc))) return -1; @@ -1240,6 +1247,23 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, goto error; } =20 +#if WITH_LIBNBD + /* if the disk source was misconfigured, nbdkit will not produce an er= ror + * until somebody connects to the socket and tries to access the nbd + * export. This results in poor user experience because the only error= we + * would get from qemu is something like "Requested export not availab= le". + * So let's try to access it ourselves so that we can error out early = and + * provide a useful message to the user. + */ + nbd =3D nbd_create(); + if (nbd_connect_unix(nbd, proc->socketfile) < 0) { + VIR_WARN("nbd_connect_unix failed: %s", nbd_get_error()); + nbd_close(nbd); + goto error; + } + nbd_close(nbd); +#endif + if (qemuNbdkitProcessStartMonitor(proc, vm) < 0) goto error; =20 --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252239; cv=none; d=zohomail.com; s=zohoarc; b=SEpogpAmdGPPAYLJCVV6Rs8ubvEJB1JklAR6MCfFQTIFhrZQr9pmeJ69TUzKIDYtUux1SWrzaMzqQVBNrvWqgo8USi4GdhbKpkjFNCnzhPv0bGocE8nb1rUmNV5jqYLWxVziUOuBXsjAYonxaOwa9vRObM6/aZMEXpARMNrzGN8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252239; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7q+OQcsDsX8ZYRm6/EdEuri0kbO4wPBTvvdFeHFykds=; b=DPtX56ZMligGJ58bxFEaWE+zQ/0rmWqyJ2n7tXG5l1LpBV6gIiTh4GCd5Lg1wqVkZ1nNwTxkm2KMy/P9aN820v5tGLs/UUWGNRypzzFy3y2f0VCTuyMUA97wKO0k4IodHUJeSOtJliQZvaVw2beyzUXBh+aNvnpGhJmvO2nDjHg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252239304470.5012344405361; Fri, 20 Jan 2023 14:03:59 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-30-NCUJHA2VMki02Fya58om2w-1; Fri, 20 Jan 2023 17:03:51 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4688929ABA26; Fri, 20 Jan 2023 22:03:37 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2F8964221F; Fri, 20 Jan 2023 22:03:37 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1A4201946A7C; Fri, 20 Jan 2023 22:03:37 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D9BEC1946A7C for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id CF71114171C2; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id ABE7614171C0 for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252238; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=7q+OQcsDsX8ZYRm6/EdEuri0kbO4wPBTvvdFeHFykds=; b=R6WShKJsHITedDE9XrocNHyjQtBeCkZrdNC/pckKzjSBRJNIdYbl+X3MzGcL4FWXtlgRoq Pw8+jF+sBDo3eJRGN7GMw9Roha1ukxm6GElRaEckB8AqtLsDjXPcXojLIpPAxhjkhY5WIP O0AFBrvjsS5crCu/yJO5I+dCSJdKuQk= X-MC-Unique: NCUJHA2VMki02Fya58om2w-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 26/31] schema: add password configuration for ssh disk Date: Fri, 20 Jan 2023 16:03:20 -0600 Message-Id: <20230120220325.1015090-27-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252239790100009 Content-Type: text/plain; charset="utf-8"; x-default="true" Right now, ssh network disks are not usable. There is some basic support in libvirt that is meant to support disk chains that have backing disks located at ssh urls, but there is no real way for a user to configure a ssh-based disk. This commit allows users to configure an ssh disk with password authentication. Implementation will follow. Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 27 ++++++++++++++------------- src/conf/schemas/domaincommon.rng | 23 ++++++++++++++++++++++- 2 files changed, 36 insertions(+), 14 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index f76c7c3d81..f0f3416f29 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -2718,7 +2718,7 @@ paravirtualized driver is specified via the ``disk`` = element. ``network`` The ``protocol`` attribute specifies the protocol to access to the requested image. Possible values are "nbd", "iscsi", "rbd", "sheepdo= g", - "gluster", "vxhs", "nfs", "http", "https", "ftp", ftps", or "tftp". + "gluster", "vxhs", "nfs", "http", "https", "ftp", ftps", "tftp", or = "ssh". =20 For any ``protocol`` other than ``nbd`` an additional attribute ``na= me`` is mandatory to specify which volume/image will be used. @@ -2870,18 +2870,19 @@ paravirtualized driver is specified via the ``disk`= ` element. ``auth`` :since:`Since libvirt 3.9.0` , the ``auth`` element is supported for= a disk ``type`` "network" that is using a ``source`` element with the - ``protocol`` attributes "rbd" or "iscsi". If present, the ``auth`` e= lement - provides the authentication credentials needed to access the source.= It - includes a mandatory attribute ``username``, which identifies the us= ername - to use during authentication, as well as a sub-element ``secret`` wi= th - mandatory attribute ``type``, to tie back to a `libvirt secret - object `__ that holds the actual password or other - credentials (the domain XML intentionally does not expose the passwo= rd, - only the reference to the object that does manage the password). Kno= wn - secret types are "ceph" for Ceph RBD network sources and "iscsi" for= CHAP - authentication of iSCSI targets. Both will require either a ``uuid`` - attribute with the UUID of the secret object or a ``usage`` attribute - matching the key that was specified in the secret object. + ``protocol`` attributes "rbd", "iscsi", or "ssh". If present, the + ``auth`` element provides the authentication credentials needed to a= ccess + the source. It includes a mandatory attribute ``username``, which + identifies the username to use during authentication, as well as a + sub-element ``secret`` with mandatory attribute ``type``, to tie bac= k to + a `libvirt secret object `__ that holds the actual + password or other credentials (the domain XML intentionally does not + expose the password, only the reference to the object that does mana= ge + the password). Known secret types are "ceph" for Ceph RBD network so= urces + and "iscsi" for CHAP authentication of iSCSI targets. Both will requ= ire + either a ``uuid`` attribute with the UUID of the secret object or a + ``usage`` attribute matching the key that was specified in the secret + object. ``encryption`` :since:`Since libvirt 3.9.0` , the ``encryption`` can be a sub-eleme= nt of the ``source`` element for encrypted storage sources. If present, diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index f1068c2272..79e50fd3e3 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2148,6 +2148,27 @@ =20 + + + + + + ssh + + + + + + + + + + + + + + + @@ -2155,7 +2176,6 @@ sheepdog tftp - ssh @@ -2262,6 +2282,7 @@ + --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252252; cv=none; d=zohomail.com; s=zohoarc; b=HyksOkkpo4uNxHOHth7OHcCMCcIuDBDxhMfnK8Mnn0BhFV7fBmlum8S6DJGgzQIFUlRRQxBgmPqgFY84pvd58YcNAs8PtlswAiS8Ff8p43SpyiNpWJQGJkwPAm9noLahSRdrs0pHO2CcGImg5RhIQcagJybShI0HfhP3EScqrJs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252252; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=J6uWKpGzT6MLllopOUfmJFcMEXbT9pw6v7NBz8VSDEA=; b=AB8DVAEvV69yWqEZfTWBxKepUaQYmXRMYQgjNcJX2qJZ/Yn6wZPKNBVBRiK3S6i5jNjjZjYnfqweTAqz1Zq8k126vqYLlyXPpXJ8NUV+2jP+DE2k9eSGIV5NtAzw9FD6RywV+dyMN7Ib2RQ/900IFeEIhiRGN8EOC7PBzkNbnBU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252252421128.05240889696427; Fri, 20 Jan 2023 14:04:12 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-327-qTii0Ds9M7-hvPMnRi-rUw-1; Fri, 20 Jan 2023 17:03:54 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4D0741C270C1; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 225691121330; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C03141946A7D; Fri, 20 Jan 2023 22:03:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 20E061947040 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 116A114171CA; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DF95F14171C0 for ; Fri, 20 Jan 2023 22:03:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252251; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=J6uWKpGzT6MLllopOUfmJFcMEXbT9pw6v7NBz8VSDEA=; b=Tslu4S/u8qjjGAF+Otg2HsuEHPf9o60VyGnogBccgmOPhDZ4f30dmOLXRUphVgCPGxGFQi Kz0JeWS/SH3+VeTEX7gsF13mWXeb8Dk3y/vB+2/iTJTfazd0SjTB/VD/MPTycluNTFCfCl B7L9+kBaIYr5gDdkSO20HeUTE4qjUY0= X-MC-Unique: qTii0Ds9M7-hvPMnRi-rUw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 27/31] qemu: implement password auth for ssh disks with nbdkit Date: Fri, 20 Jan 2023 16:03:21 -0600 Message-Id: <20230120220325.1015090-28-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252253905100004 Content-Type: text/plain; charset="utf-8"; x-default="true" For ssh disks that are served by nbdkit, lookup the password from the configured secret and securely pass it to the nbdkit process using fd passing. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_nbdkit.c | 87 ++++++++++--------- .../disk-network-ssh-password.args.disk0 | 8 ++ ...k-network-ssh-password.args.disk0.pipe.778 | 1 + .../disk-network-ssh.args.disk1 | 8 ++ .../disk-network-ssh.args.disk1.pipe.778 | 1 + tests/qemunbdkittest.c | 1 + ...sk-network-ssh-password.x86_64-latest.args | 36 ++++++++ .../disk-network-ssh-password.xml | 34 ++++++++ tests/qemuxml2argvtest.c | 1 + 9 files changed, 138 insertions(+), 39 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.dis= k0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.dis= k0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.7= 78 create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.x86_64= -latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.xml diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 98f8b70391..39afad106a 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -999,6 +999,46 @@ qemuNbdkitCommandPassDataByPipe(virCommand *cmd, } =20 =20 +static int +qemuNbdkitProcessBuildCommandAuth(virStorageAuthDef *authdef, + virCommand *cmd) +{ + g_autoptr(virConnect) conn =3D NULL; + g_autofree uint8_t *secret =3D NULL; + size_t secretlen =3D 0; + int secrettype; + + if (!authdef) + return 0; + + if ((secrettype =3D virSecretUsageTypeFromString(authdef->secrettype))= < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("invalid secret type %s"), + authdef->secrettype); + return -1; + } + + conn =3D virGetConnectSecret(); + if (virSecretGetSecretString(conn, + &authdef->seclookupdef, + secrettype, + &secret, + &secretlen) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to get auth secret for storage")); + return -1; + } + + virCommandAddArgPair(cmd, "user", authdef->username); + + if (qemuNbdkitCommandPassDataByPipe(cmd, "password", + &secret, secretlen) < 0) + return -1; + + return 0; +} + + static int qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, virCommand *cmd) @@ -1020,37 +1060,8 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess = *proc, } virCommandAddArgPair(cmd, "url", uristring); =20 - if (proc->source->auth) { - g_autoptr(virConnect) conn =3D virGetConnectSecret(); - g_autofree uint8_t *secret =3D NULL; - size_t secretlen =3D 0; - int secrettype; - virStorageAuthDef *authdef =3D proc->source->auth; - - virCommandAddArgPair(cmd, "user", - proc->source->auth->username); - - if ((secrettype =3D virSecretUsageTypeFromString(proc->source->aut= h->secrettype)) < 0) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("invalid secret type %s"), - proc->source->auth->secrettype); - return -1; - } - - if (virSecretGetSecretString(conn, - &authdef->seclookupdef, - secrettype, - &secret, - &secretlen) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("failed to get auth secret for storage")); - return -1; - } - - if (qemuNbdkitCommandPassDataByPipe(cmd, "password", - &secret, secretlen) < 0) - return -1; - } + if (proc->source->auth && qemuNbdkitProcessBuildCommandAuth(proc->sour= ce->auth, cmd) < 0) + return -1; =20 /* Create a pipe to send the cookies to the nbdkit process. */ if (proc->source->ncookies) { @@ -1079,7 +1090,6 @@ static int qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, virCommand *cmd) { - const char *user =3D NULL; virStorageNetHostDef *host =3D &proc->source->hosts[0]; g_autofree char *portstr =3D g_strdup_printf("%u", host->port); =20 @@ -1090,13 +1100,12 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess = *proc, virCommandAddArgPair(cmd, "port", portstr); virCommandAddArgPair(cmd, "path", proc->source->path); =20 - if (proc->source->auth) - user =3D proc->source->auth->username; - else if (proc->source->ssh_user) - user =3D proc->source->ssh_user; - - if (user) - virCommandAddArgPair(cmd, "user", user); + if (proc->source->auth) { + if (qemuNbdkitProcessBuildCommandAuth(proc->source->auth, cmd) < 0) + return -1; + } else if (proc->source->ssh_user) { + virCommandAddArgPair(cmd, "user", proc->source->ssh_user); + } =20 if (proc->source->ssh_host_key_check_disabled) virCommandAddArgPair(cmd, "verify-remote-host", "false"); diff --git a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 b/te= sts/qemunbdkitdata/disk-network-ssh-password.args.disk0 new file mode 100644 index 0000000000..30711f7f07 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground ssh \ +host=3Dexample.org \ +port=3D2222 \ +path=3Dtest2.img \ +user=3Dtestuser \ +password=3D-777 diff --git a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe= .778 b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk1 b/tests/qemun= bdkitdata/disk-network-ssh.args.disk1 new file mode 100644 index 0000000000..9a8a16c8d5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk1 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground ssh \ +host=3Dexample.org \ +port=3D2222 \ +path=3Dtest2.img \ +user=3Dtestuser \ +password=3D-777 diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 b/te= sts/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c index 5606e155eb..492077e56e 100644 --- a/tests/qemunbdkittest.c +++ b/tests/qemunbdkittest.c @@ -291,6 +291,7 @@ mymain(void) DO_TEST("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT_CAPS_PL= UGIN_CURL); DO_TEST("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + DO_TEST("disk-network-ssh-password", QEMU_NBDKIT_CAPS_PLUGIN_SSH); =20 qemuTestDriverFree(&driver); =20 diff --git a/tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest= .args b/tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args new file mode 100644 index 0000000000..e22ba095b1 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args @@ -0,0 +1,36 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram \ +-accel kvm \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/lib/domain= --1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-sto= rage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-network-ssh-password.xml b/tests/q= emuxml2argvdata/disk-network-ssh-password.xml new file mode 100644 index 0000000000..266acb761f --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-password.xml @@ -0,0 +1,34 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + + + + + + + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 59f416ef72..6e610185b5 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1319,6 +1319,7 @@ mymain(void) DO_TEST_CAPS_LATEST_NBDKIT("disk-network-http-nbdkit", QEMU_NBDKIT_CAP= S_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-network-ssh"); DO_TEST_CAPS_LATEST_NBDKIT("disk-network-ssh-nbdkit", QEMU_NBDKIT_CAPS= _PLUGIN_SSH); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-ssh-password", QEMU_NBDKIT_CA= PS_PLUGIN_SSH); driver.config->vxhsTLS =3D 0; VIR_FREE(driver.config->vxhsTLSx509certdir); DO_TEST_CAPS_LATEST("disk-no-boot"); --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252382; cv=none; d=zohomail.com; s=zohoarc; b=csESfX7MZ8jeu+RiX6DbnJIwzwuF0SCNTDvEnrHVR+rR/judVNSsNj3Js4mQpXo3xnmXBxWuQxEw38sJigJN6KkBb84hS8HQ4rw5g1GB+wN3n149L+urycSDH/QdGl494DaSSBZFWQ5RZsEd65dZepCTzOrqRR9AHY8MSxOZZ+U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252382; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VDVzDwfZVTVkhbWAhVfj9FdSt909VTNFPiy0PU/z+kc=; b=fyBwJ6esDM+nS5yaDXgtLPKfCjSUeiQV1HUCWRF3hY8+FWIhHmNUMzXXjH/jh0ryoN7+pBb+HBSoPoKfG2VeyZGyS6jCnQeYzfVMf6hp0r3MHZVBW4dXXLEfBcn1W2PSCiBiowvqltbV0TbyhaXVwuVw16pJ2IN9vqeb/5iyLlQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1674252382368905.6057446161781; Fri, 20 Jan 2023 14:06:22 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-516-HGxC6btuO2SOxNwxKKxtmA-1; Fri, 20 Jan 2023 17:03:51 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8778718A6471; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6A5522166B33; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 5C06919465B6; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 50A921946A6D for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 43AB814171C0; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 22E0014171C1 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252381; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=VDVzDwfZVTVkhbWAhVfj9FdSt909VTNFPiy0PU/z+kc=; b=h2lP3UEkhgO8j8skgo0wcjVQgK7Qv8Q3U027oJa3ns/WCv+9ge2zNz/8xUd76rcmLTpoAK ryVQDQbcygq2w2Rnf/mec2Id3yLVsr0tZ/sug4eiIChl3+peffeqNo2nco+gc5mXfT1BE0 xzCgkN+c3uSMYhMQwPlVS3/pkXSkWMQ= X-MC-Unique: HGxC6btuO2SOxNwxKKxtmA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 28/31] schema: add configuration for host verification of ssh disks Date: Fri, 20 Jan 2023 16:03:22 -0600 Message-Id: <20230120220325.1015090-29-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252383905100001 Content-Type: text/plain; charset="utf-8"; x-default="true" In order to make ssh disks usable, we need to be able to validate a remote host. To do this, add a xml element for ssh disks to allow the user to specify a location for a file that contains known host keys. Implementation to follow. Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 6 ++++++ src/conf/schemas/domaincommon.rng | 11 +++++++++++ 2 files changed, 17 insertions(+) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index f0f3416f29..2a4d19dcd3 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -2953,6 +2953,12 @@ paravirtualized driver is specified via the ``disk``= element. If the reconnect feature is enabled, accepts ``yes`` and ``no`` ``timeout`` The amount of seconds after which hypervisor tries to reconnect. + ``knownHosts`` + For storage accessed via the ``ssh`` protocol, this element configur= es a + path to a file containing a list of known ssh hosts to be used to ve= rify + the remote host. The location of the file is specified via the ``pat= h`` + attribute. + :since:`Since 9.1.0` =20 =20 For a "file" or "volume" disk type which represents a cdrom or floppy (= the diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index 79e50fd3e3..a632e04c3b 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2148,6 +2148,14 @@ =20 + + + + + + + + @@ -2163,6 +2171,9 @@ + + + --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252308; cv=none; d=zohomail.com; s=zohoarc; b=I5iDvYAva6SruPdI03mLXAOnl/OUULx1MmLNbtzoV/FFzEmC8MVMLmDURnD/8O08oV5v5xabALivpfTTboQawaPTarBdkqBdYBarHJdZGfLhyFE3a8FAjPNzZJAEsK8TyAarO5oTBZxRgfBN1JXRYc6D2skbEi11+ZQ7nG0c2QM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252308; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZtK2ZUwAUPNXJNeJI49DvlG896PlhOPgeA4nztWoQ5k=; b=NHa6K19OIzp7BkMHOSP9pVw5p9APmJJkH+ZtME3XLXf2II5yv4g5nL/ybQ5vevi8TBYwGzSN6mB0CMPEM+38ThsAPGBvDY5Lb5XQkMOo3QkYYNs6J6hnvUDwsFUs9M+BksVrnITva0ycy7NQYEFxNFsSfMHTghgPFAyTjix9YJ8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1674252308770715.9441992402508; Fri, 20 Jan 2023 14:05:08 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-512-9hMR7WYPNqCGKBMKTTHgNA-1; Fri, 20 Jan 2023 17:03:52 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AD49287A9EC; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8DF4439D92; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1483D1946A78; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 9964D1947040 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 7F70B14171C1; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5B12614171C0 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252307; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ZtK2ZUwAUPNXJNeJI49DvlG896PlhOPgeA4nztWoQ5k=; b=PPT89mmpr1FB/2HoTbUWTHaJzNl9olzVUatIUcXvor3xEMjCPnx2Ly2NqKHm8WGvV/WgLF pgWETiw7iw/V/XVI8uG6j3jOT/AP8q47i5ilqb2aAnAo7uTKqeWAzYEavCe5PgxFhhVhxB DNdTNKPJgFQCNKd0zgi+YYx70P6WcfY= X-MC-Unique: 9hMR7WYPNqCGKBMKTTHgNA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 29/31] qemu: implement knownHosts for ssh disks with nbdkit Date: Fri, 20 Jan 2023 16:03:23 -0600 Message-Id: <20230120220325.1015090-30-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252310124100001 Content-Type: text/plain; charset="utf-8"; x-default="true" For ssh disks that are served by nbdkit, use the configured value for knownHosts and pass it to the nbdkit process. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c | 8 ++++++++ src/conf/storage_source_conf.c | 1 + src/conf/storage_source_conf.h | 2 ++ src/qemu/qemu_nbdkit.c | 3 +++ tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 | 3 ++- tests/qemunbdkitdata/disk-network-ssh.args.disk0 | 3 ++- tests/qemuxml2argvdata/disk-network-ssh-password.xml | 1 + tests/qemuxml2argvdata/disk-network-ssh.xml | 1 + 8 files changed, 20 insertions(+), 2 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 45965fa0fa..f383bb8aaa 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7143,6 +7143,11 @@ virDomainDiskSourceNetworkParse(xmlNodePtr node, return -1; } } + if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_SSH && + (tmpnode =3D virXPathNode("./knownHosts", ctxt))) { + if (!(src->ssh_known_hosts_file =3D virXMLPropStringRequired(tmpno= de, "path"))) + return -1; + } =20 return 0; } @@ -21940,6 +21945,9 @@ virDomainDiskSourceFormatNetwork(virBuffer *attrBuf, =20 if (src->timeout) virBufferAsprintf(childBuf, "\n", src->= timeout); + + if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_SSH && src->ssh_know= n_hosts_file) + virBufferAsprintf(childBuf, "\n", src->ss= h_known_hosts_file); } =20 =20 diff --git a/src/conf/storage_source_conf.c b/src/conf/storage_source_conf.c index cecd7e811e..5d60c46cfc 100644 --- a/src/conf/storage_source_conf.c +++ b/src/conf/storage_source_conf.c @@ -1167,6 +1167,7 @@ virStorageSourceClear(virStorageSource *def) VIR_FREE(def->tlsHostname); =20 VIR_FREE(def->ssh_user); + VIR_FREE(def->ssh_known_hosts_file); =20 VIR_FREE(def->nfs_user); VIR_FREE(def->nfs_group); diff --git a/src/conf/storage_source_conf.h b/src/conf/storage_source_conf.h index 14a6825d54..a2d8b1f8bd 100644 --- a/src/conf/storage_source_conf.h +++ b/src/conf/storage_source_conf.h @@ -405,6 +405,8 @@ struct _virStorageSource { /* these must not be used apart from formatting the output JSON in the= qemu driver */ char *ssh_user; bool ssh_host_key_check_disabled; + /* additional ssh variables */ + char *ssh_known_hosts_file; =20 /* nfs_user and nfs_group store the strings passed in by the user for = NFS params. * nfs_uid and nfs_gid represent the converted/looked up ID numbers wh= ich are used diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 39afad106a..4b10df6b7c 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -1110,6 +1110,9 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *p= roc, if (proc->source->ssh_host_key_check_disabled) virCommandAddArgPair(cmd, "verify-remote-host", "false"); =20 + if (proc->source->ssh_known_hosts_file) + virCommandAddArgPair(cmd, "known-hosts", proc->source->ssh_known_h= osts_file); + return 0; } =20 diff --git a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 b/te= sts/qemunbdkitdata/disk-network-ssh-password.args.disk0 index 30711f7f07..ee2d7c3343 100644 --- a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 +++ b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 @@ -5,4 +5,5 @@ host=3Dexample.org \ port=3D2222 \ path=3Dtest2.img \ user=3Dtestuser \ -password=3D-777 +password=3D-777 \ +known-hosts=3D/path/to/knownhosts diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk0 b/tests/qemun= bdkitdata/disk-network-ssh.args.disk0 index c04dc8bb03..481b218936 100644 --- a/tests/qemunbdkitdata/disk-network-ssh.args.disk0 +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk0 @@ -3,4 +3,5 @@ nbdkit \ --foreground ssh \ host=3Dexample.org \ port=3D2222 \ -path=3Dtest.img +path=3Dtest.img \ +known-hosts=3D/path/to/ssh_known_hosts diff --git a/tests/qemuxml2argvdata/disk-network-ssh-password.xml b/tests/q= emuxml2argvdata/disk-network-ssh-password.xml index 266acb761f..bdb4cf6e35 100644 --- a/tests/qemuxml2argvdata/disk-network-ssh-password.xml +++ b/tests/qemuxml2argvdata/disk-network-ssh-password.xml @@ -22,6 +22,7 @@ + diff --git a/tests/qemuxml2argvdata/disk-network-ssh.xml b/tests/qemuxml2ar= gvdata/disk-network-ssh.xml index 355add4fea..a3aeca0c99 100644 --- a/tests/qemuxml2argvdata/disk-network-ssh.xml +++ b/tests/qemuxml2argvdata/disk-network-ssh.xml @@ -19,6 +19,7 @@ + --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252244; cv=none; d=zohomail.com; s=zohoarc; b=bknwqmRgmNodpxNaUaYVw2QlQD5RiTnm4PgGJK24HefEW34S5OdFhn20B/9+rKYgMpEf9wb5oaFIHkiFICN4k2jPUYuaqdxSpXiqmPSvGx9gtwZvpsNS5UAay9rUYcwXE77v7+2nuKAQ6Aj1Fv4+hNCxAkha09oMZXRyu4tmYio= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252244; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UK2QE3PFyLtLzA01LCMt9QcAAr0PK5khz4MgV0APSAE=; b=m4uVM6jbCKAQwLn/y6tzW0zCw0sYjxZ3TSS557V+3W2XZTPYMyoli+SOPI3cmbolIRS1CMtqSD/dHl30EwG65tl3sgC0Foa0mbTsQ1KTO2siGcpihFcyUjvwrjrF/ucIK2fo6yCU0gleQX8tT2QSqEz1w0Ld5jbjA5AUnwRc5XE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 167425224497610.756020383151963; Fri, 20 Jan 2023 14:04:04 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-642-gKveiqKAO8S1iQ7cDORd4A-1; Fri, 20 Jan 2023 17:03:54 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D71333C22755; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id B7A7E140EBF6; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 2F0841946A79; Fri, 20 Jan 2023 22:03:35 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BF9A419465B6 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B497E14171C3; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 902B314171C0 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252243; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=UK2QE3PFyLtLzA01LCMt9QcAAr0PK5khz4MgV0APSAE=; b=CvwyZH/sujcWPZUsK1cKpu2WjFeBPNNPVowzbvVe1L1fRXdpUPHs3dYiFLn9FoYMyteDPt h5gm9TNd04nzcmpO+GYdiJccNeIRnW8WvfmuRDOdCjN/2EUZHmRkstUzuTaYQ8gPAWNOb2 QiTySdA8EO8PUhj3kikdR9DSASEBbmY= X-MC-Unique: gKveiqKAO8S1iQ7cDORd4A-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 30/31] schema: add keyfile configuration for ssh disks Date: Fri, 20 Jan 2023 16:03:24 -0600 Message-Id: <20230120220325.1015090-31-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252245818100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Authenticating via key file to an ssh server is often preferable to logging in via password. In order to support this functionality add a new xml element for ssh disks that allows the user to specify a keyfile, username and optional ssh-agent socket location. Example configuration: ... ... Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 8 ++++++++ src/conf/schemas/domaincommon.rng | 22 +++++++++++++++++++++- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 2a4d19dcd3..3952da79d5 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -2945,6 +2945,14 @@ paravirtualized driver is specified via the ``disk``= element. of these attributes is omitted, then that field is assumed to be the default value for the current system. If both ``user`` and ``group`` are intended to be default, then the entire element may be omitted. + + When using an ``ssh`` protocol, this element is used to enable + authentication via ssh keys. In this configuration, the element has = three + attributes. The ``username`` attribute specifies the name of the use= r on + the remote server. A path to an ssh key can be specified in the + ``keyfile`` attribute. If the ssh key is password-protected, the key= can + be added to an ssh-agent and the path to the ssh-agent socket can be + specified in the ``agentsock`` attribute. ``reconnect`` For disk type ``vhostuser`` configures reconnect timeout if the conn= ection is lost. It has two mandatory attributes: diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index a632e04c3b..be212ae75b 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2156,6 +2156,22 @@ =20 + + + + + + + + + + + + + + + + @@ -2175,11 +2191,15 @@ - + + + + + --=20 2.39.0 From nobody Sun May 19 05:00:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1674252312; cv=none; d=zohomail.com; s=zohoarc; b=M05aQKDx5zlPK5qTUhSpzup6vTAI+KjLPlvzChEwWI7HYKGAiLRRdm8cXHtrGP/FW7/ZLUPm7ptp17aAtdc6sNJXs5//ev3COIU8a+f432UunwdzAmr+W7MJbkCNpBrby0jYqauqMs/p94y0kkS+gs6fXShcGQskDaZD/U86JyI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674252312; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NOI42/KsjxdMOck3utO4dDpC1xSoabbeufLmW/6MxHI=; b=kOgbSyIUnHZri5MvqoEgedYz9X14oyueI1E+RGcLj84cPKEAQTO+nURqWvv5vvARpdx9rUQK5jNPdAElCsrLbConUzz3Wz84iuXKyDuSXsblsJXRVE0Sl5Hrte/WLrgMkPs7agsuBur3Eb8LkB6DqPit/YPK+j88bIdF+ZICq38= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 16742523120011011.6733094902124; Fri, 20 Jan 2023 14:05:12 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-166-SwoG9hRbPg26JCboAcVV4w-1; Fri, 20 Jan 2023 17:04:00 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F342281B54D; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE6162166B2B; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D3D1D19465B6; Fri, 20 Jan 2023 22:03:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id F2B9A1946A77 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id E515F14171C3; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.18.185]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C43D714171C0 for ; Fri, 20 Jan 2023 22:03:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674252311; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=NOI42/KsjxdMOck3utO4dDpC1xSoabbeufLmW/6MxHI=; b=IQvu6dHbeNyiYkqvbhcRXa3f7FRU4CJycvYW0zEpfDJ6OmSZcfez7nlAxIi+ao2ltEiz+x RvqZzzgAuQiBpJgP7hmWZsLA01T51FaCGJD4fRoVjcXBXcMfw9VvGpri/V9UFdhUgV7KUI 8V/p/q4u97mok7sps1NyT4SkwnyoRRo= X-MC-Unique: SwoG9hRbPg26JCboAcVV4w-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v4 31/31] qemu: implement keyfile auth for ssh disk with nbdkit Date: Fri, 20 Jan 2023 16:03:25 -0600 Message-Id: <20230120220325.1015090-32-jjongsma@redhat.com> In-Reply-To: <20230120220325.1015090-1-jjongsma@redhat.com> References: <20230120220325.1015090-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1674252314179100005 Content-Type: text/plain; charset="utf-8"; x-default="true" For ssh disks that are served by nbdkit, we can support logging in with an ssh key file. Pass the path to the configured key file and the username to the nbdkit process. The key file may be password protected, and libvirt cannot prompt the user for a password to unlock it. But if the adminstrator adds this key to an ssh agent, they can configure the disk with the path to the unix socket for the ssh agent so libvirt can pass this socket path to nbdkit and we can make use of these keys. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c | 36 +++++++++++++++---- src/conf/storage_source_conf.c | 2 ++ src/conf/storage_source_conf.h | 6 ++-- src/qemu/qemu_nbdkit.c | 11 ++++-- .../disk-network-ssh-key.args.disk0 | 10 ++++++ .../disk-network-ssh.args.disk2 | 9 +++++ tests/qemunbdkittest.c | 1 + .../qemuxml2argvdata/disk-network-ssh-key.xml | 33 +++++++++++++++++ 8 files changed, 97 insertions(+), 11 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk2 create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-key.xml diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index f383bb8aaa..e901afb1e2 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7143,10 +7143,21 @@ virDomainDiskSourceNetworkParse(xmlNodePtr node, return -1; } } - if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_SSH && - (tmpnode =3D virXPathNode("./knownHosts", ctxt))) { - if (!(src->ssh_known_hosts_file =3D virXMLPropStringRequired(tmpno= de, "path"))) - return -1; + if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_SSH) { + if ((tmpnode =3D virXPathNode("./knownHosts", ctxt))) { + if (!(src->ssh_known_hosts_file =3D virXMLPropStringRequired(t= mpnode, "path"))) + return -1; + } + if ((tmpnode =3D virXPathNode("./identity", ctxt))) { + if (!(src->ssh_keyfile =3D virXMLPropStringRequired(tmpnode, "= keyfile"))) + return -1; + + if (!(src->ssh_user =3D virXMLPropStringRequired(tmpnode, "use= rname"))) + return -1; + + /* optional ssh-agent socket location */ + src->ssh_agent =3D virXMLPropString(tmpnode, "agentsock"); + } } =20 return 0; @@ -21946,8 +21957,21 @@ virDomainDiskSourceFormatNetwork(virBuffer *attrBu= f, if (src->timeout) virBufferAsprintf(childBuf, "\n", src->= timeout); =20 - if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_SSH && src->ssh_know= n_hosts_file) - virBufferAsprintf(childBuf, "\n", src->ss= h_known_hosts_file); + if (src->protocol =3D=3D VIR_STORAGE_NET_PROTOCOL_SSH) { + if (src->ssh_known_hosts_file) + virBufferAsprintf(childBuf, "\n", src= ->ssh_known_hosts_file); + if (src->ssh_keyfile) { + virBufferAddLit(childBuf, "ssh_ke= yfile); + if (src->ssh_user) + virBufferEscapeString(childBuf, " username=3D'%s'", src->s= sh_user); + if (src->ssh_agent) + virBufferEscapeString(childBuf, " agentsock=3D'%s'", src->= ssh_agent); + + virBufferAddLit(childBuf, "/>\n"); + } + } } =20 =20 diff --git a/src/conf/storage_source_conf.c b/src/conf/storage_source_conf.c index 5d60c46cfc..4b8397420b 100644 --- a/src/conf/storage_source_conf.c +++ b/src/conf/storage_source_conf.c @@ -1168,6 +1168,8 @@ virStorageSourceClear(virStorageSource *def) =20 VIR_FREE(def->ssh_user); VIR_FREE(def->ssh_known_hosts_file); + VIR_FREE(def->ssh_keyfile); + VIR_FREE(def->ssh_agent); =20 VIR_FREE(def->nfs_user); VIR_FREE(def->nfs_group); diff --git a/src/conf/storage_source_conf.h b/src/conf/storage_source_conf.h index a2d8b1f8bd..ed4deaf58c 100644 --- a/src/conf/storage_source_conf.h +++ b/src/conf/storage_source_conf.h @@ -401,12 +401,12 @@ struct _virStorageSource { =20 bool hostcdrom; /* backing device is a cdrom */ =20 - /* passthrough variables for the ssh driver which we don't handle prop= erly */ - /* these must not be used apart from formatting the output JSON in the= qemu driver */ + /* ssh variables */ char *ssh_user; bool ssh_host_key_check_disabled; - /* additional ssh variables */ char *ssh_known_hosts_file; + char *ssh_keyfile; + char *ssh_agent; =20 /* nfs_user and nfs_group store the strings passed in by the user for = NFS params. * nfs_uid and nfs_gid represent the converted/looked up ID numbers wh= ich are used diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 4b10df6b7c..dfdd95fca1 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -1103,10 +1103,17 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess = *proc, if (proc->source->auth) { if (qemuNbdkitProcessBuildCommandAuth(proc->source->auth, cmd) < 0) return -1; - } else if (proc->source->ssh_user) { - virCommandAddArgPair(cmd, "user", proc->source->ssh_user); + } else { + if (proc->source->ssh_keyfile) + virCommandAddArgPair(cmd, "identity", proc->source->ssh_keyfil= e); + + if (proc->source->ssh_user) + virCommandAddArgPair(cmd, "user", proc->source->ssh_user); } =20 + if (proc->source->ssh_agent) + virCommandAddEnvPair(cmd, "SSH_AUTH_SOCK", proc->source->ssh_agent= ); + if (proc->source->ssh_host_key_check_disabled) virCommandAddArgPair(cmd, "verify-remote-host", "false"); =20 diff --git a/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 b/tests/q= emunbdkitdata/disk-network-ssh-key.args.disk0 new file mode 100644 index 0000000000..9842300f70 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 @@ -0,0 +1,10 @@ +SSH_AUTH_SOCK=3D/path/to/agent/socket \ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground ssh \ +host=3Dexample.org \ +port=3D2222 \ +path=3Dtest2.img \ +identity=3D/path/to/id_rsa \ +user=3Dmyuser \ +known-hosts=3D/path/to/ssh_known_hosts diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk2 b/tests/qemun= bdkitdata/disk-network-ssh.args.disk2 new file mode 100644 index 0000000000..e269a34351 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk2 @@ -0,0 +1,9 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground ssh \ +host=3Dexample.org \ +port=3D2222 \ +path=3Dtest2.img \ +identity=3D/path/to/id_rsa \ +user=3Dmyuser \ +known-hosts=3D/path/to/ssh_known_hosts diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c index 492077e56e..e507df6e42 100644 --- a/tests/qemunbdkittest.c +++ b/tests/qemunbdkittest.c @@ -292,6 +292,7 @@ mymain(void) DO_TEST("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); DO_TEST("disk-network-ssh-password", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + DO_TEST("disk-network-ssh-key", QEMU_NBDKIT_CAPS_PLUGIN_SSH); =20 qemuTestDriverFree(&driver); =20 diff --git a/tests/qemuxml2argvdata/disk-network-ssh-key.xml b/tests/qemuxm= l2argvdata/disk-network-ssh-key.xml new file mode 100644 index 0000000000..e8e62476d4 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-key.xml @@ -0,0 +1,33 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + + + + + + + + + + + + + + + + + + --=20 2.39.0