From nobody Fri May 3 07:56:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1579193816; cv=none; d=zohomail.com; s=zohoarc; b=SQay0oL36h9LhOLQYwZYwnY5X3FmMGPRhQOqVpxwANs1RJSbhsVZcFrNznNHHYRlYl2wTozfOkejpJUgEz+4DE6ePrzeAX6uuq00esLquZN1yOQuT1FuLenJaiktp2IK8aM8CXnHIjvGrPMmcuze1WNq4dP/E7dUVkMsLqMLMzY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579193816; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GnnfZNOyfPP3dIaV+VhbbckFOGdHXsLJRCbF3neKUZo=; b=EafztoObpYlRCFbfUN+OzzBu2ISiaQ2PtWCGyVgsEOcoJNh6M3Vo/SSbJJUAe5DA9K+tdNFp1rfgbDRf8vd41Szld3ttVDaGmXrHC/LAItCvwujW9Gs6LV2xMJPe8Cr94wXlAjgd28/Nod4eMEr0Zj7rzXiV57D6SQeKXpEzWr0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1579193816165127.05263554194755; Thu, 16 Jan 2020 08:56:56 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-342-GaJe4l00NmSiDVRFhNcjBQ-1; Thu, 16 Jan 2020 11:56:53 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2001F18A6EC4; Thu, 16 Jan 2020 16:56:48 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E849819C5B; Thu, 16 Jan 2020 16:56:47 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id BD9FD18218AA; Thu, 16 Jan 2020 16:56:45 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 00GGuigF002304 for ; Thu, 16 Jan 2020 11:56:44 -0500 Received: by smtp.corp.redhat.com (Postfix) id D342F196AE; Thu, 16 Jan 2020 16:56:44 +0000 (UTC) Received: from angien.redhat.com (unknown [10.43.2.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C59819C5B for ; Thu, 16 Jan 2020 16:56:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579193814; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=GnnfZNOyfPP3dIaV+VhbbckFOGdHXsLJRCbF3neKUZo=; b=IpjJQLA4g721Mz8uM3U7FSX3PhqnrnP08vHm1NCLI3zqCBre98IqU52QOL4xpaFzFd4zLJ 182s1SO9gMfBeVZJ4HEFYmnwk4JwdSM6tTlhDkBNmJBDcd77L3fOgetxFTfuihtKDkSLvZ +xpo6uW4U+7S3c5u59tVGI9e+rcn5zs= From: Peter Krempa To: libvir-list@redhat.com Subject: [libvirt RFC 1/2] tests: storage: Add test data for json specified raw image with offset/size Date: Thu, 16 Jan 2020 17:56:39 +0100 Message-Id: <8add2a0abea142c066f2e8577b2434dcb979e363.1579193759.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: GaJe4l00NmSiDVRFhNcjBQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" QEMU allows specifying the offset and size into a raw file to expose a sub-slice of the image to the guest with the raw driver. Libvirt currently doesn't support it but we can add test case for future reference. Signed-off-by: Peter Krempa Tested-by: Richard W.M. Jones --- tests/virstoragetest.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/tests/virstoragetest.c b/tests/virstoragetest.c index 2862758752..58e15c0cc4 100644 --- a/tests/virstoragetest.c +++ b/tests/virstoragetest.c @@ -1559,6 +1559,16 @@ mymain(void) "\n" " \n" "\n"); + TEST_BACKING_PARSE_FULL("json:{ \"file\": { \"driver\": \"raw\"," + "\"offset\": 10752," + "\"size\": 4063232," + "\"file\": { \"driver\": \"= file\"," + "\"filename\": = \"/tmp/testfle\"" + "}" + "}" + "}", + "\n", 0); + #endif /* WITH_YAJL */ cleanup: --=20 2.24.1 From nobody Fri May 3 07:56:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1579193816; cv=none; d=zohomail.com; s=zohoarc; b=GcSGSud/XibQQlsveU96xU09hLy8ZZTyNxGTg/Y7ereECbzOq4kXceWMLZ7hbsKQ8IbZG5uAvfQpoX7PyGaJl9AduxPf3bNZUIYcdo88t7i1GOZ5B7vKlbiqY7QBGOS7mEJYtvqzcVbodhGFuQemkZwyszaeQRlHRP/7WAHhQw4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579193816; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=t7Vjr7fkZDA9yO2dCJrd0hw+meBKS3LLPwnVKFUfar4=; b=ipLPfUVZThrgVCb3tGYRQ6uaM7mpfKEMJZ9Y/qmBB7pPg8UVYAQ3Mm/lRlMIkjKrycHs4wWFQ55+fpVUf4x+4xzYJrEWaA/xUpQpDY7TlEb75HAhfcX22SetiSl92ti/IzAGRWd51HZaTCZsPymFMhMbgx+crcM0d0nqkC/mi0A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1579193816456485.6436871422218; Thu, 16 Jan 2020 08:56:56 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-354-0bztepCGMay0F9diyYUkPg-1; Thu, 16 Jan 2020 11:56:53 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 43875100550E; Thu, 16 Jan 2020 16:56:48 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 127585C1D8; Thu, 16 Jan 2020 16:56:48 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id BF4DB87087; Thu, 16 Jan 2020 16:56:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 00GGujPX002313 for ; Thu, 16 Jan 2020 11:56:45 -0500 Received: by smtp.corp.redhat.com (Postfix) id A6FB0196AE; Thu, 16 Jan 2020 16:56:45 +0000 (UTC) Received: from angien.redhat.com (unknown [10.43.2.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2EA1519C5B for ; Thu, 16 Jan 2020 16:56:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579193815; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=t7Vjr7fkZDA9yO2dCJrd0hw+meBKS3LLPwnVKFUfar4=; b=Cmmuh2ehnzjtHpBuzV3rFV7CWrE7diABZ9avVy18pMMKqFQhQATeEiZydthWkz8+6kssVy TfhgKPvuTKs2iyFeq6hU31EBm/sL4cCgCLbD5mp2U9wxeBXm4SR1WkXV3MjP8jh91efeDO qOPmX8IlhcWVbwA1Fn+shdnk5Gwx3go= From: Peter Krempa To: libvir-list@redhat.com Subject: [libvirt RFC 2/2] qemu: Handle 'offset' and 'size' parameters of 'raw' driver Date: Thu, 16 Jan 2020 17:56:40 +0100 Message-Id: <4b8e0bd263f8f4a63e0c124eda9d21d02fbd55f4.1579193759.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: 0bztepCGMay0F9diyYUkPg-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Prior to using -blockdev if a user created an overlay file with some json properties we'd not put them on the command line at all. This meant that qemu then obeyed them. Now since we do configure all known backing files, we must also pass in all of the parameters. Otherwise we'd configure something else than the user expected. The most prominent one now is the 'offset' and 'size' of the raw driver which we completely ignored. This is an RFC to implement properly the parameters above to see whether upstream accepts it or not. This patch is not meant to be pushed like this but to start discussion. Another option is to detect when we know that the image can't be passed through (we do that partially for images with authentication) and possibly just skip formatting them and let qemu handle it. But that really seems like a step back now that we've put this much effort into blockdev. Obviously it's missing the parser bits and schema and docs but I'd like to know whether the design of the XML makes any sense. The bug was reported by Rich Jones here: https://bugzilla.redhat.com/show_bug.cgi?id=3D1791788 Tested-by: Richard W.M. Jones --- src/conf/domain_conf.c | 19 +++++++++++++++++++ src/qemu/qemu_block.c | 10 ++++------ src/util/virstoragefile.c | 21 +++++++++++++++++---- src/util/virstoragefile.h | 4 ++++ tests/virstoragetest.c | 7 ++++++- 5 files changed, 50 insertions(+), 11 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 2a8a04cacb..62872dfb2f 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -24183,6 +24183,23 @@ virDomainDiskSourceFormatPrivateData(virBufferPtr = buf, } +static void +virDomainDiskSourceFormatOptions(virBufferPtr buf, + virStorageSourcePtr src) +{ + g_auto(virBuffer) attrBuf =3D VIR_BUFFER_INITIALIZER; + + if (src->format =3D=3D VIR_STORAGE_FILE_RAW) { + if (src->offset > 0) + virBufferAsprintf(&attrBuf, " offset=3D'%llu'", src->offset); + if (src->size > 0) + virBufferAsprintf(&attrBuf, " size=3D'%llu'", src->size); + } + + virXMLFormatElement(buf, "options", &attrBuf, NULL); +} + + /** * virDomainDiskSourceFormat: * @buf: output buffer @@ -24253,6 +24270,8 @@ virDomainDiskSourceFormat(virBufferPtr buf, return -1; } + virDomainDiskSourceFormatOptions(&childBuf, src); + if (src->type !=3D VIR_STORAGE_TYPE_NETWORK) virDomainSourceDefFormatSeclabel(&childBuf, src->nseclabels, src->seclabels, flags); diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index eab21bc107..21977a319e 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -1190,14 +1190,12 @@ qemuBlockStorageSourceGetFormatRawProps(virStorageS= ourcePtr src, secretalias =3D srcPriv->encinfo->s.aes.alias; } - /* currently unhandled properties for the 'raw' driver: - * 'offset' - * 'size' - */ - if (virJSONValueObjectAdd(props, "s:driver", driver, - "S:key-secret", secretalias, NULL) < 0) + "S:key-secret", secretalias, + "P:offset", src->offset, + "P:size", src->size, + NULL) < 0) return -1; return 0; diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c index 1397f532fd..22a243a2eb 100644 --- a/src/util/virstoragefile.c +++ b/src/util/virstoragefile.c @@ -2270,6 +2270,8 @@ virStorageSourceCopy(const virStorageSource *src, def->type =3D src->type; def->protocol =3D src->protocol; def->format =3D src->format; + def->offset =3D src->offset; + def->size =3D src->size; def->capacity =3D src->capacity; def->allocation =3D src->allocation; def->has_allocation =3D src->has_allocation; @@ -3511,10 +3513,21 @@ virStorageSourceParseBackingJSONRaw(virStorageSourc= ePtr src, virJSONValuePtr json, int opaque G_GNUC_UNUSED) { - /* There are no interesting attributes in raw driver. - * Treat it as pass-through. - */ - return virStorageSourceParseBackingJSONInternal(src, json); + int rc; + + if ((rc =3D virStorageSourceParseBackingJSONInternal(src, json)) < 0) + return rc; + + if ((virJSONValueObjectHasKey(json, "offset") && + virJSONValueObjectGetNumberUlong(json, "offset", &src->offset) < = 0) || + (virJSONValueObjectHasKey(json, "size") && + virJSONValueObjectGetNumberUlong(json, "size", &src->size) < 0)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("malformed 'size' or 'offset' property of 'raw' d= river")); + return -1; + } + + return rc; } diff --git a/src/util/virstoragefile.h b/src/util/virstoragefile.h index 39e50a989d..5a86f32e95 100644 --- a/src/util/virstoragefile.h +++ b/src/util/virstoragefile.h @@ -286,6 +286,10 @@ struct _virStorageSource { bool nocow; bool sparse; + /* properties of the raw driver. Unspecified if 0. */ + unsigned long long offset; + unsigned long long size; + virStoragePermsPtr perms; virStorageTimestampsPtr timestamps; unsigned long long capacity; /* in bytes, 0 if unknown */ diff --git a/tests/virstoragetest.c b/tests/virstoragetest.c index 58e15c0cc4..d9af2628d0 100644 --- a/tests/virstoragetest.c +++ b/tests/virstoragetest.c @@ -606,6 +606,9 @@ testBackingParse(const void *args) if (!src) return 0; + /* allow tests pass format */ + src->format =3D VIR_STORAGE_FILE_RAW; + if (src && !data->expect) { fprintf(stderr, "parsing of backing store string '%s' should " "have failed\n", data->backing); @@ -1567,7 +1570,9 @@ mymain(void) "}" "}" "}", - "\n", 0); + "\n" + " \n" + "\n", 0); #endif /* WITH_YAJL */ --=20 2.24.1