From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772354; cv=none; d=zohomail.com; s=zohoarc; b=YGljZwutoNFcx8TV7JWbFEh6leDJ9EcJaM5hGq83Xl/4zZbNMb7qaoZpZLF3VDyt1p6VaTZ6NhCIz7NMHtJZ1C3x3UzFghgtO94FyXaW6zll02ckbBhdGT4Sco8BEvdk7goR0Rpx1wHCJCShJxNDPCl/KPVBpGxR6Cen9fwCOuA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772354; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=S6HorQNQM3PX4VmnOgK/Y0BNInrkfPwZYps7CJndr9E=; b=Z9GteiEGSloes7/PBJ1IN798spXeMhKygIE1wIl8Kdv3pE/UAYj7hMUT9Hb+i5hzuV1gNxLlzx6yOD57DjwJV08BHOeAQPhEBfRBoqBxKta4W2VFagXG7k5yib2BR71V5yGjGT4k8IBr/aLbuKk4GZPSZaga4Sm45HhdAyJgOrg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1593772354443303.3767241455437; Fri, 3 Jul 2020 03:32:34 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-21-Pm6ePMKxOKCyv4TuM5i0-w-1; Fri, 03 Jul 2020 06:32:31 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 83518800406; Fri, 3 Jul 2020 10:32:25 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 64CFF79242; Fri, 3 Jul 2020 10:32:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 360CE6C9CB; Fri, 3 Jul 2020 10:32:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT215016957 for ; Fri, 3 Jul 2020 06:29:02 -0400 Received: by smtp.corp.redhat.com (Postfix) id 8D0187BD49; Fri, 3 Jul 2020 10:29:02 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0C7D979242 for ; Fri, 3 Jul 2020 10:29:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772353; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=S6HorQNQM3PX4VmnOgK/Y0BNInrkfPwZYps7CJndr9E=; b=J84jPZZoY9LvA0ron1HraCzNi0u3ax4CEfwN1i6mmTmNQuIqQu3TVY8fZ2T6kkqvFzhOtJ e9mDQtUARlO2rprYOVAl484ofBQZl+1eksAAQW7W73WhROOJHkTeJYbHyC2lfG5VbqjwPz EpWhh5fkkhMzrfsGTAuzLFRfS54UvsA= X-MC-Unique: Pm6ePMKxOKCyv4TuM5i0-w-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 1/9] libvirt-storage: Document volume upload/download stream format Date: Fri, 3 Jul 2020 12:28:42 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" For libvirt, the volume is just a binary blob and it doesn't interpret data on volume upload/download. But as it turns out, this unspoken assumption is not clear to our users. Document it explicitly. Suggested in: https://bugzilla.redhat.com/show_bug.cgi?id=3D1851023#c17 Signed-off-by: Michal Privoznik --- src/libvirt-storage.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/libvirt-storage.c b/src/libvirt-storage.c index a45c8b98c1..8738f6dd14 100644 --- a/src/libvirt-storage.c +++ b/src/libvirt-storage.c @@ -1590,7 +1590,9 @@ virStorageVolCreateXMLFrom(virStoragePoolPtr pool, * * Download the content of the volume as a stream. If @length * is zero, then the remaining contents of the volume after - * @offset will be downloaded. + * @offset will be downloaded. Please note, that the data is + * not interpreted and therefore data received by stream + * client are in the very same format the volume is in. * * If VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM is set in @flags * effective transmission of holes is enabled. This assumes using @@ -1663,7 +1665,9 @@ virStorageVolDownload(virStorageVolPtr vol, * will fail if @offset + @length exceeds the size of the * volume. Otherwise, if @length is non-zero, an error * will be raised if an attempt is made to upload greater - * than @length bytes of data. + * than @length bytes of data. Please note, that the data is + * not interpreted and therefore data sent by stream client + * must be in the very same format the volume is in. * * If VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM is set in @flags * effective transmission of holes is enabled. This assumes using --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772354; cv=none; d=zohomail.com; s=zohoarc; b=VBBjhOT81rS5jrSEoULpVmH1en1DOBoMYKrwbSzwX4GV9+pMJECZsP0v/akptUmKvDg9pO6vSQxNh1hW+bV7dwBwekcgCHT456g5Vm2J2kInVdmuxGunTVm+yMBGwHDy24S6mE+RgiXeK5CjjfMbIntRjiZiLYP0/wUAhsfMdts= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772354; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=s9Snrs4t53DN8m45mB921Odf+rYuenmY3l3hlmKm1H8=; b=AuJ3MtNjKuBRkp6ZEBlVKMmF7Wia3xC2W+UThCmN77oPrAG4kpNdWizjATlpDvoCr1ZvE0EfR26aa6loPPNiTTUqdu19ZokJLKhDu8pIUSUAOYXOvwrql+/kv0MCmw9j6nPrbgLnPVW3oFPd+s9DyhqIKLkYpFKAfiESLhJoB4Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1593772354636349.80698210107323; Fri, 3 Jul 2020 03:32:34 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-342--Qq-vMHONjW49rBNsuyljg-1; Fri, 03 Jul 2020 06:32:31 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2D28318A8225; Fri, 3 Jul 2020 10:32:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0A2B31053B05; Fri, 3 Jul 2020 10:32:26 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id BA8DD1809561; Fri, 3 Jul 2020 10:32:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT3Qf016969 for ; Fri, 3 Jul 2020 06:29:03 -0400 Received: by smtp.corp.redhat.com (Postfix) id 6C9887BD43; Fri, 3 Jul 2020 10:29:03 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id E01B679242 for ; Fri, 3 Jul 2020 10:29:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772353; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=s9Snrs4t53DN8m45mB921Odf+rYuenmY3l3hlmKm1H8=; b=Caw1S3rKPtVtHc74fyUp0qdMzZllZeSra1CpzvKXtQzDbC8Q5XNK2pGSbjHBw8wUwfwX9X jow0GMdRIDr6kuAWdtt345WyqZuei09grNSRo58e+I66TQsDWshN8QONYK7kaUvB0Cv5By XjXw9U5o17K67WMYrWOUi/dZZuhreQM= X-MC-Unique: -Qq-vMHONjW49rBNsuyljg-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 2/9] virstring: Introduce virStringIsNull() Date: Fri, 3 Jul 2020 12:28:43 +0200 Message-Id: <072101003d55f0f76d84f5717ae425f6a825da3b.1593771693.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This function will be used to detect zero buffers (which are going to be interpreted as hole in virStream later). I shamelessly took inspiration from coreutils. Signed-off-by: Michal Privoznik --- src/libvirt_private.syms | 1 + src/util/virstring.c | 40 ++++++++++++++++++++++++++++++++++++++++ src/util/virstring.h | 2 ++ 3 files changed, 43 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index ae0e253ab9..1d80aeb833 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3197,6 +3197,7 @@ virStringHasChars; virStringHasControlChars; virStringHasSuffix; virStringIsEmpty; +virStringIsNull; virStringIsPrintable; virStringListAdd; virStringListAutoFree; diff --git a/src/util/virstring.c b/src/util/virstring.c index e9e792f3bf..7f6871d5ab 100644 --- a/src/util/virstring.c +++ b/src/util/virstring.c @@ -1404,3 +1404,43 @@ int virStringParseYesNo(const char *str, bool *resul= t) =20 return 0; } + + +/** + * virStringIsNull: + * @buf: buffer to check + * @len: the length of the buffer + * + * For given buffer @buf and its size @len determine whether + * it contains only zero bytes or not. + * + * Returns: true if buffer is full of zero bytes, + * false otherwise. + */ +bool virStringIsNull(const char *buf, size_t len) +{ + const char *p =3D buf; + + if (!len) + return true; + + /* Check up to 16 first bytes. memcmp() below uses block of sizeof(lon= g) + * size (in most cases on 64bits is 8 bytes), doing twice the size jus= t to + * be safe. */ + for (;;) { + if (*p) + return false; + + p++; + len--; + + if (!len) + return true; + + if ((len & (2 * sizeof(long))) =3D=3D 0) + break; + } + + /* Now we know first 16 bytes are NUL, memcmp with self. */ + return memcmp(buf, p, len) =3D=3D 0; +} diff --git a/src/util/virstring.h b/src/util/virstring.h index 360c68395c..d0e54358ac 100644 --- a/src/util/virstring.h +++ b/src/util/virstring.h @@ -185,6 +185,8 @@ int virStringParsePort(const char *str, int virStringParseYesNo(const char *str, bool *result) G_GNUC_WARN_UNUSED_RESULT; +bool virStringIsNull(const char *buf, size_t len); + /** * VIR_AUTOSTRINGLIST: * --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772330; cv=none; d=zohomail.com; s=zohoarc; b=DQG6fYjkCKRDjt70JkbKx2QOgaM4fd/sUBgxEeVbltpoHnmAzr+vgBmkqrpdrErmPrkUBKrEOEvhW9TTVQoQgBfXfqg/7bM8lH8azRqDh/eai5PMAzVsgBuhLmlGKUCCNanSYMb2iTzjPLL9AIG9RelwR3MC8O2KVlHP6sLbyz8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772330; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/pvhm1s87vc/iYhopuxeQDEf85bjKWtbuvNobhYPiAQ=; b=GJ7oXAD45xDM6hV6cLgDWJuBYbnXeeCb1BRNim7HJc5xeflun+WJVh1P5ukV0sH36UZAm1sEADOh5MQlqdJAfwdO2zrGx/mNoIudsA2E30b7wBbKL29qWiMck8UmO6L65u1gMIBALwdh8/qoTwbv4ULVV31v0IDqyt8SEUtVnt8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1593772330612862.3329850148357; Fri, 3 Jul 2020 03:32:10 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-275-qYiStJR9NxanLz8pTqLXTQ-1; Fri, 03 Jul 2020 06:32:07 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 87A2D64AD0; Fri, 3 Jul 2020 10:32:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 565FA79242; Fri, 3 Jul 2020 10:32:00 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E20E06C9C7; Fri, 3 Jul 2020 10:31:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT4Yn016974 for ; Fri, 3 Jul 2020 06:29:04 -0400 Received: by smtp.corp.redhat.com (Postfix) id 4C72D7BD43; Fri, 3 Jul 2020 10:29:04 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id BFE0879242 for ; Fri, 3 Jul 2020 10:29:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772329; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=/pvhm1s87vc/iYhopuxeQDEf85bjKWtbuvNobhYPiAQ=; b=Lf4Wz2+oV6lkVuUmJ7MegCp4xu1Ll3pw6J6Q9nbqSPlcnG0P6LSh7CRDxawt0VD3Y+1fyJ in8DOo4qc0f8GJJV+ey0xrnMmFCp4SIsFot01005FUTWxzckGC3HG+E2EJjATMTznbrshQ Ez5O69vTJSAqn9m4AxYrnyVWcHN7iFY= X-MC-Unique: qYiStJR9NxanLz8pTqLXTQ-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 3/9] virfile: Introduce virFileInDataDetectZeroes() Date: Fri, 3 Jul 2020 12:28:44 +0200 Message-Id: <5e40db80ed2e4e1954955b6571748e02075319ef.1593771693.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" For given file descriptor determine if the current position it is in plus 1MiB (arbitrary chosen value) consists solely from zero bytes or not. This is a block device friendly version of virFileInData(). Signed-off-by: Michal Privoznik --- src/libvirt_private.syms | 1 + src/util/virfile.c | 67 ++++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 4 +++ 3 files changed, 72 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 1d80aeb833..6b5a751788 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2070,6 +2070,7 @@ virFileGetMountSubtree; virFileGetXAttr; virFileGetXAttrQuiet; virFileInData; +virFileInDataDetectZeroes; virFileIsCDROM; virFileIsDir; virFileIsExecutable; diff --git a/src/util/virfile.c b/src/util/virfile.c index c034df5931..18b5096b88 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4064,6 +4064,73 @@ virFileInData(int fd G_GNUC_UNUSED, #endif /* !HAVE_DECL_SEEK_HOLE */ =20 =20 +#define DETECT_ZEORES_BLOCK_SIZE (1 * 1024 * 1024) + +/* + * virFileInDataDetectZeroes: + * @fd: file to check + * @inData: true if current position in the @fd is in data section + * @length: amount of bytes until the end of the current section + * + * This behaves exactly like virFileInData() except it doesn't use SEEK_DA= TA + * and SEEK_HOLE rather than a buffer into which it reads data from @fd and + * detects if the buffer is full of zeroes. Therefore, it is safe to use on + * special files (e.g. block devices). On the other hand it sees only + * DETECT_ZEORES_BLOCK_SIZE ahead. + * + * Returns: 0 on success, + * -1 otherwise. + */ +int +virFileInDataDetectZeroes(int fd, + int *inData, + long long *length) +{ + const size_t buflen =3D DETECT_ZEORES_BLOCK_SIZE; + g_autofree char *buf =3D NULL; + off_t cur; + ssize_t r; + int ret =3D -1; + + /* Get current position */ + cur =3D lseek(fd, 0, SEEK_CUR); + if (cur =3D=3D (off_t) -1) { + virReportSystemError(errno, "%s", + _("Unable to get current position in file")); + goto cleanup; + } + + buf =3D g_new0(char, buflen); + + if ((r =3D saferead(fd, buf, buflen)) < 0) { + virReportSystemError(errno, "%s", + _("Unable to read from file")); + goto cleanup; + } + + *inData =3D !virStringIsNull(buf, r); + *length =3D MIN(r, buflen); + ret =3D 0; + + cleanup: + /* At any rate, reposition back to where we started. */ + if (cur !=3D (off_t) -1) { + int theerrno =3D errno; + + if (lseek(fd, cur, SEEK_SET) =3D=3D (off_t) -1) { + virReportSystemError(errno, "%s", + _("unable to restore position in file")); + ret =3D -1; + if (theerrno =3D=3D 0) + theerrno =3D errno; + } + + errno =3D theerrno; + } + return ret; +} + + /** * virFileReadValueInt: * @value: pointer to int to be filled in with the value diff --git a/src/util/virfile.h b/src/util/virfile.h index 7a92364a5c..9a5eade609 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -352,6 +352,10 @@ int virFileInData(int fd, int *inData, long long *length); =20 +int virFileInDataDetectZeroes(int fd, + int *inData, + long long *length); + G_DEFINE_AUTOPTR_CLEANUP_FUNC(virFileWrapperFd, virFileWrapperFdFree); =20 int virFileGetXAttr(const char *path, --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772352; cv=none; d=zohomail.com; s=zohoarc; b=ZaCktSPRqEcQTeQ3XrRTJoQ7gRRJNsqwcusGupVE9huCotYDCJ5Jq029HKOVaXrv1tyWbI6ttVockruXPLNoEPgBUq3Mp3hbTKnviAZOMcVqfD8+gGdLm3+fhuTF3i8CtElfxGpWZn+0MnC5TlbSdFbgde6NT24hX2aMxnEWFpI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772352; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/UW50IMzLK0fDKp9M0omUd4XLUKGp7krwUQWd6pVmJc=; b=mGSytQJ20JzUDLSVifin2E3fEpF7DIFYhobCm9v0f44MVBcwEArKXjhUdDCmy6SG7Sg7UtVHCDwevcEmuaxVnTIDpswehzjMMAdwKiLEsJ7lXTk8WXNrooIWmD7vY2qKtEoRtbDCm1V9h8yQtd3ORBNIzyXomc+cmSSKxT+kLG4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1593772352777353.6227905890448; Fri, 3 Jul 2020 03:32:32 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-414-3WoRUrpgMCSpbRB9feQbVQ-1; Fri, 03 Jul 2020 06:32:29 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 03F6D18A8223; Fri, 3 Jul 2020 10:32:24 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D96F47611C; Fri, 3 Jul 2020 10:32:23 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6D4341809547; Fri, 3 Jul 2020 10:32:23 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT5S8016984 for ; Fri, 3 Jul 2020 06:29:05 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2C74E7BD43; Fri, 3 Jul 2020 10:29:05 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9FC0379242 for ; Fri, 3 Jul 2020 10:29:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772351; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=/UW50IMzLK0fDKp9M0omUd4XLUKGp7krwUQWd6pVmJc=; b=KfTJgqAB+8lJqWUTVx7XJvxC5IWrLx7YMmO/SCaNN/B5jVqwT8BAKv3wGtihsqDkIB3vUC IZF+/y5waU/KEwWxolyzA8xM6AnebEWoIdLNjQqrf+/3ipGo9doQ+Wm28Pw8l4WsdjqZEY XbGZKsCjWZCN3QoHgRH2ait+nuQRaBU= X-MC-Unique: 3WoRUrpgMCSpbRB9feQbVQ-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 4/9] virsh: Pass virshStreamCallbackDataPtr to virshStreamSink() and virshStreamSkip() Date: Fri, 3 Jul 2020 12:28:45 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" These callback will need to know more that the FD they are working on. Pass the structure that is passed to other stream callbacks (e.g. virshStreamSource() or virshStreamSourceSkip()) instead of inventing a new one. Signed-off-by: Michal Privoznik --- tools/virsh-util.c | 10 +++++----- tools/virsh-volume.c | 6 +++++- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/tools/virsh-util.c b/tools/virsh-util.c index 932d6d0849..89f15efd08 100644 --- a/tools/virsh-util.c +++ b/tools/virsh-util.c @@ -146,9 +146,9 @@ virshStreamSink(virStreamPtr st G_GNUC_UNUSED, size_t nbytes, void *opaque) { - int *fd =3D opaque; + virshStreamCallbackDataPtr cbData =3D opaque; =20 - return safewrite(*fd, bytes, nbytes); + return safewrite(cbData->fd, bytes, nbytes); } =20 =20 @@ -186,13 +186,13 @@ virshStreamSkip(virStreamPtr st G_GNUC_UNUSED, long long offset, void *opaque) { - int *fd =3D opaque; + virshStreamCallbackDataPtr cbData =3D opaque; off_t cur; =20 - if ((cur =3D lseek(*fd, offset, SEEK_CUR)) =3D=3D (off_t) -1) + if ((cur =3D lseek(cbData->fd, offset, SEEK_CUR)) =3D=3D (off_t) -1) return -1; =20 - if (ftruncate(*fd, cur) < 0) + if (ftruncate(cbData->fd, cur) < 0) return -1; =20 return 0; diff --git a/tools/virsh-volume.c b/tools/virsh-volume.c index 72394915d8..5cbc2efb7a 100644 --- a/tools/virsh-volume.c +++ b/tools/virsh-volume.c @@ -793,6 +793,7 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) unsigned long long offset =3D 0, length =3D 0; bool created =3D false; virshControlPtr priv =3D ctl->privData; + virshStreamCallbackData cbData; unsigned int flags =3D 0; =20 if (vshCommandOptULongLong(ctl, cmd, "offset", &offset) < 0) @@ -820,6 +821,9 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) created =3D true; } =20 + cbData.ctl =3D ctl; + cbData.fd =3D fd; + if (!(st =3D virStreamNew(priv->conn, 0))) { vshError(ctl, _("cannot create a new stream")); goto cleanup; @@ -830,7 +834,7 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) goto cleanup; } =20 - if (virStreamSparseRecvAll(st, virshStreamSink, virshStreamSkip, &fd) = < 0) { + if (virStreamSparseRecvAll(st, virshStreamSink, virshStreamSkip, &cbDa= ta) < 0) { vshError(ctl, _("cannot receive data from volume %s"), name); goto cleanup; } --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772358; cv=none; d=zohomail.com; s=zohoarc; b=VzFBuh9l+x+WIYCi0ETh6UMMzfzxWmQKb66IeiwgfbrorkwZcnLv67uhClVsMVqACUzZHj3imqnCWrCKrs19BISRFmMUbaFqVjNnY72tMz1sG9cCGHT49fkG3SCaV7/3u55w3iNKQSntqvW8Js7SzYpD61jAczVMpuJHKD00SPQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772358; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=C3g/PCvVDm0zZJyvtMAZa1Vg0ZtgVdgF8mhfLJiEgos=; b=a4SzYIqOve2um87BKlHzo2ulhW7+GmkXEt9zJv5hpLcEOC79oG53ofnsyDkErYWpjHh8f07QK1pC1IxgX6aq7nElJEuRgPQTghoU0WaDZ0Qiw0N9YrJD8Xprd9d8HJF7nFNhAfA6Z7Wgpi94xLAtjSwI5qGuDoYc+sPt0jlOCFg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 159377235812632.4138415265229; Fri, 3 Jul 2020 03:32:38 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-287-ozWmYDjANdSt_1G6G7ntqQ-1; Fri, 03 Jul 2020 06:32:34 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BF88E1083E85; Fri, 3 Jul 2020 10:32:28 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9FCD62DE71; Fri, 3 Jul 2020 10:32:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 721D06C9D1; Fri, 3 Jul 2020 10:32:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT6CT016989 for ; Fri, 3 Jul 2020 06:29:06 -0400 Received: by smtp.corp.redhat.com (Postfix) id 0DA0379242; Fri, 3 Jul 2020 10:29:06 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8120B7BD4A for ; Fri, 3 Jul 2020 10:29:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772356; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=C3g/PCvVDm0zZJyvtMAZa1Vg0ZtgVdgF8mhfLJiEgos=; b=AJJ1yXb0Yzg+/d573SjfRvrrR3N71XCCQ5fBfjnswvrZwYHKZhoQ2OB/zarTS8sFXujL47 bJ9HopS+WIGaX34sHAEQntR0JvecDVehbxtePxUZUX1QDLdmIRaBVSHPzvbjvh0DWVMHgx HKY/ucsAiQ2Iw+i5f+J6NWeZd8ahWu4= X-MC-Unique: ozWmYDjANdSt_1G6G7ntqQ-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 5/9] virsh: Track if vol-upload or vol-download work over a block device Date: Fri, 3 Jul 2020 12:28:46 +0200 Message-Id: <61900eb94ac3fe28d33c4c4feb6201b182a39b44.1593771693.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We can't use virFileInData() with block devices, but we could use new virFileInDataDetectZeroes(). But to decide we need to know if the FD we are reading data from / writing data to is a block device. Store this information in _virshStreamCallbackData. Signed-off-by: Michal Privoznik --- tools/virsh-util.h | 1 + tools/virsh-volume.c | 14 ++++++++++++++ 2 files changed, 15 insertions(+) diff --git a/tools/virsh-util.h b/tools/virsh-util.h index 72653d9735..9ef28cfe0a 100644 --- a/tools/virsh-util.h +++ b/tools/virsh-util.h @@ -72,6 +72,7 @@ typedef virshStreamCallbackData *virshStreamCallbackDataP= tr; struct _virshStreamCallbackData { vshControl *ctl; int fd; + bool isBlock; }; =20 int diff --git a/tools/virsh-volume.c b/tools/virsh-volume.c index 5cbc2efb7a..20823e2d10 100644 --- a/tools/virsh-volume.c +++ b/tools/virsh-volume.c @@ -679,6 +679,7 @@ cmdVolUpload(vshControl *ctl, const vshCmd *cmd) virshControlPtr priv =3D ctl->privData; unsigned int flags =3D 0; virshStreamCallbackData cbData; + struct stat sb; =20 if (vshCommandOptULongLong(ctl, cmd, "offset", &offset) < 0) return false; @@ -697,8 +698,14 @@ cmdVolUpload(vshControl *ctl, const vshCmd *cmd) goto cleanup; } =20 + if (fstat(fd, &sb) < 0) { + vshError(ctl, _("unable to stat %s"), file); + goto cleanup; + } + cbData.ctl =3D ctl; cbData.fd =3D fd; + cbData.isBlock =3D !!S_ISBLK(sb.st_mode); =20 if (vshCommandOptBool(cmd, "sparse")) flags |=3D VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM; @@ -795,6 +802,7 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) virshControlPtr priv =3D ctl->privData; virshStreamCallbackData cbData; unsigned int flags =3D 0; + struct stat sb; =20 if (vshCommandOptULongLong(ctl, cmd, "offset", &offset) < 0) return false; @@ -821,8 +829,14 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) created =3D true; } =20 + if (fstat(fd, &sb) < 0) { + vshError(ctl, _("unable to stat %s"), file); + goto cleanup; + } + cbData.ctl =3D ctl; cbData.fd =3D fd; + cbData.isBlock =3D !!S_ISBLK(sb.st_mode); =20 if (!(st =3D virStreamNew(priv->conn, 0))) { vshError(ctl, _("cannot create a new stream")); --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772360; cv=none; d=zohomail.com; s=zohoarc; b=lRifhnGNVmEsiyAlq5626sbqUAGv8Gfx2Uca/mAv4fbzMNCiqV9LCYuxe/CcwK8A1fhH9AqzBUeUjt5yTv+D0DOzIMK/sEmbZEpb943j8BHRQ/QEzgF1fv9KZzG028MJj8PnW/XxfutmjDfhzfFpR/kB8AoLKibUu2MG3Kfxq+E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772360; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4iaslwF3zmWy+jtD7l+8UjwLEm5V9FlaqxEw3Ms2V0M=; b=b/s+6JJfbRWrWI/8bnMRQxdvp3jbxuIhouSNMgyGIvWFrxx8zKU2sFjzlQGTKDN/axxTEamsk511bTfVDeON62tWwHhSQaBFhCdv3JeXn6QIAGJUwhfBC/ljTlvskQPdC67vRNmy1pUfMNp/MvrB8DWN/AO/IAnMonPQqLCx8qA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1593772360900925.3215955553986; Fri, 3 Jul 2020 03:32:40 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-163-kEkOSQw7O7CtpEgTPyFt7Q-1; Fri, 03 Jul 2020 06:32:37 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0019F800406; Fri, 3 Jul 2020 10:32:32 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D26161053B1B; Fri, 3 Jul 2020 10:32:31 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id A0028180043A; Fri, 3 Jul 2020 10:32:31 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT6PE016997 for ; Fri, 3 Jul 2020 06:29:06 -0400 Received: by smtp.corp.redhat.com (Postfix) id E13D77BD56; Fri, 3 Jul 2020 10:29:06 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60DEE79242 for ; Fri, 3 Jul 2020 10:29:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772359; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=4iaslwF3zmWy+jtD7l+8UjwLEm5V9FlaqxEw3Ms2V0M=; b=KQnIk60R/keux/oL7K8tRzIrBCsBOowjv1gLxbSDDD+EEe8x81BqqDNvIhbXHqOR2JQE08 +/NapDp78qfnAWjrkljdmjGdRB1Dj+ZFAR6DURz6Sjes8lAB0Kl4mfYw4fDXzpmTA25xu3 ad5zDzrutpf6DI0enk0y07WbZbUaLy0= X-MC-Unique: kEkOSQw7O7CtpEgTPyFt7Q-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 6/9] virshStreamSkip: Emulate skip for block devices Date: Fri, 3 Jul 2020 12:28:47 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This callback is called when the server sends us STREAM_HOLE meaning there is no real data, only zeroes. For regular files we would just seek() beyond EOF and ftruncate() to create the hole. But for block devices this won't work. Not only we can't seek() beyond EOF, and ftruncate() will fail, this approach won't fill the device with zeroes. We have to do it manually. Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D1852528 Signed-off-by: Michal Privoznik --- tools/virsh-util.c | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) diff --git a/tools/virsh-util.c b/tools/virsh-util.c index 89f15efd08..884261eb49 100644 --- a/tools/virsh-util.c +++ b/tools/virsh-util.c @@ -189,11 +189,33 @@ virshStreamSkip(virStreamPtr st G_GNUC_UNUSED, virshStreamCallbackDataPtr cbData =3D opaque; off_t cur; =20 - if ((cur =3D lseek(cbData->fd, offset, SEEK_CUR)) =3D=3D (off_t) -1) - return -1; + if (cbData->isBlock) { + g_autofree char * buf =3D NULL; + const size_t buflen =3D 1 * 1024 * 1024; /* 1MiB */ =20 - if (ftruncate(cbData->fd, cur) < 0) - return -1; + /* While for files it's enough to lseek() and ftruncate() to create + * a hole which would emulate zeroes on read(), for block devices + * we have to write zeroes to read() zeroes. And we have to write + * @got bytes of zeroes. Do that in smaller chunks though.*/ + + buf =3D g_new0(char, buflen); + + while (offset) { + size_t count =3D MIN(offset, buflen); + ssize_t r; + + if ((r =3D safewrite(cbData->fd, buf, count)) < 0) + return -1; + + offset -=3D r; + } + } else { + if ((cur =3D lseek(cbData->fd, offset, SEEK_CUR)) =3D=3D (off_t) -= 1) + return -1; + + if (ftruncate(cbData->fd, cur) < 0) + return -1; + } =20 return 0; } --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772356; cv=none; d=zohomail.com; s=zohoarc; b=WELWIeTEdIbwwWNtGzBv4sVEj12xFJQ5a9frXFQt2AdxgwU77gRxY5yIQe41wNLyY27J6P2fOz9sjNxPWfqWb4GXTVefK0XuQAlWM5SmAksj4zcs4iaiZ8we6Ehqlkad7n2+vydt3J1ehnmlscJMCIaKIAN7HOSrN3HEOHSBZEg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772356; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SdatcooXYI0UwtS7sa3lhn1b2Cbxe7q3USqb9L49O04=; b=VzQubqraqCXk41o3zu8um8738nRmlChsYWQvNiu8zJ6t1JwUeKMeRPHL+NdCIQVY6aAP99ow/zOkMA5CWoHqNBiy8uOiH4CTjTq+Xs/GxFjAgAIlwV7rkXJFBcgmub6XgzohHRQGydCwK7RFJy6TLJJHm1tZpSKWRHtaHjXPID0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1593772356821403.8037152616041; Fri, 3 Jul 2020 03:32:36 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-170-nQ3g45-SPxqtN2jg1DknRA-1; Fri, 03 Jul 2020 06:32:33 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 81D9C800404; Fri, 3 Jul 2020 10:32:28 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6322673FF6; Fri, 3 Jul 2020 10:32:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 167156C9CD; Fri, 3 Jul 2020 10:32:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT7L3017015 for ; Fri, 3 Jul 2020 06:29:07 -0400 Received: by smtp.corp.redhat.com (Postfix) id C286D7BD56; Fri, 3 Jul 2020 10:29:07 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40A6D79242 for ; Fri, 3 Jul 2020 10:29:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772355; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=SdatcooXYI0UwtS7sa3lhn1b2Cbxe7q3USqb9L49O04=; b=PIBKIUl7w9sZCrDUQFGjt2JQotQkyiyHuAwS+UInKkZnbB5XjbmS6/oRoAGpvbAuzFuu0m GkJ6IKNVyigdPMORKo/8Z+cr6D7FiyktNsYV1NCLxboPl/hH13g4cRq/Jdjjqma0oWLI6a xWslaSSCWueElZymNdyWb+085ETeaCw= X-MC-Unique: nQ3g45-SPxqtN2jg1DknRA-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 7/9] virfdstream: Allow sparse stream vol-download Date: Fri, 3 Jul 2020 12:28:48 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" When handling sparse stream, a thread is executed. This thread runs a read() or write() loop (depending what API is called; in this case it's virStorageVolDownload() and this the thread run read() loop). The read() is handled in virFDStreamThreadDoRead() which is then data/hole section aware, meaning it uses virFileInData() to detect data and hole sections and sends TYPE_DATA or TYPE_HOLE virStream messages accordingly. However, virFileInData() does not work with block devices. Simply because block devices don't have data and hole sections. But we can use new virFileInDataDetectZeroes() which is block device friendly for that. Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D1852528 Signed-off-by: Michal Privoznik --- src/util/virfdstream.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/src/util/virfdstream.c b/src/util/virfdstream.c index 1c32be47a9..b5406fe690 100644 --- a/src/util/virfdstream.c +++ b/src/util/virfdstream.c @@ -396,6 +396,7 @@ struct _virFDStreamThreadData { size_t length; bool doRead; bool sparse; + bool isBlock; int fdin; char *fdinname; int fdout; @@ -419,6 +420,7 @@ virFDStreamThreadDataFree(virFDStreamThreadDataPtr data) static ssize_t virFDStreamThreadDoRead(virFDStreamDataPtr fdst, bool sparse, + bool isBlock, const int fdin, const int fdout, const char *fdinname, @@ -435,8 +437,13 @@ virFDStreamThreadDoRead(virFDStreamDataPtr fdst, ssize_t got; =20 if (sparse && *dataLen =3D=3D 0) { - if (virFileInData(fdin, &inData, §ionLen) < 0) - goto error; + if (isBlock) { + if (virFileInDataDetectZeroes(fdin, &inData, §ionLen) < 0) + goto error; + } else { + if (virFileInData(fdin, &inData, §ionLen) < 0) + goto error; + } =20 if (length && sectionLen > length - total) @@ -575,6 +582,7 @@ virFDStreamThread(void *opaque) virStreamPtr st =3D data->st; size_t length =3D data->length; bool sparse =3D data->sparse; + bool isBlock =3D data->isBlock; int fdin =3D data->fdin; char *fdinname =3D data->fdinname; int fdout =3D data->fdout; @@ -611,7 +619,7 @@ virFDStreamThread(void *opaque) } =20 if (doRead) - got =3D virFDStreamThreadDoRead(fdst, sparse, + got =3D virFDStreamThreadDoRead(fdst, sparse, isBlock, fdin, fdout, fdinname, fdoutname, length, total, @@ -1289,6 +1297,7 @@ virFDStreamOpenFileInternal(virStreamPtr st, threadData->st =3D virObjectRef(st); threadData->length =3D length; threadData->sparse =3D sparse; + threadData->isBlock =3D !!S_ISBLK(sb.st_mode); =20 if ((oflags & O_ACCMODE) =3D=3D O_RDONLY) { threadData->fdin =3D fd; --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772355; cv=none; d=zohomail.com; s=zohoarc; b=BmXnJ+kxb/7/ZSX5g/n5Ac5Yr/rSp/hWHsog18A9GqdNnJugROaqgCULwzJBrNvWz3eFhZrRdp3LvIBFXv+2hhQ3oBLRnPrjbkeoce/9wmHBC0pU6z9z6m+jwkrsbKoorYEOn/kcR3IPPMCKdp7MMrNPfgLJd4h2esPF0IOOMM8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772355; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Uee6XayEdsGgEj2ZTEA57z0Qt2dko4kss9iNldYC1j8=; b=NTzMYBLhKhChufs8Um43svYolH+9k42IILZsj+t+cZa39/cgCWTIxRQuyye82EDWStQEvAN4lbfvVs0oLxRNo7XmWGjKjKdXT1EGQezEBKuO5OwiWgsznQnLRUEICsC1/wLOWl6duqwR7zWx6++anX4GEy2OwRnYWvRmVwUCf0Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1593772355714950.0676179068548; Fri, 3 Jul 2020 03:32:35 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-484-EKF5VakBNLu21xGifnbzVw-1; Fri, 03 Jul 2020 06:32:32 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F2504107ACF6; Fri, 3 Jul 2020 10:32:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D350673FEF; Fri, 3 Jul 2020 10:32:26 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9F4C21809563; Fri, 3 Jul 2020 10:32:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT8n5017021 for ; Fri, 3 Jul 2020 06:29:08 -0400 Received: by smtp.corp.redhat.com (Postfix) id A28207BD56; Fri, 3 Jul 2020 10:29:08 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2187E79242 for ; Fri, 3 Jul 2020 10:29:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772354; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Uee6XayEdsGgEj2ZTEA57z0Qt2dko4kss9iNldYC1j8=; b=akGb+z4MGg0dW+SpntrnV7etrYgaRJ3YzYKeYBqca7ycdCxAvJzDjPRaljuG1tfrNA1ooB LDCco7ec8pfNLkzuDQugmmuQV8OESYvSWwXsQ/H0m3dTPwgWSd7+AoQb/+3q/hqQCQMqct O8No2jDkPPEk/raCQhQzp8axxOs5eDU= X-MC-Unique: EKF5VakBNLu21xGifnbzVw-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 8/9] virshStreamInData: Handle block devices Date: Fri, 3 Jul 2020 12:28:49 +0200 Message-Id: <151f263a28a4b3bf25b56ab93db46aeb2099cf12.1593771693.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This is very similar to previous commit. The virshStreamInData() callback is used by virStreamSparseSendAll() to detect whether the file the data is read from is in data or hole section. The SendAll() will then send corresponding type of virStream message to make server create a hole or write actual data. But the callback uses virFileInData() even for block devices, which results in an error. Switch to virFileInDataDetectZeroes() for block devices. Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D1852528 Signed-off-by: Michal Privoznik --- tools/virsh-util.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/tools/virsh-util.c b/tools/virsh-util.c index 884261eb49..867f69577f 100644 --- a/tools/virsh-util.c +++ b/tools/virsh-util.c @@ -230,12 +230,20 @@ virshStreamInData(virStreamPtr st G_GNUC_UNUSED, virshStreamCallbackDataPtr cbData =3D opaque; vshControl *ctl =3D cbData->ctl; int fd =3D cbData->fd; - int ret; =20 - if ((ret =3D virFileInData(fd, inData, offset)) < 0) - vshError(ctl, "%s", _("Unable to get current position in stream")); + if (cbData->isBlock) { + if (virFileInDataDetectZeroes(fd, inData, offset) < 0) { + vshError(ctl, "%s", _("Unable to get current position in strea= m")); + return -1; + } + } else { + if (virFileInData(fd, inData, offset) < 0) { + vshError(ctl, "%s", _("Unable to get current position in strea= m")); + return -1; + } + } =20 - return ret; + return 0; } =20 =20 --=20 2.26.2 From nobody Mon May 6 20:22:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593772367; cv=none; d=zohomail.com; s=zohoarc; b=mNYEGrVz0r77zfwLk+1XEVzD/GvCpLOHNOCLB3I12ifjoSpKJCu1u5XYbhImXylLaSzpqX/FUbqzcTE+IIhEAx2jwFScdhhoikG8/M97qkVEpRNKB6PT/MrU0J3qGSs9xI5P3OTUb1wDwscOF7yK8fDzfuw9sQNyyw2fNwfznSg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593772367; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dJdCRdXtdDJ4IsTXmj+KPwiEHAtIt7p83hqbdSxAioA=; b=Efl6Whi3CpInBsJZa1IgePY050BktgJc57ccm970rCktuzsR2XgO1S0mrHgZcTsNN09+gAU+Yd5yenI8xRROmpPQFunlCTvqHbCnf/TLsAZ3WRicTeGDa6eh9Y5b7INV10ITJvO0P5hA2UikXPYq1VZ+WWEytiZCDqrPUr426UQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1593772367189907.7936122585918; Fri, 3 Jul 2020 03:32:47 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-387-tGcsmR8IMD2iwZG8l8DlTg-1; Fri, 03 Jul 2020 06:32:44 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 32CE5800C60; Fri, 3 Jul 2020 10:32:38 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 11DBC1053B05; Fri, 3 Jul 2020 10:32:38 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D6CA61806B0C; Fri, 3 Jul 2020 10:32:37 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063AT9Jl017026 for ; Fri, 3 Jul 2020 06:29:09 -0400 Received: by smtp.corp.redhat.com (Postfix) id 8193B7BD56; Fri, 3 Jul 2020 10:29:09 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 016B579242 for ; Fri, 3 Jul 2020 10:29:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593772366; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=dJdCRdXtdDJ4IsTXmj+KPwiEHAtIt7p83hqbdSxAioA=; b=cR75Wft0pyBUaghrqZFhuL7sslYjsB0zc6Sf3HhQvg4Pi1dGVys73Uky1/CGOFqBKYR9S2 SZ0DTlaedLEzUz5phWY+fzI4FQZL+wIWb+VQoYwzPm/XvwN+6YtOsXK9G+30fPeUDDA6RQ 9yczhXkA16opYKR3ES8ToQLZk33XOGs= X-MC-Unique: tGcsmR8IMD2iwZG8l8DlTg-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 9/9] virfdstream: Emulate skip for block devices Date: Fri, 3 Jul 2020 12:28:50 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This is similar to one of previous patches. When receiving stream (on virStorageVolUpload() and subsequent virStreamSparseSendAll()) we may receive a hole. If the volume we are saving the incoming data into is a regular file we just lseek() and ftruncate() to create the hole. But this won't work if the file is a block device. If that is the case we must write zeroes so that any subsequent reader reads nothing just zeroes (just like they would from a hole in a regular file). Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D1852528 Signed-off-by: Michal Privoznik --- src/util/virfdstream.c | 59 +++++++++++++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 15 deletions(-) diff --git a/src/util/virfdstream.c b/src/util/virfdstream.c index b5406fe690..0894be0341 100644 --- a/src/util/virfdstream.c +++ b/src/util/virfdstream.c @@ -512,6 +512,7 @@ virFDStreamThreadDoRead(virFDStreamDataPtr fdst, static ssize_t virFDStreamThreadDoWrite(virFDStreamDataPtr fdst, bool sparse, + bool isBlock, const int fdin, const int fdout, const char *fdinname, @@ -519,7 +520,6 @@ virFDStreamThreadDoWrite(virFDStreamDataPtr fdst, { ssize_t got =3D 0; virFDStreamMsgPtr msg =3D fdst->msg; - off_t off; bool pop =3D false; =20 switch (msg->type) { @@ -547,19 +547,48 @@ virFDStreamThreadDoWrite(virFDStreamDataPtr fdst, } =20 got =3D msg->stream.hole.len; - off =3D lseek(fdout, got, SEEK_CUR); - if (off =3D=3D (off_t) -1) { - virReportSystemError(errno, - _("unable to seek in %s"), - fdoutname); - return -1; - } - - if (ftruncate(fdout, off) < 0) { - virReportSystemError(errno, - _("unable to truncate %s"), - fdoutname); - return -1; + if (isBlock) { + g_autofree char * buf =3D NULL; + const size_t buflen =3D 1 * 1024 * 1024; /* 1MiB */ + size_t toWrite =3D got; + + /* While for files it's enough to lseek() and ftruncate() to c= reate + * a hole which would emulate zeroes on read(), for block devi= ces + * we have to write zeroes to read() zeroes. And we have to wr= ite + * @got bytes of zeroes. Do that in smaller chunks though.*/ + + buf =3D g_new0(char, buflen); + + while (toWrite) { + size_t count =3D MIN(toWrite, buflen); + ssize_t r; + + if ((r =3D safewrite(fdout, buf, count)) < 0) { + virReportSystemError(errno, + _("Unable to write %s"), + fdoutname); + return -1; + } + + toWrite -=3D r; + } + } else { + off_t off; + + off =3D lseek(fdout, got, SEEK_CUR); + if (off =3D=3D (off_t) -1) { + virReportSystemError(errno, + _("unable to seek in %s"), + fdoutname); + return -1; + } + + if (ftruncate(fdout, off) < 0) { + virReportSystemError(errno, + _("unable to truncate %s"), + fdoutname); + return -1; + } } =20 pop =3D true; @@ -625,7 +654,7 @@ virFDStreamThread(void *opaque) length, total, &dataLen, buflen); else - got =3D virFDStreamThreadDoWrite(fdst, sparse, + got =3D virFDStreamThreadDoWrite(fdst, sparse, isBlock, fdin, fdout, fdinname, fdoutname); =20 --=20 2.26.2