From nobody Tue Apr 30 02:31:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593775983; cv=none; d=zohomail.com; s=zohoarc; b=AasXDwgmV7RulVL9UF+yd5VYyNVP9TQ9lUAHUSjBRGwXEU1dm2mbSccC6ovGqpmH9RztbKgbdqK8XEHo8pD0WypYFQIuWkD23QYxL0RBQqaT2LUHF6kdV8tMJ8OsKXTXaJBGyDqB6GWke6ohWprg93iM09DmlZwzwhUOV08X8cA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593775983; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=suPjew6BFOV+ZhARWXoG77dYzRaEb3oskpXvE+xHGRs=; b=Je625KWr66cfNbuiz5AcnsaOfJjzkcTTHp33R4MBAIGVuApQJEruZErIUOkQDPEkyxRXDsKNl2SfuxXQakSJcmNOvqWNlaRLRmtERBXG7FJ6F2XxF9wH5dYLtAFqRmA8fEu8U8Bej+L/nYxi9uSQPJ4GowpGYwUYq/nBw7QGgHA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1593775983725213.81975321609696; Fri, 3 Jul 2020 04:33:03 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-296-DwrWlCtMMOG6qXk3TQqvBw-1; Fri, 03 Jul 2020 07:32:15 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EAFF3100CC84; Fri, 3 Jul 2020 11:32:04 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9C6287AC72; Fri, 3 Jul 2020 11:32:03 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8E9576C9CD; Fri, 3 Jul 2020 11:32:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063BV6SQ023613 for ; Fri, 3 Jul 2020 07:31:06 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7F6FA7AC62; Fri, 3 Jul 2020 11:31:06 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3259778B0 for ; Fri, 3 Jul 2020 11:31:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593775982; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=suPjew6BFOV+ZhARWXoG77dYzRaEb3oskpXvE+xHGRs=; b=izUxdqLL0uKUuHe/x5rKVi/C1u8auemcCQ8ASPh6OGhPf5smAp5yZhV16LnZQRv9fGPtz9 SW+qL29hcveBodTHETEw0kEK5SOfJ9GXJZcSkuFhhMXz/QXoB8pOHInnZ+ZHjRre/Tu0bp Y9cKkQRB1o8N4RdhREutSn81TidC/Bc= X-MC-Unique: DwrWlCtMMOG6qXk3TQqvBw-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [libvirt-python][PATCH 1/3] MANIFEST: Distribute sparsestream.py example Date: Fri, 3 Jul 2020 13:30:58 +0200 Message-Id: <0301d07889a1e54dc78a7e2ab933d7ee5cce6226.1593775792.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Signed-off-by: Michal Privoznik Reviewed-by: Pavel Hrdina --- MANIFEST.in | 1 + 1 file changed, 1 insertion(+) diff --git a/MANIFEST.in b/MANIFEST.in index da7cbae..fd76f3c 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -13,6 +13,7 @@ include examples/event-test.py include examples/guest-vcpus/guest-vcpu-daemon.py include examples/guest-vcpus/guest-vcpu.py include examples/nodestats.py +include examples/sparsestream.py include examples/topology.py include generator.py include libvirt-lxc-override-api.xml --=20 2.26.2 From nobody Tue Apr 30 02:31:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593776057; cv=none; d=zohomail.com; s=zohoarc; b=lcqNfxWd1jO+tFwtgkqdcdm9xim2eros7rX81goORsI11qtHvx25QI9FkYuxKP9mE46C+HJlZMoDBw3kxxAMl3AZNqyX/HHECwoQr0leqH+Zwitrn4mr/BuhUUVOWaqvd3vN9X84M1tXNYAWJD1eY0S3BUmYyAYP8lwEzunMMhI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593776057; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RJNR4K43LJePEoaLJMu0wYkgHMjKkt90dcIw27qu9to=; b=gzwdMGsELsSlV0WAeJRfawAvMGrHqyKBKwxGqlxzH/4bBEHg5sYV8QWi9q2mM9zYmxRH1GotCallC7AOQdIrH01KW2zPxSq6sA74eJD/H6nz34i1XyPvQuOqoOd2UoQchqEWO1qH8hKgnLOdoPxVoIEnqTU3s6MyuuT1MLwhhKc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1593776057796299.79376944001046; Fri, 3 Jul 2020 04:34:17 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-499-Pe3s6djVOcaGQM-9QFnhzw-1; Fri, 03 Jul 2020 07:34:14 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7100419200C2; Fri, 3 Jul 2020 11:34:09 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 323B173FC7; Fri, 3 Jul 2020 11:34:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B07976C9CB; Fri, 3 Jul 2020 11:34:08 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063BV7Ha023621 for ; Fri, 3 Jul 2020 07:31:07 -0400 Received: by smtp.corp.redhat.com (Postfix) id 5F6F4778B0; Fri, 3 Jul 2020 11:31:07 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id D2FAB7AC71 for ; Fri, 3 Jul 2020 11:31:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593776056; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=RJNR4K43LJePEoaLJMu0wYkgHMjKkt90dcIw27qu9to=; b=I7X8v4olHKmj/NvFzFs2KbFaVsXEOiXtoLcp++kARFmPs+BKOilR9QHos+gQtfurcKVwR2 oF+BmHccRuwLg5m9NkAtoEfAJFGdUrIlDvIvIYUEW8WsI9z9C//qqxtYOiu5d6uFj6A4Nn EPEEmcqYHkDHaQat6wBw01NdaM0S2hA= X-MC-Unique: Pe3s6djVOcaGQM-9QFnhzw-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [libvirt-python][PATCH 2/3] virStream: Use larger buffer for sendAll/recvAll methods Date: Fri, 3 Jul 2020 13:30:59 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" There are four methods which receive/send entire stream (sendAll(), recvAll(), sparseSendAll() and sparseRecvAll()). All these have an intermediary buffer which is either filled by incoming stream and passed to a user provided callback to handle the data, or the other way round - user fills it with data they want to send and the buffer is handed over to virStream. But the buffer is incredibly small which leads to smaller packets being sent and thus increased overhead. What we can do is to use the same buffer as their C counterparts do (e.g. virStreamSendAll()) - they all use VIR_NET_MESSAGE_LEGACY_PAYLOAD_MAX long buffer (which is the maximum size of a stream packet we send) - this is almost exactly 256KiB (it's 256KiB - 24B for the header). Signed-off-by: Michal Privoznik Reviewed-by: Pavel Hrdina --- libvirt-override-virStream.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libvirt-override-virStream.py b/libvirt-override-virStream.py index 901c2e6..8880218 100644 --- a/libvirt-override-virStream.py +++ b/libvirt-override-virStream.py @@ -39,7 +39,7 @@ return os.write(fd, buf) """ while True: - got =3D self.recv(1024*64) + got =3D self.recv(262120) if got =3D=3D -2: raise libvirtError("cannot use recvAll with " "nonblocking stream") @@ -74,7 +74,7 @@ """ while True: try: - got =3D handler(self, 1024*64, opaque) + got =3D handler(self, 262120, opaque) except: e =3D sys.exc_info()[1] try: @@ -189,7 +189,7 @@ # actually allocate the hole """ while True: - want =3D 64 * 1024 + want =3D 262120 got =3D self.recvFlags(want, VIR_STREAM_RECV_STOP_AT_HOLE) if got =3D=3D -2: raise libvirtError("cannot use sparseRecvAll with " @@ -251,7 +251,7 @@ self.abort() continue =20 - want =3D 64 * 1024 + want =3D 262120 if (want > sectionLen): want =3D sectionLen =20 --=20 2.26.2 From nobody Tue Apr 30 02:31:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593775913; cv=none; d=zohomail.com; s=zohoarc; b=iML2YG7AZ//Lp0Gl8T7oZc0JJGCBx7Lc2ivHDLEqBBDqzlF89/Td/77QzPtkRS2NfH5xAEZQh8e68K8QAKf4rmh3R7LDkNxiMqyQrAstfQhavfcjrnWbl3baA2HyeoHrnrehzunqK0m0kMv71fqEzeASEmrBVx+cwwCS9LJzx60= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593775913; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=60lXpYV+B0rBEhcwVaFhDIetv3RyHQqrmYLGfArO2Rk=; b=A8zFgCD1kshxz6x0qeXA801oge6FlvYGi4WXPs5/WmS1/3LhGNmiScexBroYVm6esV/lcpQiEYIJMD7ectGMp8FQKlrRb8SgeuA8LEyvXQsRLcbjxsFLsy6G8ByOCh7lQBCkSYfrWFOzI1ux5T0BQlenwnpIH3TKPeKWlSSDC68= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1593775913564126.2070840220589; Fri, 3 Jul 2020 04:31:53 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-264-ulsaUQr2NLe2K-Dbgv1u7g-1; Fri, 03 Jul 2020 07:31:49 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0D791107ACF2; Fri, 3 Jul 2020 11:31:44 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CD8C07BD42; Fri, 3 Jul 2020 11:31:43 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3C4511809547; Fri, 3 Jul 2020 11:31:43 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 063BV8hJ023630 for ; Fri, 3 Jul 2020 07:31:08 -0400 Received: by smtp.corp.redhat.com (Postfix) id 4039D7AC72; Fri, 3 Jul 2020 11:31:08 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id B30D5778B0 for ; Fri, 3 Jul 2020 11:31:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593775911; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=60lXpYV+B0rBEhcwVaFhDIetv3RyHQqrmYLGfArO2Rk=; b=CihwVbB+sBY+PeCAuYvdtdh69Y4lU/W45yWSdc8+vyFZGuqI5VyzLGh0AHnNXCokTBD4Ug HUtawDuID6hpAt9xtjfcOVNpXbL98LzBNoIXdCX27DW4saXpiySoKf6XiS6veMHxFy5kw4 qymsozrhegzewzwu2x21x9qagxfEVe4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593775911; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=60lXpYV+B0rBEhcwVaFhDIetv3RyHQqrmYLGfArO2Rk=; b=CihwVbB+sBY+PeCAuYvdtdh69Y4lU/W45yWSdc8+vyFZGuqI5VyzLGh0AHnNXCokTBD4Ug HUtawDuID6hpAt9xtjfcOVNpXbL98LzBNoIXdCX27DW4saXpiySoKf6XiS6veMHxFy5kw4 qymsozrhegzewzwu2x21x9qagxfEVe4= X-MC-Unique: ulsaUQr2NLe2K-Dbgv1u7g-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [libvirt-python][PATCH 3/3] examples: Update sparsestream.py so that it handles block devices Date: Fri, 3 Jul 2020 13:31:00 +0200 Message-Id: <57d1de17bbeace763aef260c4f394b60864d0ae1.1593775792.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This patch is similar to what I sent for libvirt. It extends callbacks so that they work with block devices. Signed-off-by: Michal Privoznik --- examples/sparsestream.py | 73 +++++++++++++++++++++++++++++----------- 1 file changed, 54 insertions(+), 19 deletions(-) diff --git a/examples/sparsestream.py b/examples/sparsestream.py index d7c09b7..4e96d43 100755 --- a/examples/sparsestream.py +++ b/examples/sparsestream.py @@ -4,27 +4,27 @@ # Authors: # Michal Privoznik =20 -import libvirt, sys, os +import libvirt, stat, sys, os =20 -def bytesWriteHandler(stream, buf, opaque): - fd =3D opaque - return os.write(fd, buf) +def bytesWriteHandler(stream, buf, cbData): + return os.write(cbData["fd"], buf) =20 -def bytesReadHandler(stream, nbytes, opaque): - fd =3D opaque - return os.read(fd, nbytes) +def bytesReadHandler(stream, nbytes, cbData): + return os.read(cbData["fd"], nbytes) =20 -def recvSkipHandler(stream, length, opaque): - fd =3D opaque - cur =3D os.lseek(fd, length, os.SEEK_CUR) - return os.ftruncate(fd, cur) +def recvSkipHandler(stream, length, cbData): + fd =3D cbData["fd"] + if cbData["isBlock"]: + os.write(fd, b'\x00' * length) + else : + cur =3D os.lseek(fd, length, os.SEEK_CUR) + os.ftruncate(fd, cur) + return 0 =20 -def sendSkipHandler(stream, length, opaque): - fd =3D opaque - return os.lseek(fd, length, os.SEEK_CUR) +def sendSkipHandler(stream, length, cbData): + return os.lseek(cbData["fd"], length, os.SEEK_CUR) =20 -def holeHandler(stream, opaque): - fd =3D opaque +def inDataRegular(fd): cur =3D os.lseek(fd, 0, os.SEEK_CUR) =20 try: @@ -73,13 +73,45 @@ def holeHandler(stream, opaque): os.lseek(fd, cur, os.SEEK_SET) return [inData, sectionLen] =20 +def inDataDetectZeores(fd): + cur =3D os.lseek(fd, 0, os.SEEK_CUR) + chunksize =3D 32 * 1024 * 1024 # 2MiB + + buf =3D os.read(fd, chunksize) + + zeroes =3D bytes(len(buf)) + inData =3D buf !=3D zeroes + sectionLen =3D len(buf) + print("cur: %d inData: %s sectionLen: %d" % (cur, inData, sectionLen)) + + os.lseek(fd, cur, os.SEEK_SET) + return [inData, sectionLen] + + +def holeHandler(stream, cbData): + if cbData["isBlock"]: + return inDataDetectZeores(cbData["fd"]) + else: + return inDataRegular(cbData["fd"]) + def download(vol, st, filename): offset =3D 0 length =3D 0 + isBlock =3D False =20 - fd =3D os.open(filename, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode= =3D0o0660) + try: + sb =3D os.stat(filename) + isBlock =3D stat.S_ISBLK(sb.st_mode) + except FileNotFoundError: + pass + + if isBlock: + fd =3D os.open(filename, os.O_WRONLY, mode=3D0o0660) + else: + fd =3D os.open(filename, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mo= de=3D0o0660) + cbData =3D {"fd": fd, "isBlock": isBlock} vol.download(st, offset, length, libvirt.VIR_STORAGE_VOL_DOWNLOAD_SPAR= SE_STREAM) - st.sparseRecvAll(bytesWriteHandler, recvSkipHandler, fd) + st.sparseRecvAll(bytesWriteHandler, recvSkipHandler, cbData) =20 os.close(fd) =20 @@ -87,9 +119,12 @@ def upload(vol, st, filename): offset =3D 0 length =3D 0 =20 + sb =3D os.stat(filename) + isBlock =3D stat.S_ISBLK(sb.st_mode) fd =3D os.open(filename, os.O_RDONLY) + cbData =3D {"fd": fd, "isBlock": isBlock} vol.upload(st, offset, length, libvirt.VIR_STORAGE_VOL_UPLOAD_SPARSE_S= TREAM) - st.sparseSendAll(bytesReadHandler, holeHandler, sendSkipHandler, fd) + st.sparseSendAll(bytesReadHandler, holeHandler, sendSkipHandler, cbDat= a) =20 os.close(fd) =20 --=20 2.26.2