From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943474665304.32284217477877; Tue, 16 May 2017 07:04:34 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BE1A7A6E14; Tue, 16 May 2017 14:04:27 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5B82C5C54B; Tue, 16 May 2017 14:04:27 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4FBEC4A48C; Tue, 16 May 2017 14:04:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4K51027674 for ; Tue, 16 May 2017 10:04:20 -0400 Received: by smtp.corp.redhat.com (Postfix) id 3F984860ED; Tue, 16 May 2017 14:04:20 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBDF062926 for ; Tue, 16 May 2017 14:04:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com BE1A7A6E14 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com BE1A7A6E14 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:41 +0200 Message-Id: <4dab73a63cb4cba42772949a45762515b77098b9.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 01/31] virfdstream: Use messages instead of pipe X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 16 May 2017 14:04:28 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" One big downside of using the pipe to transfer the data is that we can really transfer just bare data. No metadata can be carried through unless some formatted messages are introduced. That would be quite painful to achieve so let's use a message queue. It's fairly easy to exchange info between threads now that iohelper is no longer used. The reason why we cannot use the FD for plain files directly is that despite us setting noblock flag on the FD, any read()/write() blocks regardless (which is a show stopper since those parts of the code are run from the event loop) and poll() reports such FD as always readable/writable - even though the subsequent operation might block. The pipe is still not gone though. It is used to signal to even loop that an event occurred (e.g. data are available for reading in the queue, or vice versa). Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/util/virfdstream.c | 402 ++++++++++++++++++++++++++++++++++++++++++---= ---- 1 file changed, 350 insertions(+), 52 deletions(-) diff --git a/src/util/virfdstream.c b/src/util/virfdstream.c index 5ce78fe58..4b42939e7 100644 --- a/src/util/virfdstream.c +++ b/src/util/virfdstream.c @@ -49,6 +49,27 @@ =20 VIR_LOG_INIT("fdstream"); =20 +typedef enum { + VIR_FDSTREAM_MSG_TYPE_DATA, +} virFDStreamMsgType; + +typedef struct _virFDStreamMsg virFDStreamMsg; +typedef virFDStreamMsg *virFDStreamMsgPtr; +struct _virFDStreamMsg { + virFDStreamMsgPtr next; + + virFDStreamMsgType type; + + union { + struct { + char *buf; + size_t len; + size_t offset; + } data; + } stream; +}; + + /* Tunnelled migration stream support */ typedef struct virFDStreamData virFDStreamData; typedef virFDStreamData *virFDStreamDataPtr; @@ -80,18 +101,25 @@ struct virFDStreamData { =20 /* Thread data */ virThreadPtr thread; + virCond threadCond; int threadErr; bool threadQuit; + bool threadAbort; + bool threadDoRead; + virFDStreamMsgPtr msg; }; =20 static virClassPtr virFDStreamDataClass; =20 +static void virFDStreamMsgQueueFree(virFDStreamMsgPtr *queue); + static void virFDStreamDataDispose(void *obj) { virFDStreamDataPtr fdst =3D obj; =20 VIR_DEBUG("obj=3D%p", fdst); + virFDStreamMsgQueueFree(&fdst->msg); } =20 static int virFDStreamDataOnceInit(void) @@ -108,6 +136,89 @@ static int virFDStreamDataOnceInit(void) VIR_ONCE_GLOBAL_INIT(virFDStreamData) =20 =20 +static int +virFDStreamMsgQueuePush(virFDStreamDataPtr fdst, + virFDStreamMsgPtr msg, + int fd, + const char *fdname) +{ + virFDStreamMsgPtr *tmp =3D &fdst->msg; + char c =3D '1'; + + while (*tmp) + tmp =3D &(*tmp)->next; + + *tmp =3D msg; + virCondSignal(&fdst->threadCond); + + if (safewrite(fd, &c, sizeof(c)) !=3D sizeof(c)) { + virReportSystemError(errno, + _("Unable to write to %s"), + fdname); + return -1; + } + + return 0; +} + + +static virFDStreamMsgPtr +virFDStreamMsgQueuePop(virFDStreamDataPtr fdst, + int fd, + const char *fdname) +{ + virFDStreamMsgPtr tmp =3D fdst->msg; + char c; + + if (tmp) { + fdst->msg =3D tmp->next; + tmp->next =3D NULL; + } + + virCondSignal(&fdst->threadCond); + + if (saferead(fd, &c, sizeof(c)) !=3D sizeof(c)) { + virReportSystemError(errno, + _("Unable to read from %s"), + fdname); + return NULL; + } + + return tmp; +} + + +static void +virFDStreamMsgFree(virFDStreamMsgPtr msg) +{ + if (!msg) + return; + + switch (msg->type) { + case VIR_FDSTREAM_MSG_TYPE_DATA: + VIR_FREE(msg->stream.data.buf); + break; + } + + VIR_FREE(msg); +} + + +static void +virFDStreamMsgQueueFree(virFDStreamMsgPtr *queue) +{ + virFDStreamMsgPtr tmp =3D *queue; + + while (tmp) { + virFDStreamMsgPtr next =3D tmp->next; + virFDStreamMsgFree(tmp); + tmp =3D next; + } + + *queue =3D NULL; +} + + static int virFDStreamRemoveCallback(virStreamPtr stream) { virFDStreamDataPtr fdst =3D stream->privateData; @@ -273,6 +384,7 @@ typedef virFDStreamThreadData *virFDStreamThreadDataPtr; struct _virFDStreamThreadData { virStreamPtr st; size_t length; + bool doRead; int fdin; char *fdinname; int fdout; @@ -293,6 +405,86 @@ virFDStreamThreadDataFree(virFDStreamThreadDataPtr dat= a) } =20 =20 +static ssize_t +virFDStreamThreadDoRead(virFDStreamDataPtr fdst, + const int fdin, + const int fdout, + const char *fdinname, + const char *fdoutname, + size_t buflen) +{ + virFDStreamMsgPtr msg =3D NULL; + char *buf =3D NULL; + ssize_t got; + + if (VIR_ALLOC(msg) < 0) + goto error; + + if (VIR_ALLOC_N(buf, buflen) < 0) + goto error; + + if ((got =3D saferead(fdin, buf, buflen)) < 0) { + virReportSystemError(errno, + _("Unable to read %s"), + fdinname); + goto error; + } + + msg->type =3D VIR_FDSTREAM_MSG_TYPE_DATA; + msg->stream.data.buf =3D buf; + msg->stream.data.len =3D got; + buf =3D NULL; + + virFDStreamMsgQueuePush(fdst, msg, fdout, fdoutname); + msg =3D NULL; + + return got; + + error: + VIR_FREE(buf); + virFDStreamMsgFree(msg); + return -1; +} + + +static ssize_t +virFDStreamThreadDoWrite(virFDStreamDataPtr fdst, + const int fdin, + const int fdout, + const char *fdinname, + const char *fdoutname) +{ + ssize_t got; + virFDStreamMsgPtr msg =3D fdst->msg; + bool pop =3D false; + + switch (msg->type) { + case VIR_FDSTREAM_MSG_TYPE_DATA: + got =3D safewrite(fdout, + msg->stream.data.buf + msg->stream.data.offset, + msg->stream.data.len - msg->stream.data.offset); + if (got < 0) { + virReportSystemError(errno, + _("Unable to write %s"), + fdoutname); + return -1; + } + + msg->stream.data.offset +=3D got; + + pop =3D msg->stream.data.offset =3D=3D msg->stream.data.len; + break; + } + + if (pop) { + virFDStreamMsgQueuePop(fdst, fdin, fdinname); + virFDStreamMsgFree(msg); + } + + return got; +} + + static void virFDStreamThread(void *opaque) { @@ -304,14 +496,12 @@ virFDStreamThread(void *opaque) int fdout =3D data->fdout; char *fdoutname =3D data->fdoutname; virFDStreamDataPtr fdst =3D st->privateData; - char *buf =3D NULL; + bool doRead =3D fdst->threadDoRead; size_t buflen =3D 256 * 1024; size_t total =3D 0; =20 virObjectRef(fdst); - - if (VIR_ALLOC_N(buf, buflen) < 0) - goto error; + virObjectLock(fdst); =20 while (1) { ssize_t got; @@ -323,39 +513,56 @@ virFDStreamThread(void *opaque) if (buflen =3D=3D 0) break; /* End of requested data from client */ =20 - if ((got =3D saferead(fdin, buf, buflen)) < 0) { - virReportSystemError(errno, - _("Unable to read %s"), - fdinname); + while (doRead =3D=3D (fdst->msg !=3D NULL) && + !fdst->threadQuit) { + if (virCondWait(&fdst->threadCond, &fdst->parent.lock)) { + virReportSystemError(errno, "%s", + _("failed to wait on condition")); + goto error; + } + } + + if (fdst->threadQuit) { + /* If stream abort was requested, quit early. */ + if (fdst->threadAbort) + goto cleanup; + + /* Otherwise flush buffers and quit gracefully. */ + if (doRead =3D=3D (fdst->msg !=3D NULL)) + break; + } + + if (doRead) + got =3D virFDStreamThreadDoRead(fdst, + fdin, fdout, + fdinname, fdoutname, + buflen); + else + got =3D virFDStreamThreadDoWrite(fdst, + fdin, fdout, + fdinname, fdoutname); + + if (got < 0) goto error; - } =20 if (got =3D=3D 0) break; =20 total +=3D got; - - if (safewrite(fdout, buf, got) < 0) { - virReportSystemError(errno, - _("Unable to write %s"), - fdoutname); - goto error; - } } =20 cleanup: + fdst->threadQuit =3D true; + virObjectUnlock(fdst); if (!virObjectUnref(fdst)) st->privateData =3D NULL; VIR_FORCE_CLOSE(fdin); VIR_FORCE_CLOSE(fdout); virFDStreamThreadDataFree(data); - VIR_FREE(buf); return; =20 error: - virObjectLock(fdst); fdst->threadErr =3D errno; - virObjectUnlock(fdst); goto cleanup; } =20 @@ -368,6 +575,10 @@ virFDStreamJoinWorker(virFDStreamDataPtr fdst, if (!fdst->thread) return 0; =20 + fdst->threadAbort =3D streamAbort; + fdst->threadQuit =3D true; + virCondSignal(&fdst->threadCond); + /* Give the thread a chance to lock the FD stream object. */ virObjectUnlock(fdst); virThreadJoin(fdst->thread); @@ -381,6 +592,7 @@ virFDStreamJoinWorker(virFDStreamDataPtr fdst, ret =3D 0; cleanup: VIR_FREE(fdst->thread); + virCondDestroy(&fdst->threadCond); return ret; } =20 @@ -427,11 +639,14 @@ virFDStreamCloseInt(virStreamPtr st, bool streamAbort) fdst->abortCallbackDispatching =3D false; } =20 - /* mutex locked */ - ret =3D VIR_CLOSE(fdst->fd); if (virFDStreamJoinWorker(fdst, streamAbort) < 0) ret =3D -1; =20 + /* mutex locked */ + if ((ret =3D VIR_CLOSE(fdst->fd)) < 0) + virReportSystemError(errno, "%s", + _("Unable to close")); + st->privateData =3D NULL; =20 /* call the internal stream closing callback */ @@ -468,7 +683,8 @@ virFDStreamAbort(virStreamPtr st) static int virFDStreamWrite(virStreamPtr st, const char *bytes, size_t nby= tes) { virFDStreamDataPtr fdst =3D st->privateData; - int ret; + virFDStreamMsgPtr msg =3D NULL; + int ret =3D -1; =20 if (nbytes > INT_MAX) { virReportSystemError(ERANGE, "%s", @@ -496,25 +712,51 @@ static int virFDStreamWrite(virStreamPtr st, const ch= ar *bytes, size_t nbytes) nbytes =3D fdst->length - fdst->offset; } =20 - retry: - ret =3D write(fdst->fd, bytes, nbytes); - if (ret < 0) { - VIR_WARNINGS_NO_WLOGICALOP_EQUAL_EXPR - if (errno =3D=3D EAGAIN || errno =3D=3D EWOULDBLOCK) { - VIR_WARNINGS_RESET - ret =3D -2; - } else if (errno =3D=3D EINTR) { - goto retry; - } else { - ret =3D -1; - virReportSystemError(errno, "%s", + if (fdst->thread) { + char *buf; + + if (fdst->threadQuit) { + virReportSystemError(EBADF, "%s", _("cannot write to stream")); + return -1; + } + + if (VIR_ALLOC(msg) < 0 || + VIR_ALLOC_N(buf, nbytes) < 0) + goto cleanup; + + memcpy(buf, bytes, nbytes); + msg->type =3D VIR_FDSTREAM_MSG_TYPE_DATA; + msg->stream.data.buf =3D buf; + msg->stream.data.len =3D nbytes; + + virFDStreamMsgQueuePush(fdst, msg, fdst->fd, "pipe"); + msg =3D NULL; + ret =3D nbytes; + } else { + retry: + ret =3D write(fdst->fd, bytes, nbytes); + if (ret < 0) { + VIR_WARNINGS_NO_WLOGICALOP_EQUAL_EXPR + if (errno =3D=3D EAGAIN || errno =3D=3D EWOULDBLOCK) { + VIR_WARNINGS_RESET + ret =3D -2; + } else if (errno =3D=3D EINTR) { + goto retry; + } else { + ret =3D -1; + virReportSystemError(errno, "%s", + _("cannot write to stream")); + } } - } else if (fdst->length) { - fdst->offset +=3D ret; } =20 + if (fdst->length) + fdst->offset +=3D ret; + + cleanup: virObjectUnlock(fdst); + virFDStreamMsgFree(msg); return ret; } =20 @@ -522,7 +764,7 @@ static int virFDStreamWrite(virStreamPtr st, const char= *bytes, size_t nbytes) static int virFDStreamRead(virStreamPtr st, char *bytes, size_t nbytes) { virFDStreamDataPtr fdst =3D st->privateData; - int ret; + int ret =3D -1; =20 if (nbytes > INT_MAX) { virReportSystemError(ERANGE, "%s", @@ -548,24 +790,70 @@ static int virFDStreamRead(virStreamPtr st, char *byt= es, size_t nbytes) nbytes =3D fdst->length - fdst->offset; } =20 - retry: - ret =3D read(fdst->fd, bytes, nbytes); - if (ret < 0) { - VIR_WARNINGS_NO_WLOGICALOP_EQUAL_EXPR - if (errno =3D=3D EAGAIN || errno =3D=3D EWOULDBLOCK) { - VIR_WARNINGS_RESET - ret =3D -2; - } else if (errno =3D=3D EINTR) { - goto retry; - } else { - ret =3D -1; - virReportSystemError(errno, "%s", - _("cannot read from stream")); + if (fdst->thread) { + virFDStreamMsgPtr msg =3D NULL; + + while (!(msg =3D fdst->msg)) { + if (fdst->threadQuit) { + if (nbytes) { + virReportSystemError(EBADF, "%s", + _("stream is not open")); + } else { + ret =3D 0; + } + goto cleanup; + } else { + virObjectUnlock(fdst); + virCondSignal(&fdst->threadCond); + virObjectLock(fdst); + } + } + + if (msg->type !=3D VIR_FDSTREAM_MSG_TYPE_DATA) { + /* Nope, nope, I'm outta here */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("unexpected message type")); + goto cleanup; + } + + if (nbytes > msg->stream.data.len - msg->stream.data.offset) + nbytes =3D msg->stream.data.len - msg->stream.data.offset; + + memcpy(bytes, + msg->stream.data.buf + msg->stream.data.offset, + nbytes); + + msg->stream.data.offset +=3D nbytes; + if (msg->stream.data.offset =3D=3D msg->stream.data.len) { + virFDStreamMsgQueuePop(fdst, fdst->fd, "pipe"); + virFDStreamMsgFree(msg); + } + + ret =3D nbytes; + + } else { + retry: + ret =3D read(fdst->fd, bytes, nbytes); + if (ret < 0) { + VIR_WARNINGS_NO_WLOGICALOP_EQUAL_EXPR + if (errno =3D=3D EAGAIN || errno =3D=3D EWOULDBLOCK) { + VIR_WARNINGS_RESET + ret =3D -2; + } else if (errno =3D=3D EINTR) { + goto retry; + } else { + ret =3D -1; + virReportSystemError(errno, "%s", + _("cannot read from stream")); + } + goto cleanup; } - } else if (fdst->length) { - fdst->offset +=3D ret; } =20 + if (fdst->length) + fdst->offset +=3D ret; + + cleanup: virObjectUnlock(fdst); return ret; } @@ -610,11 +898,19 @@ static int virFDStreamOpenInternal(virStreamPtr st, st->privateData =3D fdst; =20 if (threadData) { + fdst->threadDoRead =3D threadData->doRead; + /* Create the thread after fdst and st were initialized. * The thread worker expects them to be that way. */ if (VIR_ALLOC(fdst->thread) < 0) goto error; =20 + if (virCondInit(&fdst->threadCond) < 0) { + virReportSystemError(errno, "%s", + _("cannot initialize condition variable")= ); + goto error; + } + if (virThreadCreate(fdst->thread, true, virFDStreamThread, @@ -783,6 +1079,7 @@ virFDStreamOpenFileInternal(virStreamPtr st, VIR_STRDUP(threadData->fdoutname, "pipe") < 0) goto error; tmpfd =3D pipefds[0]; + threadData->doRead =3D true; } else { threadData->fdin =3D pipefds[0]; threadData->fdout =3D fd; @@ -790,6 +1087,7 @@ virFDStreamOpenFileInternal(virStreamPtr st, VIR_STRDUP(threadData->fdoutname, path) < 0) goto error; tmpfd =3D pipefds[1]; + threadData->doRead =3D false; } } =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 149494349297958.64678295841168; Tue, 16 May 2017 07:04:52 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 334E5796ED; Tue, 16 May 2017 14:04:37 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BE5BA7D94F; Tue, 16 May 2017 14:04:36 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E61354A493; Tue, 16 May 2017 14:04:35 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4MhD027683 for ; Tue, 16 May 2017 10:04:22 -0400 Received: by smtp.corp.redhat.com (Postfix) id 10889860E6; Tue, 16 May 2017 14:04:21 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C3F262926 for ; Tue, 16 May 2017 14:04:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 334E5796ED Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 334E5796ED From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:42 +0200 Message-Id: <711d26e73d74c43ea3775ffddbb4231a6606e771.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 02/31] util: Introduce virFileInData X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 16 May 2017 14:04:38 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This function takes a FD and determines whether the current position is in data section or in a hole. In addition to that, it also determines how much bytes are there remaining till the current section ends. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/libvirt_private.syms | 1 + src/util/virfile.c | 82 +++++++++++++++++++ src/util/virfile.h | 4 + tests/virfiletest.c | 203 +++++++++++++++++++++++++++++++++++++++++++= ++++ 4 files changed, 290 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index bbe283529..4102a002b 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1628,6 +1628,7 @@ virFileGetHugepageSize; virFileGetMountReverseSubtree; virFileGetMountSubtree; virFileHasSuffix; +virFileInData; virFileIsAbsPath; virFileIsDir; virFileIsExecutable; diff --git a/src/util/virfile.c b/src/util/virfile.c index ea44a647c..5b10f9489 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -3793,6 +3793,88 @@ virFileComparePaths(const char *p1, const char *p2) cleanup: VIR_FREE(res1); VIR_FREE(res2); + + return ret; +} + + +int +virFileInData(int fd, + int *inData, + long long *length) +{ + int ret =3D -1; + off_t cur, data, hole, end; + + /* Get current position */ + cur =3D lseek(fd, 0, SEEK_CUR); + if (cur =3D=3D (off_t) -1) { + virReportSystemError(errno, "%s", + _("Unable to get current position in file")); + goto cleanup; + } + + /* Now try to get data and hole offsets */ + data =3D lseek(fd, cur, SEEK_DATA); + + /* There are four options: + * 1) data =3D=3D cur; @cur is in data + * 2) data > cur; @cur is in a hole, next data at @data + * 3) data < 0, errno =3D ENXIO; either @cur is in trailing hole, or @= cur is beyond EOF. + * 4) data < 0, errno !=3D ENXIO; we learned nothing + */ + + if (data =3D=3D (off_t) -1) { + /* cases 3 and 4 */ + if (errno !=3D ENXIO) { + virReportSystemError(errno, "%s", + _("Unable to seek to data")); + goto cleanup; + } + + *inData =3D 0; + /* There are two situations now. There is always an + * implicit hole at EOF. However, there might be a + * trailing hole just before EOF too. If that's the case + * report it. */ + if ((end =3D lseek(fd, 0, SEEK_END)) =3D=3D (off_t) -1) { + virReportSystemError(errno, "%s", + _("Unable to seek to EOF")); + goto cleanup; + } + *length =3D end - cur; + } else if (data > cur) { + /* case 2 */ + *inData =3D 0; + *length =3D data - cur; + } else { + /* case 1 */ + *inData =3D 1; + + /* We don't know where does the next hole start. Let's + * find out. Here we get the same 4 possibilities as + * described above.*/ + hole =3D lseek(fd, data, SEEK_HOLE); + if (hole =3D=3D (off_t) -1 || hole =3D=3D data) { + /* cases 1, 3 and 4 */ + /* Wait a second. The reason why we are here is + * because we are in data. But at the same time we + * are in a trailing hole? Wut!? Do the best what we + * can do here. */ + virReportSystemError(errno, "%s", + _("unable to seek to hole")); + goto cleanup; + } else { + /* case 2 */ + *length =3D (hole - data); + } + } + + ret =3D 0; + cleanup: + /* At any rate, reposition back to where we started. */ + if (cur !=3D (off_t) -1) + ignore_value(lseek(fd, cur, SEEK_SET)); return ret; } =20 diff --git a/src/util/virfile.h b/src/util/virfile.h index 38e938f87..57ceb8072 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -348,4 +348,8 @@ int virFileReadValueString(char **value, const char *fo= rmat, ...) ATTRIBUTE_FMT_PRINTF(2, 3); =20 =20 +int virFileInData(int fd, + int *inData, + long long *length); + #endif /* __VIR_FILE_H */ diff --git a/tests/virfiletest.c b/tests/virfiletest.c index 702a76a50..a93bee01a 100644 --- a/tests/virfiletest.c +++ b/tests/virfiletest.c @@ -21,6 +21,7 @@ #include =20 #include +#include =20 #include "testutils.h" #include "virfile.h" @@ -118,6 +119,190 @@ testFileSanitizePath(const void *opaque) } =20 =20 +static int +makeSparseFile(const off_t offsets[], + const bool startData); + +#ifdef __linux__ +/* Create a sparse file. @offsets in KiB. */ +static int +makeSparseFile(const off_t offsets[], + const bool startData) +{ + int fd =3D -1; + char path[] =3D abs_builddir "fileInData.XXXXXX"; + off_t len =3D 0; + size_t i; + + if ((fd =3D mkostemp(path, O_CLOEXEC|O_RDWR)) < 0) + goto error; + + if (unlink(path) < 0) + goto error; + + for (i =3D 0; offsets[i] !=3D (off_t) -1; i++) + len +=3D offsets[i] * 1024; + + while (len) { + const char buf[] =3D "abcdefghijklmnopqrstuvwxyz"; + off_t toWrite =3D sizeof(buf); + + if (toWrite > len) + toWrite =3D len; + + if (safewrite(fd, buf, toWrite) < 0) { + fprintf(stderr, "unable to write to %s (errno=3D%d)\n", path, = errno); + goto error; + } + + len -=3D toWrite; + } + + len =3D 0; + for (i =3D 0; offsets[i] !=3D (off_t) -1; i++) { + bool inData =3D startData; + + if (i % 2) + inData =3D !inData; + + if (!inData && + fallocate(fd, + FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, + len, offsets[i] * 1024) < 0) { + fprintf(stderr, "unable to punch a hole at offset %lld length = %lld\n", + (long long) len, (long long) offsets[i]); + goto error; + } + + len +=3D offsets[i] * 1024; + } + + if (lseek(fd, 0, SEEK_SET) =3D=3D (off_t) -1) { + fprintf(stderr, "unable to lseek (errno=3D%d)\n", errno); + goto error; + } + + return fd; + error: + VIR_FORCE_CLOSE(fd); + return -1; +} + +#else /* !__linux__ */ + +static int +makeSparseFile(const off_t offsets[] ATTRIBUTE_UNUSED, + const bool startData ATTRIBUTE_UNUSED) +{ + return -1; +} + +#endif /* !__linux__ */ + + +#define EXTENT 4 +static bool +holesSupported(void) +{ + off_t offsets[] =3D {EXTENT, EXTENT, EXTENT, -1}; + off_t tmp; + int fd; + bool ret =3D false; + + if ((fd =3D makeSparseFile(offsets, true)) < 0) + goto cleanup; + + /* The way this works is: there are 4K of data followed by 4K hole fol= lowed + * by 4K hole again. Check if the filesystem we are running the test s= uite + * on supports holes. */ + if ((tmp =3D lseek(fd, 0, SEEK_DATA)) =3D=3D (off_t) -1) + goto cleanup; + + if (tmp !=3D 0) + goto cleanup; + + if ((tmp =3D lseek(fd, tmp, SEEK_HOLE)) =3D=3D (off_t) -1) + goto cleanup; + + if (tmp !=3D EXTENT * 1024) + goto cleanup; + + if ((tmp =3D lseek(fd, tmp, SEEK_DATA)) =3D=3D (off_t) -1) + goto cleanup; + + if (tmp !=3D 2 * EXTENT * 1024) + goto cleanup; + + if ((tmp =3D lseek(fd, tmp, SEEK_HOLE)) =3D=3D (off_t) -1) + goto cleanup; + + if (tmp !=3D 3 * EXTENT * 1024) + goto cleanup; + + ret =3D true; + cleanup: + VIR_FORCE_CLOSE(fd); + return ret; +} + + +struct testFileInData { + bool startData; /* whether the list of offsets starts with data se= ction */ + off_t *offsets; +}; + + +static int +testFileInData(const void *opaque) +{ + const struct testFileInData *data =3D opaque; + int fd =3D -1; + int ret =3D -1; + size_t i; + + if ((fd =3D makeSparseFile(data->offsets, data->startData)) < 0) + goto cleanup; + + for (i =3D 0; data->offsets[i] !=3D (off_t) -1; i++) { + bool shouldInData =3D data->startData; + int realInData; + long long shouldLen; + long long realLen; + + if (i % 2) + shouldInData =3D !shouldInData; + + if (virFileInData(fd, &realInData, &realLen) < 0) + goto cleanup; + + if (realInData !=3D shouldInData) { + fprintf(stderr, "Unexpected data/hole. Expected %s got %s\n", + shouldInData ? "data" : "hole", + realInData ? "data" : "hole"); + goto cleanup; + } + + shouldLen =3D data->offsets[i] * 1024; + if (realLen !=3D shouldLen) { + fprintf(stderr, "Unexpected section length. Expected %lld got = %lld\n", + shouldLen, realLen); + goto cleanup; + } + + if (lseek(fd, shouldLen, SEEK_CUR) < 0) { + fprintf(stderr, "Unable to seek\n"); + goto cleanup; + } + } + + ret =3D 0; + + cleanup: + VIR_FORCE_CLOSE(fd); + return ret; +} + + static int mymain(void) { @@ -186,6 +371,24 @@ mymain(void) DO_TEST_SANITIZE_PATH_SAME("gluster://bar.baz/fooo//hoo"); DO_TEST_SANITIZE_PATH_SAME("gluster://bar.baz/fooo///////hoo"); =20 +#define DO_TEST_IN_DATA(inData, ...) = \ + do { = \ + off_t offsets[] =3D {__VA_ARGS__, -1}; = \ + struct testFileInData data =3D { = \ + .startData =3D inData, .offsets =3D offsets, = \ + }; = \ + if (virTestRun(virTestCounterNext(), testFileInData, &data) < 0) = \ + ret =3D -1; = \ + } while (0) + + if (holesSupported()) { + DO_TEST_IN_DATA(true, 4, 4, 4); + DO_TEST_IN_DATA(false, 4, 4, 4); + DO_TEST_IN_DATA(true, 8, 8, 8); + DO_TEST_IN_DATA(false, 8, 8, 8); + DO_TEST_IN_DATA(true, 8, 16, 32, 64, 128, 256, 512); + DO_TEST_IN_DATA(false, 8, 16, 32, 64, 128, 256, 512); + } return ret !=3D 0 ? EXIT_FAILURE : EXIT_SUCCESS; } =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 149494347673888.51950077532774; Tue, 16 May 2017 07:04:36 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 65A2178ECB; Tue, 16 May 2017 14:04:29 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D358F860F9; Tue, 16 May 2017 14:04:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 682B3180BAF6; Tue, 16 May 2017 14:04:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4Lko027682 for ; Tue, 16 May 2017 10:04:22 -0400 Received: by smtp.corp.redhat.com (Postfix) id D5065860ED; Tue, 16 May 2017 14:04:21 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D06262926 for ; Tue, 16 May 2017 14:04:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 65A2178ECB Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 65A2178ECB From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:43 +0200 Message-Id: <5331d32201087cee4727172537285b778c36ae37.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 03/31] Introduce virStreamRecvFlags X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 16 May 2017 14:04:32 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch is adding the virStreamRecvFlags as a variant to the virStreamRecv function in order to allow for future expansion of functionality for processing sparse streams using a @flags argument. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- include/libvirt/libvirt-stream.h | 5 ++++ src/driver-stream.h | 7 +++++ src/libvirt-stream.c | 59 ++++++++++++++++++++++++++++++++++++= ++++ src/libvirt_public.syms | 5 ++++ 4 files changed, 76 insertions(+) diff --git a/include/libvirt/libvirt-stream.h b/include/libvirt/libvirt-str= eam.h index 831640d56..bee25168b 100644 --- a/include/libvirt/libvirt-stream.h +++ b/include/libvirt/libvirt-stream.h @@ -45,6 +45,11 @@ int virStreamRecv(virStreamPtr st, char *data, size_t nbytes); =20 +int virStreamRecvFlags(virStreamPtr st, + char *data, + size_t nbytes, + unsigned int flags); + =20 /** * virStreamSourceFunc: diff --git a/src/driver-stream.h b/src/driver-stream.h index 85b4e3bc7..d4b048018 100644 --- a/src/driver-stream.h +++ b/src/driver-stream.h @@ -35,6 +35,12 @@ typedef int char *data, size_t nbytes); =20 +typedef int +(*virDrvStreamRecvFlags)(virStreamPtr st, + char *data, + size_t nbytes, + unsigned int flags); + typedef int (*virDrvStreamEventAddCallback)(virStreamPtr stream, int events, @@ -61,6 +67,7 @@ typedef virStreamDriver *virStreamDriverPtr; struct _virStreamDriver { virDrvStreamSend streamSend; virDrvStreamRecv streamRecv; + virDrvStreamRecvFlags streamRecvFlags; virDrvStreamEventAddCallback streamEventAddCallback; virDrvStreamEventUpdateCallback streamEventUpdateCallback; virDrvStreamEventRemoveCallback streamEventRemoveCallback; diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c index 8384b3720..7535deb3c 100644 --- a/src/libvirt-stream.c +++ b/src/libvirt-stream.c @@ -285,6 +285,65 @@ virStreamRecv(virStreamPtr stream, } =20 =20 +/** + * virStreamRecvFlags: + * @stream: pointer to the stream object + * @data: buffer to read into from stream + * @nbytes: size of @data buffer + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Reads a series of bytes from the stream. This method may + * block the calling application for an arbitrary amount + * of time. + * + * This is just like virStreamRecv except this one has extra + * @flags. Calling this function with no @flags set (equal to + * zero) is equivalent to calling virStreamRecv(stream, data, nbytes). + * + * Returns 0 when the end of the stream is reached, at + * which time the caller should invoke virStreamFinish() + * to get confirmation of stream completion. + * + * Returns -1 upon error, at which time the stream will + * be marked as aborted, and the caller should now release + * the stream with virStreamFree. + * + * Returns -2 if there is no data pending to be read & the + * stream is marked as non-blocking. + */ +int +virStreamRecvFlags(virStreamPtr stream, + char *data, + size_t nbytes, + unsigned int flags) +{ + VIR_DEBUG("stream=3D%p, data=3D%p, nbytes=3D%zu flags=3D%x", + stream, data, nbytes, flags); + + virResetLastError(); + + virCheckStreamReturn(stream, -1); + virCheckNonNullArgGoto(data, error); + + if (stream->driver && + stream->driver->streamRecvFlags) { + int ret; + ret =3D (stream->driver->streamRecvFlags)(stream, data, nbytes, fl= ags); + if (ret =3D=3D -2) + return -2; + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(stream->conn); + return -1; +} + + /** * virStreamSendAll: * @stream: pointer to the stream object diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 428cf2e19..d50b36a24 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -759,4 +759,9 @@ LIBVIRT_3.1.0 { virDomainSetVcpu; } LIBVIRT_3.0.0; =20 +LIBVIRT_3.4.0 { + global: + virStreamRecvFlags; +} LIBVIRT_3.1.0; + # .... define new API here using predicted next version number .... --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943885415284.77691588101845; Tue, 16 May 2017 07:11:25 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D8985C059720; Tue, 16 May 2017 14:10:44 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4257A7D96A; Tue, 16 May 2017 14:10:36 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D926B4BB74; Tue, 16 May 2017 14:10:20 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4MNL027691 for ; Tue, 16 May 2017 10:04:23 -0400 Received: by smtp.corp.redhat.com (Postfix) id A4F3B860E6; Tue, 16 May 2017 14:04:22 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2DE8A62926 for ; Tue, 16 May 2017 14:04:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D8985C059720 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D8985C059720 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:44 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 04/31] Implement virStreamRecvFlags to some drivers X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 16 May 2017 14:11:21 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" There are three virStreamDriver's currently supported: * virFDStream * remote driver * ESX driver For now, backend virStreamRecvFlags support for only remote driver and ESX driver is sufficient. Future patches will update virFDStream. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/esx/esx_stream.c | 16 +++++++++++++++- src/remote/remote_driver.c | 21 +++++++++++++++++---- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/src/esx/esx_stream.c b/src/esx/esx_stream.c index fb9abbca6..b820b38ee 100644 --- a/src/esx/esx_stream.c +++ b/src/esx/esx_stream.c @@ -252,12 +252,17 @@ esxStreamSend(virStreamPtr stream, const char *data, = size_t nbytes) } =20 static int -esxStreamRecv(virStreamPtr stream, char *data, size_t nbytes) +esxStreamRecvFlags(virStreamPtr stream, + char *data, + size_t nbytes, + unsigned int flags) { int result =3D -1; esxStreamPrivate *priv =3D stream->privateData; int status; =20 + virCheckFlags(0, -1); + if (nbytes =3D=3D 0) return 0; =20 @@ -317,6 +322,14 @@ esxStreamRecv(virStreamPtr stream, char *data, size_t = nbytes) return result; } =20 +static int +esxStreamRecv(virStreamPtr stream, + char *data, + size_t nbytes) +{ + return esxStreamRecvFlags(stream, data, nbytes, 0); +} + static void esxFreeStreamPrivate(esxStreamPrivate **priv) { @@ -369,6 +382,7 @@ esxStreamAbort(virStreamPtr stream) virStreamDriver esxStreamDriver =3D { .streamSend =3D esxStreamSend, .streamRecv =3D esxStreamRecv, + .streamRecvFlags =3D esxStreamRecvFlags, /* FIXME: streamAddCallback missing */ /* FIXME: streamUpdateCallback missing */ /* FIXME: streamRemoveCallback missing */ diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 77250ea56..e79e796f2 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -5641,15 +5641,19 @@ remoteStreamSend(virStreamPtr st, =20 =20 static int -remoteStreamRecv(virStreamPtr st, - char *data, - size_t nbytes) +remoteStreamRecvFlags(virStreamPtr st, + char *data, + size_t nbytes, + unsigned int flags) { - VIR_DEBUG("st=3D%p data=3D%p nbytes=3D%zu", st, data, nbytes); + VIR_DEBUG("st=3D%p data=3D%p nbytes=3D%zu flags=3D%x", + st, data, nbytes, flags); struct private_data *priv =3D st->conn->privateData; virNetClientStreamPtr privst =3D st->privateData; int rv; =20 + virCheckFlags(0, -1); + if (virNetClientStreamRaiseError(privst)) return -1; =20 @@ -5671,6 +5675,14 @@ remoteStreamRecv(virStreamPtr st, return rv; } =20 +static int +remoteStreamRecv(virStreamPtr st, + char *data, + size_t nbytes) +{ + return remoteStreamRecvFlags(st, data, nbytes, 0); +} + struct remoteStreamCallbackData { virStreamPtr st; virStreamEventCallback cb; @@ -5843,6 +5855,7 @@ remoteStreamAbort(virStreamPtr st) =20 static virStreamDriver remoteStreamDrv =3D { .streamRecv =3D remoteStreamRecv, + .streamRecvFlags =3D remoteStreamRecvFlags, .streamSend =3D remoteStreamSend, .streamFinish =3D remoteStreamFinish, .streamAbort =3D remoteStreamAbort, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943701481557.7200002747174; Tue, 16 May 2017 07:08:21 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 52AEC796FE; Tue, 16 May 2017 14:07:42 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A47D276616; Tue, 16 May 2017 14:07:33 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C9E1A180BAFB; Tue, 16 May 2017 14:07:08 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4NIp027703 for ; Tue, 16 May 2017 10:04:23 -0400 Received: by smtp.corp.redhat.com (Postfix) id 74B48860F0; Tue, 16 May 2017 14:04:23 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id F1A0362926 for ; Tue, 16 May 2017 14:04:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 52AEC796FE Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 52AEC796FE From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:45 +0200 Message-Id: <44419eab5acbc2087472322d9af40ff3994c1b81.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 05/31] Introduce virStreamSendHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 16 May 2017 14:08:09 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This API is used to tell the other side of the stream to skip some bytes in the stream. This can be used to create a sparse file on the receiving side of a stream. It takes @length argument, which says how big the hole is. This skipping is done from the current point of stream. Since our streams are not rewindable like regular files, we don't need @whence argument like seek(2) has. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- include/libvirt/libvirt-stream.h | 4 +++ src/driver-stream.h | 6 ++++ src/libvirt-stream.c | 61 ++++++++++++++++++++++++++++++++++++= ++++ src/libvirt_public.syms | 1 + 4 files changed, 72 insertions(+) diff --git a/include/libvirt/libvirt-stream.h b/include/libvirt/libvirt-str= eam.h index bee25168b..14c9af142 100644 --- a/include/libvirt/libvirt-stream.h +++ b/include/libvirt/libvirt-stream.h @@ -50,6 +50,10 @@ int virStreamRecvFlags(virStreamPtr st, size_t nbytes, unsigned int flags); =20 +int virStreamSendHole(virStreamPtr st, + long long length, + unsigned int flags); + =20 /** * virStreamSourceFunc: diff --git a/src/driver-stream.h b/src/driver-stream.h index d4b048018..0a5201431 100644 --- a/src/driver-stream.h +++ b/src/driver-stream.h @@ -41,6 +41,11 @@ typedef int size_t nbytes, unsigned int flags); =20 +typedef int +(*virDrvStreamSendHole)(virStreamPtr st, + long long length, + unsigned int flags); + typedef int (*virDrvStreamEventAddCallback)(virStreamPtr stream, int events, @@ -68,6 +73,7 @@ struct _virStreamDriver { virDrvStreamSend streamSend; virDrvStreamRecv streamRecv; virDrvStreamRecvFlags streamRecvFlags; + virDrvStreamSendHole streamSendHole; virDrvStreamEventAddCallback streamEventAddCallback; virDrvStreamEventUpdateCallback streamEventUpdateCallback; virDrvStreamEventRemoveCallback streamEventRemoveCallback; diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c index 7535deb3c..a09896dcd 100644 --- a/src/libvirt-stream.c +++ b/src/libvirt-stream.c @@ -344,6 +344,67 @@ virStreamRecvFlags(virStreamPtr stream, } =20 =20 +/** + * virStreamSendHole: + * @stream: pointer to the stream object + * @length: number of bytes to skip + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Rather than transmitting empty file space, this API directs + * the @stream target to create @length bytes of empty space. + * This API would be used when uploading or downloading sparsely + * populated files to avoid the needless copy of empty file + * space. + * + * An example using this with a hypothetical file upload API + * looks like: + * + * virStream st; + * + * while (1) { + * char buf[4096]; + * size_t len; + * if (..in hole...) { + * ..get hole size... + * virStreamSendHole(st, len, 0); + * } else { + * ...read len bytes... + * virStreamSend(st, buf, len); + * } + * } + * + * Returns 0 on success, + * -1 error + */ +int +virStreamSendHole(virStreamPtr stream, + long long length, + unsigned int flags) +{ + VIR_DEBUG("stream=3D%p, length=3D%lld flags=3D%x", + stream, length, flags); + + virResetLastError(); + + virCheckStreamReturn(stream, -1); + + if (stream->driver && + stream->driver->streamSendHole) { + int ret; + ret =3D (stream->driver->streamSendHole)(stream, length, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(stream->conn); + return -1; +} + + /** * virStreamSendAll: * @stream: pointer to the stream object diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index d50b36a24..3be7cc6a0 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -762,6 +762,7 @@ LIBVIRT_3.1.0 { LIBVIRT_3.4.0 { global: virStreamRecvFlags; + virStreamSendHole; } LIBVIRT_3.1.0; =20 # .... define new API here using predicted next version number .... --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944102813568.9713268002399; Tue, 16 May 2017 07:15:02 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A2DB6804F1; Tue, 16 May 2017 14:14:08 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4731C7FB6C; Tue, 16 May 2017 14:13:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0307D180BAF1; Tue, 16 May 2017 14:13:38 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4OaM027712 for ; Tue, 16 May 2017 10:04:24 -0400 Received: by smtp.corp.redhat.com (Postfix) id 45213860E9; Tue, 16 May 2017 14:04:24 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id C15C1860E6 for ; Tue, 16 May 2017 14:04:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A2DB6804F1 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A2DB6804F1 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:46 +0200 Message-Id: <2f9f360e5fd90323e64863e5eb0a0a12f8926e36.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 06/31] Introduce virStreamRecvHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 16 May 2017 14:15:00 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This function is basically a counterpart for virStreamSendHole(). If one side of a stream called virStreamSendHole() the other should call virStreamRecvHole() to get the size of the hole. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- include/libvirt/libvirt-stream.h | 4 ++++ src/driver-stream.h | 6 ++++++ src/libvirt-stream.c | 44 ++++++++++++++++++++++++++++++++++++= ++++ src/libvirt_public.syms | 1 + 4 files changed, 55 insertions(+) diff --git a/include/libvirt/libvirt-stream.h b/include/libvirt/libvirt-str= eam.h index 14c9af142..feaa8ad64 100644 --- a/include/libvirt/libvirt-stream.h +++ b/include/libvirt/libvirt-stream.h @@ -54,6 +54,10 @@ int virStreamSendHole(virStreamPtr st, long long length, unsigned int flags); =20 +int virStreamRecvHole(virStreamPtr, + long long *length, + unsigned int flags); + =20 /** * virStreamSourceFunc: diff --git a/src/driver-stream.h b/src/driver-stream.h index 0a5201431..0fb56ebd2 100644 --- a/src/driver-stream.h +++ b/src/driver-stream.h @@ -46,6 +46,11 @@ typedef int long long length, unsigned int flags); =20 +typedef int +(*virDrvStreamRecvHole)(virStreamPtr st, + long long *length, + unsigned int flags); + typedef int (*virDrvStreamEventAddCallback)(virStreamPtr stream, int events, @@ -74,6 +79,7 @@ struct _virStreamDriver { virDrvStreamRecv streamRecv; virDrvStreamRecvFlags streamRecvFlags; virDrvStreamSendHole streamSendHole; + virDrvStreamRecvHole streamRecvHole; virDrvStreamEventAddCallback streamEventAddCallback; virDrvStreamEventUpdateCallback streamEventUpdateCallback; virDrvStreamEventRemoveCallback streamEventRemoveCallback; diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c index a09896dcd..dc0dc9ea3 100644 --- a/src/libvirt-stream.c +++ b/src/libvirt-stream.c @@ -405,6 +405,50 @@ virStreamSendHole(virStreamPtr stream, } =20 =20 +/** + * virStreamRecvHole: + * @stream: pointer to the stream object + * @length: number of bytes to skip + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * This API is used to determine the @length in bytes of the + * empty space to be created in a @stream's target file when + * uploading or downloading sparsely populated files. This is the + * counterpart to virStreamSendHole(). + * + * Returns 0 on success, + * -1 on error or when there's currently no hole in the stream + */ +int +virStreamRecvHole(virStreamPtr stream, + long long *length, + unsigned int flags) +{ + VIR_DEBUG("stream=3D%p, length=3D%p flags=3D%x", + stream, length, flags); + + virResetLastError(); + + virCheckStreamReturn(stream, -1); + virCheckNonNullArgReturn(length, -1); + + if (stream->driver && + stream->driver->streamRecvHole) { + int ret; + ret =3D (stream->driver->streamRecvHole)(stream, length, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(stream->conn); + return -1; +} + + /** * virStreamSendAll: * @stream: pointer to the stream object diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 3be7cc6a0..b73cc8af1 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -762,6 +762,7 @@ LIBVIRT_3.1.0 { LIBVIRT_3.4.0 { global: virStreamRecvFlags; + virStreamRecvHole; virStreamSendHole; } LIBVIRT_3.1.0; =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943503594956.7899715264595; Tue, 16 May 2017 07:05:03 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 04EFC4E4C4; Tue, 16 May 2017 14:04:38 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 82D967D972; Tue, 16 May 2017 14:04:37 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EA307180BAFA; Tue, 16 May 2017 14:04:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4Pqt027717 for ; Tue, 16 May 2017 10:04:25 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2776E860E0; Tue, 16 May 2017 14:04:25 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id A414162926 for ; Tue, 16 May 2017 14:04:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 04EFC4E4C4 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 04EFC4E4C4 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:47 +0200 Message-Id: <36978f89ec0a65dbcb8b0069dd9e8a51bd172180.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 07/31] Introduce VIR_STREAM_RECV_STOP_AT_HOLE flag X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 16 May 2017 14:04:59 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add a new flag to virStreamRecvFlags in order to handle being able to stop reading from the stream so that the consumer can generate a "hole" in stream target. Generation of a hole replaces the need to receive and handle a sequence of zero bytes for sparse stream targets. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- include/libvirt/libvirt-stream.h | 4 ++++ src/libvirt-stream.c | 34 +++++++++++++++++++++++++++++++++- 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/include/libvirt/libvirt-stream.h b/include/libvirt/libvirt-str= eam.h index feaa8ad64..c4baaf7a3 100644 --- a/include/libvirt/libvirt-stream.h +++ b/include/libvirt/libvirt-stream.h @@ -45,6 +45,10 @@ int virStreamRecv(virStreamPtr st, char *data, size_t nbytes); =20 +typedef enum { + VIR_STREAM_RECV_STOP_AT_HOLE =3D (1 << 0), +} virStreamRecvFlagsValues; + int virStreamRecvFlags(virStreamPtr st, char *data, size_t nbytes, diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c index dc0dc9ea3..bedb6159a 100644 --- a/src/libvirt-stream.c +++ b/src/libvirt-stream.c @@ -290,7 +290,7 @@ virStreamRecv(virStreamPtr stream, * @stream: pointer to the stream object * @data: buffer to read into from stream * @nbytes: size of @data buffer - * @flags: extra flags; not used yet, so callers should always pass 0 + * @flags: bitwise-OR of virStreamRecvFlagsValues * * Reads a series of bytes from the stream. This method may * block the calling application for an arbitrary amount @@ -300,6 +300,33 @@ virStreamRecv(virStreamPtr stream, * @flags. Calling this function with no @flags set (equal to * zero) is equivalent to calling virStreamRecv(stream, data, nbytes). * + * If flag VIR_STREAM_RECV_STOP_AT_HOLE is set, this function + * will stop reading from stream if it has reached a hole. In + * that case, -3 is returned and virStreamRecvHole() should be + * called to get the hole size. An example using this flag might + * look like this: + * + * while (1) { + * char buf[4096]; + * + * int ret =3D virStreamRecvFlags(st, buf, len, VIR_STREAM_STOP_AT_HOL= E); + * if (ret < 0) { + * if (ret =3D=3D -3) { + * long long len; + * ret =3D virStreamRecvHole(st, &len, 0); + * if (ret < 0) { + * ...error.. + * } else { + * ...seek len bytes in target... + * } + * } else { + * return -1; + * } + * } else { + * ...write buf to target... + * } + * } + * * Returns 0 when the end of the stream is reached, at * which time the caller should invoke virStreamFinish() * to get confirmation of stream completion. @@ -310,6 +337,9 @@ virStreamRecv(virStreamPtr stream, * * Returns -2 if there is no data pending to be read & the * stream is marked as non-blocking. + * + * Returns -3 if there is a hole in stream and caller requested + * to stop at a hole. */ int virStreamRecvFlags(virStreamPtr stream, @@ -331,6 +361,8 @@ virStreamRecvFlags(virStreamPtr stream, ret =3D (stream->driver->streamRecvFlags)(stream, data, nbytes, fl= ags); if (ret =3D=3D -2) return -2; + if (ret =3D=3D -3) + return -3; if (ret < 0) goto error; return ret; --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944301832124.67771702425; Tue, 16 May 2017 07:18:21 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 91DE042BCE; Tue, 16 May 2017 14:17:39 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A41C76292C; Tue, 16 May 2017 14:17:31 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EB76D180BAF6; Tue, 16 May 2017 14:17:11 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4Qjd027751 for ; Tue, 16 May 2017 10:04:26 -0400 Received: by smtp.corp.redhat.com (Postfix) id F23C5860E4; Tue, 16 May 2017 14:04:25 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7A42F860E0 for ; Tue, 16 May 2017 14:04:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 91DE042BCE Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 91DE042BCE From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:48 +0200 Message-Id: <435d9f8dd25108ff907c04d215b2936dec525241.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 08/31] Introduce virStreamSparseRecvAll X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 16 May 2017 14:18:06 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is just a wrapper over new functions that have been just introduced: virStreamRecvFlags(), virStreamRecvHole(). It's very similar to virStreamRecvAll() except it handles sparse streams well. Signed-off-by: Michal Privoznik --- include/libvirt/libvirt-stream.h | 33 ++++++++++- src/libvirt-stream.c | 123 +++++++++++++++++++++++++++++++++++= ++++ src/libvirt_public.syms | 1 + 3 files changed, 154 insertions(+), 3 deletions(-) diff --git a/include/libvirt/libvirt-stream.h b/include/libvirt/libvirt-str= eam.h index c4baaf7a3..a5e69a1c1 100644 --- a/include/libvirt/libvirt-stream.h +++ b/include/libvirt/libvirt-stream.h @@ -104,9 +104,9 @@ int virStreamSendAll(virStreamPtr st, * @nbytes: size of the data array * @opaque: optional application provided data * - * The virStreamSinkFunc callback is used together - * with the virStreamRecvAll function for libvirt to - * provide the data that has been received. + * The virStreamSinkFunc callback is used together with the + * virStreamRecvAll or virStreamSparseRecvAll functions for + * libvirt to provide the data that has been received. * * The callback will be invoked multiple times, * providing data in small chunks. The application @@ -129,6 +129,33 @@ int virStreamRecvAll(virStreamPtr st, virStreamSinkFunc handler, void *opaque); =20 +/** + * virStreamSinkHoleFunc: + * @st: the stream object + * @length: stream hole size + * @opaque: optional application provided data + * + * This callback is used together with the virStreamSparseRecvAll + * function for libvirt to provide the size of a hole that + * occurred in the stream. + * + * The callback may be invoked multiple times as holes are found + * during processing a stream. The application should create the + * hole in the stream target and then return. A return value of + * -1 at any time will abort the receive operation. + * + * Returns 0 on success, + * -1 upon error + */ +typedef int (*virStreamSinkHoleFunc)(virStreamPtr st, + long long length, + void *opaque); + +int virStreamSparseRecvAll(virStreamPtr stream, + virStreamSinkFunc handler, + virStreamSinkHoleFunc holeHandler, + void *opaque); + typedef enum { VIR_STREAM_EVENT_READABLE =3D (1 << 0), VIR_STREAM_EVENT_WRITABLE =3D (1 << 1), diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c index bedb6159a..6bf4c4f29 100644 --- a/src/libvirt-stream.c +++ b/src/libvirt-stream.c @@ -668,6 +668,129 @@ virStreamRecvAll(virStreamPtr stream, } =20 =20 +/** + * virStreamSparseRecvAll: + * @stream: pointer to the stream object + * @handler: sink callback for writing data to application + * @holeHandler: stream hole callback for skipping holes + * @opaque: application defined data + * + * Receive the entire data stream, sending the data to the + * requested data sink @handler and calling the skip @holeHandler + * to generate holes for sparse stream targets. This is simply a + * convenient alternative to virStreamRecvFlags, for apps that do + * blocking-I/O. + * + * An example using this with a hypothetical file download + * API looks like: + * + * int mysink(virStreamPtr st, const char *buf, int nbytes, void *opaque= ) { + * int *fd =3D opaque; + * + * return write(*fd, buf, nbytes); + * } + * + * int myskip(virStreamPtr st, long long offset, void *opaque) { + * int *fd =3D opaque; + * + * return lseek(*fd, offset, SEEK_CUR) =3D=3D (off_t) -1 ? -1 : 0; + * } + * + * virStreamPtr st =3D virStreamNew(conn, 0); + * int fd =3D open("demo.iso", O_WRONLY); + * + * virConnectDownloadSparseFile(conn, st); + * if (virStreamSparseRecvAll(st, mysink, myskip, &fd) < 0) { + * ...report an error ... + * goto done; + * } + * if (virStreamFinish(st) < 0) + * ...report an error... + * virStreamFree(st); + * close(fd); + * + * Note that @opaque data is shared between both @handler and + * @holeHandler callbacks. + * + * Returns 0 if all the data was successfully received. The caller + * should invoke virStreamFinish(st) to flush the stream upon + * success and then virStreamFree(st). + * + * Returns -1 upon any error, with virStreamAbort() already + * having been called, so the caller need only call virStreamFree(). + */ +int +virStreamSparseRecvAll(virStreamPtr stream, + virStreamSinkFunc handler, + virStreamSinkHoleFunc holeHandler, + void *opaque) +{ + char *bytes =3D NULL; + size_t want =3D VIR_NET_MESSAGE_LEGACY_PAYLOAD_MAX; + const unsigned int flags =3D VIR_STREAM_RECV_STOP_AT_HOLE; + int ret =3D -1; + + VIR_DEBUG("stream=3D%p handler=3D%p holeHandler=3D%p opaque=3D%p", + stream, handler, holeHandler, opaque); + + virResetLastError(); + + virCheckStreamReturn(stream, -1); + virCheckNonNullArgGoto(handler, cleanup); + virCheckNonNullArgGoto(holeHandler, cleanup); + + if (stream->flags & VIR_STREAM_NONBLOCK) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("data sinks cannot be used for non-blocking strea= ms")); + goto cleanup; + } + + if (VIR_ALLOC_N(bytes, want) < 0) + goto cleanup; + + for (;;) { + int got, offset =3D 0; + long long holeLen; + const unsigned int holeFlags =3D 0; + + got =3D virStreamRecvFlags(stream, bytes, want, flags); + if (got =3D=3D -3) { + if (virStreamRecvHole(stream, &holeLen, holeFlags) < 0) { + virStreamAbort(stream); + goto cleanup; + } + + if (holeHandler(stream, holeLen, opaque) < 0) { + virStreamAbort(stream); + goto cleanup; + } + continue; + } else if (got < 0) { + goto cleanup; + } else if (got =3D=3D 0) { + break; + } + while (offset < got) { + int done; + done =3D (handler)(stream, bytes + offset, got - offset, opaqu= e); + if (done < 0) { + virStreamAbort(stream); + goto cleanup; + } + offset +=3D done; + } + } + ret =3D 0; + + cleanup: + VIR_FREE(bytes); + + if (ret !=3D 0) + virDispatchError(stream->conn); + + return ret; +} + /** * virStreamEventAddCallback: * @stream: pointer to the stream object diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index b73cc8af1..37fc4e224 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -764,6 +764,7 @@ LIBVIRT_3.4.0 { virStreamRecvFlags; virStreamRecvHole; virStreamSendHole; + virStreamSparseRecvAll; } LIBVIRT_3.1.0; =20 # .... define new API here using predicted next version number .... --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944573121140.9773491063579; Tue, 16 May 2017 07:22:53 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 28CEF7F7DE; Tue, 16 May 2017 14:21:59 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0C0388F638; Tue, 16 May 2017 14:21:44 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 841614A48D; Tue, 16 May 2017 14:21:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4QnE027768 for ; Tue, 16 May 2017 10:04:26 -0400 Received: by smtp.corp.redhat.com (Postfix) id C2481860E9; Tue, 16 May 2017 14:04:26 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B2AA860E4 for ; Tue, 16 May 2017 14:04:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 28CEF7F7DE Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 28CEF7F7DE From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:49 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 09/31] Introduce virStreamSparseSendAll X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 16 May 2017 14:22:51 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is just a wrapper over new function that have been just introduced: virStreamSendHole() . It's very similar to virStreamSendAll() except it handles sparse streams well. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- include/libvirt/libvirt-stream.h | 65 +++++++++++++++- src/libvirt-stream.c | 159 +++++++++++++++++++++++++++++++++++= ++++ src/libvirt_public.syms | 1 + 3 files changed, 222 insertions(+), 3 deletions(-) diff --git a/include/libvirt/libvirt-stream.h b/include/libvirt/libvirt-str= eam.h index a5e69a1c1..d18d43140 100644 --- a/include/libvirt/libvirt-stream.h +++ b/include/libvirt/libvirt-stream.h @@ -71,9 +71,9 @@ int virStreamRecvHole(virStreamPtr, * @nbytes: size of the data array * @opaque: optional application provided data * - * The virStreamSourceFunc callback is used together - * with the virStreamSendAll function for libvirt to - * obtain the data that is to be sent. + * The virStreamSourceFunc callback is used together with + * the virStreamSendAll and virStreamSparseSendAll functions + * for libvirt to obtain the data that is to be sent. * * The callback will be invoked multiple times, * fetching data in small chunks. The application @@ -96,6 +96,65 @@ int virStreamSendAll(virStreamPtr st, virStreamSourceFunc handler, void *opaque); =20 +/** + * virStreamSourceHoleFunc: + * @st: the stream object + * @inData: are we in data section + * @length: how long is the section we are currently in + * @opaque: optional application provided data + * + * The virStreamSourceHoleFunc callback is used together with the + * virStreamSparseSendAll function for libvirt to obtain the + * length of section stream is currently in. + * + * Moreover, upon successful return, @length should be updated + * with how many bytes are left until the current section ends + * (either data section or hole section). Also the stream is + * currently in data section, @inData should be set to a non-zero + * value and vice versa. + * + * NB: there's an implicit hole at the end of each file. If + * that's the case, @inData and @length should be both set to 0. + * + * This function should not adjust the current position within + * the file. + * + * Returns 0 on success, + * -1 upon error + */ +typedef int (*virStreamSourceHoleFunc)(virStreamPtr st, + int *inData, + long long *length, + void *opaque); + +/** + * virStreamSourceSkipFunc: + * @st: the stream object + * @length: stream hole size + * @opaque: optional application provided data + * + * This callback is used together with the virStreamSparseSendAll + * to skip holes in the underlying file as reported by + * virStreamSourceHoleFunc. + * + * The callback may be invoked multiple times as holes are found + * during processing a stream. The application should skip + * processing the hole in the stream source and then return. + * A return value of -1 at any time will abort the send operation. + * + * Returns 0 on success, + * -1 upon error. + */ +typedef int (*virStreamSourceSkipFunc)(virStreamPtr st, + long long length, + void *opaque); + +int virStreamSparseSendAll(virStreamPtr st, + virStreamSourceFunc handler, + virStreamSourceHoleFunc holeHandler, + virStreamSourceSkipFunc skipHandler, + void *opaque); + /** * virStreamSinkFunc: * diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c index 6bf4c4f29..4cbe5eee1 100644 --- a/src/libvirt-stream.c +++ b/src/libvirt-stream.c @@ -574,6 +574,165 @@ virStreamSendAll(virStreamPtr stream, } =20 =20 +/** + * virStreamSparseSendAll: + * @stream: pointer to the stream object + * @handler: source callback for reading data from application + * @holeHandler: source callback for determining holes + * @skipHandler: skip holes as reported by @holeHandler + * @opaque: application defined data + * + * Send the entire data stream, reading the data from the + * requested data source. This is simply a convenient alternative + * to virStreamSend, for apps that do blocking-I/O. + * + * An example using this with a hypothetical file upload + * API looks like + * + * int mysource(virStreamPtr st, char *buf, int nbytes, void *opaque) { + * int *fd =3D opaque; + * + * return read(*fd, buf, nbytes); + * } + * + * int myskip(virStreamPtr st, long long offset, void *opaque) { + * int *fd =3D opaque; + * + * return lseek(*fd, offset, SEEK_CUR) =3D=3D (off_t) -1 ? -1 : 0; + * } + * + * int myindata(virStreamPtr st, int *inData, + * long long *offset, void *opaque) { + * int *fd =3D opaque; + * + * if (@fd in hole) { + * *inData =3D 0; + * *offset =3D holeSize; + * } else { + * *inData =3D 1; + * *offset =3D dataSize; + * } + * + * return 0; + * } + * + * virStreamPtr st =3D virStreamNew(conn, 0); + * int fd =3D open("demo.iso", O_RDONLY); + * + * virConnectUploadFile(conn, st); + * if (virStreamSparseSendAll(st, + * mysource, + * myindata, + * myskip, + * &fd) < 0) { + * ...report an error ... + * goto done; + * } + * if (virStreamFinish(st) < 0) + * ...report an error... + * virStreamFree(st); + * close(fd); + * + * Note that @opaque data are shared between @handler, @holeHandler and @s= kipHandler. + * + * Returns 0 if all the data was successfully sent. The caller + * should invoke virStreamFinish(st) to flush the stream upon + * success and then virStreamFree. + * + * Returns -1 upon any error, with virStreamAbort() already + * having been called, so the caller need only call + * virStreamFree(). + */ +int virStreamSparseSendAll(virStreamPtr stream, + virStreamSourceFunc handler, + virStreamSourceHoleFunc holeHandler, + virStreamSourceSkipFunc skipHandler, + void *opaque) +{ + char *bytes =3D NULL; + size_t want =3D VIR_NET_MESSAGE_LEGACY_PAYLOAD_MAX; + int ret =3D -1; + unsigned long long dataLen =3D 0; + + VIR_DEBUG("stream=3D%p handler=3D%p holeHandler=3D%p opaque=3D%p", + stream, handler, holeHandler, opaque); + + virResetLastError(); + + virCheckStreamReturn(stream, -1); + virCheckNonNullArgGoto(handler, cleanup); + virCheckNonNullArgGoto(holeHandler, cleanup); + virCheckNonNullArgGoto(skipHandler, cleanup); + + if (stream->flags & VIR_STREAM_NONBLOCK) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("data sources cannot be used for non-blocking str= eams")); + goto cleanup; + } + + if (VIR_ALLOC_N(bytes, want) < 0) + goto cleanup; + + for (;;) { + int inData, got, offset =3D 0; + long long sectionLen; + const unsigned int skipFlags =3D 0; + + if (!dataLen) { + if (holeHandler(stream, &inData, §ionLen, opaque) < 0) { + virStreamAbort(stream); + goto cleanup; + } + + if (!inData && sectionLen) { + if (virStreamSendHole(stream, sectionLen, skipFlags) < 0) { + virStreamAbort(stream); + goto cleanup; + } + + if (skipHandler(stream, sectionLen, opaque) < 0) { + virReportSystemError(errno, "%s", + _("unable to skip hole")); + virStreamAbort(stream); + goto cleanup; + } + continue; + } else { + dataLen =3D sectionLen; + } + } + + if (want > dataLen) + want =3D dataLen; + + got =3D (handler)(stream, bytes, want, opaque); + if (got < 0) { + virStreamAbort(stream); + goto cleanup; + } + if (got =3D=3D 0) + break; + while (offset < got) { + int done; + done =3D virStreamSend(stream, bytes + offset, got - offset); + if (done < 0) + goto cleanup; + offset +=3D done; + dataLen -=3D done; + } + } + ret =3D 0; + + cleanup: + VIR_FREE(bytes); + + if (ret !=3D 0) + virDispatchError(stream->conn); + + return ret; +} + + /** * virStreamRecvAll: * @stream: pointer to the stream object diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 37fc4e224..fac77fbea 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -765,6 +765,7 @@ LIBVIRT_3.4.0 { virStreamRecvHole; virStreamSendHole; virStreamSparseRecvAll; + virStreamSparseSendAll; } LIBVIRT_3.1.0; =20 # .... define new API here using predicted next version number .... --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943702011226.37245247059866; Tue, 16 May 2017 07:08:22 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1BE6E8AE7F; Tue, 16 May 2017 14:07:24 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8E4D36292C; Tue, 16 May 2017 14:07:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id DB657180BAF6; Tue, 16 May 2017 14:06:59 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4Rit027782 for ; Tue, 16 May 2017 10:04:27 -0400 Received: by smtp.corp.redhat.com (Postfix) id BB100860E4; Tue, 16 May 2017 14:04:27 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43E4C860E0 for ; Tue, 16 May 2017 14:04:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1BE6E8AE7F Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1BE6E8AE7F From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:50 +0200 Message-Id: <825911d1e6227f75a8ac5bf86ac444bc4b9f3d40.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 10/31] Introduce virStreamInData X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 16 May 2017 14:08:06 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is just an internal API, that calls corresponding function in stream driver. This function will set @data=3D1 if the underlying file is in data section, or @data=3D0 if it is in a hole. At any rate, @length is set to number of bytes remaining in the section the file currently is. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/driver-stream.h | 6 ++++++ src/libvirt-stream.c | 48 ++++++++++++++++++++++++++++++++++++++++++++= ++++ src/libvirt_internal.h | 4 ++++ src/libvirt_private.syms | 1 + 4 files changed, 59 insertions(+) diff --git a/src/driver-stream.h b/src/driver-stream.h index 0fb56ebd2..f207bf0eb 100644 --- a/src/driver-stream.h +++ b/src/driver-stream.h @@ -51,6 +51,11 @@ typedef int long long *length, unsigned int flags); =20 +typedef int +(*virDrvStreamInData)(virStreamPtr st, + int *data, + long long *length); + typedef int (*virDrvStreamEventAddCallback)(virStreamPtr stream, int events, @@ -80,6 +85,7 @@ struct _virStreamDriver { virDrvStreamRecvFlags streamRecvFlags; virDrvStreamSendHole streamSendHole; virDrvStreamRecvHole streamRecvHole; + virDrvStreamInData streamInData; virDrvStreamEventAddCallback streamEventAddCallback; virDrvStreamEventUpdateCallback streamEventUpdateCallback; virDrvStreamEventRemoveCallback streamEventRemoveCallback; diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c index 4cbe5eee1..30c305035 100644 --- a/src/libvirt-stream.c +++ b/src/libvirt-stream.c @@ -481,6 +481,54 @@ virStreamRecvHole(virStreamPtr stream, } =20 =20 +/** + * virStreamInData: + * @stream: stream + * @data: are we in data or hole + * @length: length to next section + * + * This function checks the underlying stream (typically a file) + * to learn whether the current stream position lies within a + * data section or a hole. Upon return @data is set to a nonzero + * value if former is the case, or to zero if @stream is in a + * hole. Moreover, @length is updated to tell caller how many + * bytes can be read from @stream until current section changes + * (from data to a hole or vice versa). + * + * NB: there's an implicit hole at EOF. In this situation this + * function should set @data =3D false, @length =3D 0 and return 0. + * + * To sum it up: + * + * data section: @data =3D true, @length > 0 + * hole: @data =3D false, @length > 0 + * EOF: @data =3D false, @length =3D 0 + * + * Returns 0 on success, + * -1 otherwise + */ +int +virStreamInData(virStreamPtr stream, + int *data, + long long *length) +{ + VIR_DEBUG("stream=3D%p, data=3D%p, length=3D%p", stream, data, length); + + /* No checks of arguments or error resetting. This is an + * internal function that just happen to live next to some + * public functions of ours. */ + + if (stream->driver->streamInData) { + int ret; + ret =3D (stream->driver->streamInData)(stream, data, length); + return ret; + } + + virReportUnsupportedError(); + return -1; +} + + /** * virStreamSendAll: * @stream: pointer to the stream object diff --git a/src/libvirt_internal.h b/src/libvirt_internal.h index 96439d840..62f490a7d 100644 --- a/src/libvirt_internal.h +++ b/src/libvirt_internal.h @@ -294,4 +294,8 @@ virTypedParameterValidateSet(virConnectPtr conn, virTypedParameterPtr params, int nparams); =20 +int virStreamInData(virStreamPtr stream, + int *data, + long long *length); + #endif diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 4102a002b..a1447eb44 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1124,6 +1124,7 @@ virStateCleanup; virStateInitialize; virStateReload; virStateStop; +virStreamInData; =20 =20 # locking/domain_lock.h --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943879562977.9178274534931; Tue, 16 May 2017 07:11:19 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C40E25AFED; Tue, 16 May 2017 14:10:40 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4F24AB0E43; Tue, 16 May 2017 14:10:38 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 419B74BB7F; Tue, 16 May 2017 14:10:18 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4SJB027797 for ; Tue, 16 May 2017 10:04:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id 91FA96292C; Tue, 16 May 2017 14:04:28 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 18745860E0 for ; Tue, 16 May 2017 14:04:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C40E25AFED Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C40E25AFED From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:51 +0200 Message-Id: <17c079f1f27bcbc1da8d50886fb9c1fc9426e98c.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 11/31] virNetClientStreamNew: Track origin stream X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 16 May 2017 14:11:13 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add a virStreamPtr pointer to the _virNetClientStream in order to reverse track the parent stream. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/remote/remote_driver.c | 6 ++++-- src/rpc/gendispatch.pl | 2 +- src/rpc/virnetclientstream.c | 6 +++++- src/rpc/virnetclientstream.h | 3 ++- 4 files changed, 12 insertions(+), 5 deletions(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index e79e796f2..b152be523 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -6170,7 +6170,8 @@ remoteDomainMigratePrepareTunnel3(virConnectPtr dconn, memset(&args, 0, sizeof(args)); memset(&ret, 0, sizeof(ret)); =20 - if (!(netst =3D virNetClientStreamNew(priv->remoteProgram, + if (!(netst =3D virNetClientStreamNew(st, + priv->remoteProgram, REMOTE_PROC_DOMAIN_MIGRATE_PREPARE= _TUNNEL3, priv->counter))) goto done; @@ -7094,7 +7095,8 @@ remoteDomainMigratePrepareTunnel3Params(virConnectPtr= dconn, goto cleanup; } =20 - if (!(netst =3D virNetClientStreamNew(priv->remoteProgram, + if (!(netst =3D virNetClientStreamNew(st, + priv->remoteProgram, REMOTE_PROC_DOMAIN_MIGRATE_PREPARE= _TUNNEL3_PARAMS, priv->counter))) goto cleanup; diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 173189c81..e608812ce 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -1738,7 +1738,7 @@ elsif ($mode eq "client") { =20 if ($call->{streamflag} ne "none") { print "\n"; - print " if (!(netst =3D virNetClientStreamNew(priv->remoteP= rogram, $call->{constname}, priv->counter)))\n"; + print " if (!(netst =3D virNetClientStreamNew(st, priv->rem= oteProgram, $call->{constname}, priv->counter)))\n"; print " goto done;\n"; print "\n"; print " if (virNetClientAddStream(priv->client, netst) < 0)= {\n"; diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index 2105bd0a9..01761cf8d 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -36,6 +36,8 @@ VIR_LOG_INIT("rpc.netclientstream"); struct _virNetClientStream { virObjectLockable parent; =20 + virStreamPtr stream; /* Reverse pointer to parent stream */ + virNetClientProgramPtr prog; int proc; unsigned serial; @@ -133,7 +135,8 @@ virNetClientStreamEventTimer(int timer ATTRIBUTE_UNUSED= , void *opaque) } =20 =20 -virNetClientStreamPtr virNetClientStreamNew(virNetClientProgramPtr prog, +virNetClientStreamPtr virNetClientStreamNew(virStreamPtr stream, + virNetClientProgramPtr prog, int proc, unsigned serial) { @@ -145,6 +148,7 @@ virNetClientStreamPtr virNetClientStreamNew(virNetClien= tProgramPtr prog, if (!(st =3D virObjectLockableNew(virNetClientStreamClass))) return NULL; =20 + st->stream =3D stream; st->prog =3D virObjectRef(prog); st->proc =3D proc; st->serial =3D serial; diff --git a/src/rpc/virnetclientstream.h b/src/rpc/virnetclientstream.h index a0d2be9ed..e278dab85 100644 --- a/src/rpc/virnetclientstream.h +++ b/src/rpc/virnetclientstream.h @@ -32,7 +32,8 @@ typedef virNetClientStream *virNetClientStreamPtr; typedef void (*virNetClientStreamEventCallback)(virNetClientStreamPtr stre= am, int events, void *opaque); =20 -virNetClientStreamPtr virNetClientStreamNew(virNetClientProgramPtr prog, +virNetClientStreamPtr virNetClientStreamNew(virStreamPtr stream, + virNetClientProgramPtr prog, int proc, unsigned serial); =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944057027379.50536324637756; Tue, 16 May 2017 07:14:17 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B01DC80486; Tue, 16 May 2017 14:13:47 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5D4C1860F4; Tue, 16 May 2017 14:13:33 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C7DBB180BAF7; Tue, 16 May 2017 14:13:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4Tui027802 for ; Tue, 16 May 2017 10:04:29 -0400 Received: by smtp.corp.redhat.com (Postfix) id 612FF860EE; Tue, 16 May 2017 14:04:29 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id DDEA4860F7 for ; Tue, 16 May 2017 14:04:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B01DC80486 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B01DC80486 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:52 +0200 Message-Id: <3b141cd15dd38298150844b23aed1863e53a9d3b.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 12/31] Add new flag to daemonCreateClientStream and virNetClientStreamNew X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 16 May 2017 14:14:15 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add a new argument to daemonCreateClientStream in order to allow for future expansion to mark that a specific stream can be used to skip data, such as the case with sparsely populated files. The new flag will be the eventual decision point between client/server to decide whether both ends can support and want to use sparse streams. A new bool 'allowSkip' is added to both _virNetClientStream and daemonClientStream in order to perform the tracking. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- daemon/remote.c | 2 +- daemon/stream.c | 6 +++++- daemon/stream.h | 3 ++- src/remote/remote_driver.c | 6 ++++-- src/rpc/gendispatch.pl | 4 ++-- src/rpc/virnetclientstream.c | 6 +++++- src/rpc/virnetclientstream.h | 3 ++- 7 files changed, 21 insertions(+), 9 deletions(-) diff --git a/daemon/remote.c b/daemon/remote.c index 0dbb250ff..fd8542120 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -5323,7 +5323,7 @@ remoteDispatchDomainMigratePrepareTunnel3Params(virNe= tServerPtr server ATTRIBUTE =20 if (!(st =3D virStreamNew(priv->conn, VIR_STREAM_NONBLOCK)) || !(stream =3D daemonCreateClientStream(client, st, remoteProgram, - &msg->header))) + &msg->header, false))) goto cleanup; =20 if (virDomainMigratePrepareTunnel3Params(priv->conn, st, params, npara= ms, diff --git a/daemon/stream.c b/daemon/stream.c index 11c0a469d..6465463ff 100644 --- a/daemon/stream.c +++ b/daemon/stream.c @@ -52,6 +52,8 @@ struct daemonClientStream { virNetMessagePtr rx; bool tx; =20 + bool allowSkip; + daemonClientStreamPtr next; }; =20 @@ -321,7 +323,8 @@ daemonClientStream * daemonCreateClientStream(virNetServerClientPtr client, virStreamPtr st, virNetServerProgramPtr prog, - virNetMessageHeaderPtr header) + virNetMessageHeaderPtr header, + bool allowSkip) { daemonClientStream *stream; daemonClientPrivatePtr priv =3D virNetServerClientGetPrivateData(clien= t); @@ -339,6 +342,7 @@ daemonCreateClientStream(virNetServerClientPtr client, stream->serial =3D header->serial; stream->filterID =3D -1; stream->st =3D st; + stream->allowSkip =3D allowSkip; =20 return stream; } diff --git a/daemon/stream.h b/daemon/stream.h index cf76e717a..e1f106759 100644 --- a/daemon/stream.h +++ b/daemon/stream.h @@ -30,7 +30,8 @@ daemonClientStream * daemonCreateClientStream(virNetServerClientPtr client, virStreamPtr st, virNetServerProgramPtr prog, - virNetMessageHeaderPtr hdr); + virNetMessageHeaderPtr hdr, + bool allowSkip); =20 int daemonFreeClientStream(virNetServerClientPtr client, daemonClientStream *stream); diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index b152be523..aebdd47c9 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -6173,7 +6173,8 @@ remoteDomainMigratePrepareTunnel3(virConnectPtr dconn, if (!(netst =3D virNetClientStreamNew(st, priv->remoteProgram, REMOTE_PROC_DOMAIN_MIGRATE_PREPARE= _TUNNEL3, - priv->counter))) + priv->counter, + false))) goto done; =20 if (virNetClientAddStream(priv->client, netst) < 0) { @@ -7098,7 +7099,8 @@ remoteDomainMigratePrepareTunnel3Params(virConnectPtr= dconn, if (!(netst =3D virNetClientStreamNew(st, priv->remoteProgram, REMOTE_PROC_DOMAIN_MIGRATE_PREPARE= _TUNNEL3_PARAMS, - priv->counter))) + priv->counter, + false))) goto cleanup; =20 if (virNetClientAddStream(priv->client, netst) < 0) { diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index e608812ce..98625983a 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -1024,7 +1024,7 @@ elsif ($mode eq "server") { print " if (!(st =3D virStreamNew(priv->conn, VIR_STREAM_NO= NBLOCK)))\n"; print " goto cleanup;\n"; print "\n"; - print " if (!(stream =3D daemonCreateClientStream(client, s= t, remoteProgram, &msg->header)))\n"; + print " if (!(stream =3D daemonCreateClientStream(client, s= t, remoteProgram, &msg->header, false)))\n"; print " goto cleanup;\n"; print "\n"; } @@ -1738,7 +1738,7 @@ elsif ($mode eq "client") { =20 if ($call->{streamflag} ne "none") { print "\n"; - print " if (!(netst =3D virNetClientStreamNew(st, priv->rem= oteProgram, $call->{constname}, priv->counter)))\n"; + print " if (!(netst =3D virNetClientStreamNew(st, priv->rem= oteProgram, $call->{constname}, priv->counter, false)))\n"; print " goto done;\n"; print "\n"; print " if (virNetClientAddStream(priv->client, netst) < 0)= {\n"; diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index 01761cf8d..4c27f308e 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -54,6 +54,8 @@ struct _virNetClientStream { virNetMessagePtr rx; bool incomingEOF; =20 + bool allowSkip; + virNetClientStreamEventCallback cb; void *cbOpaque; virFreeCallback cbFree; @@ -138,7 +140,8 @@ virNetClientStreamEventTimer(int timer ATTRIBUTE_UNUSED= , void *opaque) virNetClientStreamPtr virNetClientStreamNew(virStreamPtr stream, virNetClientProgramPtr prog, int proc, - unsigned serial) + unsigned serial, + bool allowSkip) { virNetClientStreamPtr st; =20 @@ -152,6 +155,7 @@ virNetClientStreamPtr virNetClientStreamNew(virStreamPt= r stream, st->prog =3D virObjectRef(prog); st->proc =3D proc; st->serial =3D serial; + st->allowSkip =3D allowSkip; =20 return st; } diff --git a/src/rpc/virnetclientstream.h b/src/rpc/virnetclientstream.h index e278dab85..f3bc0672b 100644 --- a/src/rpc/virnetclientstream.h +++ b/src/rpc/virnetclientstream.h @@ -35,7 +35,8 @@ typedef void (*virNetClientStreamEventCallback)(virNetCli= entStreamPtr stream, virNetClientStreamPtr virNetClientStreamNew(virStreamPtr stream, virNetClientProgramPtr prog, int proc, - unsigned serial); + unsigned serial, + bool allowSkip); =20 bool virNetClientStreamRaiseError(virNetClientStreamPtr st); =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943524495931.5827572128594; Tue, 16 May 2017 07:05:24 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F0518C059720; Tue, 16 May 2017 14:04:37 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 416587D974; Tue, 16 May 2017 14:04:37 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2DCF9180BAFB; Tue, 16 May 2017 14:04:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4UQ6027808 for ; Tue, 16 May 2017 10:04:30 -0400 Received: by smtp.corp.redhat.com (Postfix) id 30FD5860EE; Tue, 16 May 2017 14:04:30 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADAAE62926 for ; Tue, 16 May 2017 14:04:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com F0518C059720 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com F0518C059720 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:53 +0200 Message-Id: <9e198448ea3e3e63f13dc9c331bbda7db230df21.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 13/31] RPC: Introduce virNetStreamHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 16 May 2017 14:05:15 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is going to be RPC representation for virStreamSendHole. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/rpc/virnetprotocol.x | 5 +++++ src/virnetprotocol-structs | 4 ++++ 2 files changed, 9 insertions(+) diff --git a/src/rpc/virnetprotocol.x b/src/rpc/virnetprotocol.x index 9ce33b073..cab047cb0 100644 --- a/src/rpc/virnetprotocol.x +++ b/src/rpc/virnetprotocol.x @@ -236,3 +236,8 @@ struct virNetMessageError { int int2; virNetMessageNetwork net; /* unused */ }; + +struct virNetStreamHole { + hyper length; + unsigned int flags; +}; diff --git a/src/virnetprotocol-structs b/src/virnetprotocol-structs index af4526c90..aa6e0602a 100644 --- a/src/virnetprotocol-structs +++ b/src/virnetprotocol-structs @@ -42,3 +42,7 @@ struct virNetMessageError { int int2; virNetMessageNetwork net; }; +struct virNetStreamHole { + int64_t length; + u_int flags; +}; --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944296514591.1313875307699; Tue, 16 May 2017 07:18:16 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B4AA080468; Tue, 16 May 2017 14:17:17 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 758F91725D; Tue, 16 May 2017 14:16:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EB9014BB7F; Tue, 16 May 2017 14:16:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4VE5027819 for ; Tue, 16 May 2017 10:04:31 -0400 Received: by smtp.corp.redhat.com (Postfix) id 10772860E0; Tue, 16 May 2017 14:04:31 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C04A52FCF for ; Tue, 16 May 2017 14:04:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B4AA080468 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B4AA080468 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:54 +0200 Message-Id: <1f3bb9fc69fbd7f8d57fa6b26ca0fbb72520f6b6.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 14/31] Introduce VIR_NET_STREAM_HOLE message type X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 16 May 2017 14:18:09 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is a special type of stream packet, that is bidirectional and will contain information on how much bytes are both sides skipping in the stream. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- daemon/stream.c | 3 ++- src/rpc/virnetclient.c | 1 + src/rpc/virnetprotocol.x | 12 +++++++++++- src/virnetprotocol-structs | 1 + 4 files changed, 15 insertions(+), 2 deletions(-) diff --git a/daemon/stream.c b/daemon/stream.c index 6465463ff..f44c21278 100644 --- a/daemon/stream.c +++ b/daemon/stream.c @@ -287,7 +287,8 @@ daemonStreamFilter(virNetServerClientPtr client ATTRIBU= TE_UNUSED, =20 virMutexLock(&stream->priv->lock); =20 - if (msg->header.type !=3D VIR_NET_STREAM) + if (msg->header.type !=3D VIR_NET_STREAM && + msg->header.type !=3D VIR_NET_STREAM_HOLE) goto cleanup; =20 if (!virNetServerProgramMatches(stream->prog, msg)) diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c index 837a8a707..95cd9a6c7 100644 --- a/src/rpc/virnetclient.c +++ b/src/rpc/virnetclient.c @@ -1284,6 +1284,7 @@ virNetClientCallDispatch(virNetClientPtr client) return virNetClientCallDispatchMessage(client); =20 case VIR_NET_STREAM: /* Stream protocol */ + case VIR_NET_STREAM_HOLE: /* Sparse stream protocol*/ return virNetClientCallDispatchStream(client); =20 default: diff --git a/src/rpc/virnetprotocol.x b/src/rpc/virnetprotocol.x index cab047cb0..ee9899059 100644 --- a/src/rpc/virnetprotocol.x +++ b/src/rpc/virnetprotocol.x @@ -143,6 +143,14 @@ const VIR_NET_MESSAGE_NUM_FDS_MAX =3D 32; * * status =3D=3D VIR_NET_ERROR * remote_error Error information * + * - type =3D=3D VIR_NET_STREAM_HOLE + * * status =3D=3D VIR_NET_CONTINUE + * byte[] hole data + * * status =3D=3D VIR_NET_ERROR + * remote_error error information + * * status =3D=3D VIR_NET_OK + * + * */ enum virNetMessageType { /* client -> server. args from a method call */ @@ -156,7 +164,9 @@ enum virNetMessageType { /* client -> server. args from a method call, with passed FDs */ VIR_NET_CALL_WITH_FDS =3D 4, /* server -> client. reply/error from a method call, with passed FDs */ - VIR_NET_REPLY_WITH_FDS =3D 5 + VIR_NET_REPLY_WITH_FDS =3D 5, + /* either direction, stream hole data packet */ + VIR_NET_STREAM_HOLE =3D 6 }; =20 enum virNetMessageStatus { diff --git a/src/virnetprotocol-structs b/src/virnetprotocol-structs index aa6e0602a..b36581f86 100644 --- a/src/virnetprotocol-structs +++ b/src/virnetprotocol-structs @@ -6,6 +6,7 @@ enum virNetMessageType { VIR_NET_STREAM =3D 3, VIR_NET_CALL_WITH_FDS =3D 4, VIR_NET_REPLY_WITH_FDS =3D 5, + VIR_NET_STREAM_HOLE =3D 6, }; enum virNetMessageStatus { VIR_NET_OK =3D 0, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944536714584.2146661709636; Tue, 16 May 2017 07:22:16 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BB4918B96B; Tue, 16 May 2017 14:21:16 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9928C777E2; Tue, 16 May 2017 14:21:06 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 74C89180BAF7; Tue, 16 May 2017 14:20:40 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4VXT027831 for ; Tue, 16 May 2017 10:04:31 -0400 Received: by smtp.corp.redhat.com (Postfix) id E33C8860F0; Tue, 16 May 2017 14:04:31 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6AA14860E6 for ; Tue, 16 May 2017 14:04:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com BB4918B96B Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com BB4918B96B From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:55 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 15/31] Teach wireshark plugin about VIR_NET_STREAM_HOLE X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 16 May 2017 14:22:05 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Ideally, this would be generated, but to achieve that corresponding XDR definitions needed to go into a different .x file. But they belong just to the one that they are right now. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- tools/wireshark/src/packet-libvirt.c | 52 ++++++++++++++++++++++++++++++++= ++++ tools/wireshark/src/packet-libvirt.h | 2 ++ 2 files changed, 54 insertions(+) diff --git a/tools/wireshark/src/packet-libvirt.c b/tools/wireshark/src/pac= ket-libvirt.c index 260161e98..a1f5a34f4 100644 --- a/tools/wireshark/src/packet-libvirt.c +++ b/tools/wireshark/src/packet-libvirt.c @@ -50,8 +50,12 @@ static int hf_libvirt_serial =3D -1; static int hf_libvirt_status =3D -1; static int hf_libvirt_stream =3D -1; static int hf_libvirt_num_of_fds =3D -1; +static int hf_libvirt_stream_hole_length =3D -1; +static int hf_libvirt_stream_hole_flags =3D -1; +static int hf_libvirt_stream_hole =3D -1; int hf_libvirt_unknown =3D -1; static gint ett_libvirt =3D -1; +static gint ett_libvirt_stream_hole =3D -1; =20 #define XDR_PRIMITIVE_DISSECTOR(xtype, ctype, ftype) \ static gboolean \ @@ -326,6 +330,33 @@ dissect_libvirt_payload_xdr_data(tvbuff_t *tvb, proto_= tree *tree, gint payload_l dissect_libvirt_fds(tvb, start + payload_length, nfds); } =20 +static gboolean +dissect_xdr_stream_hole(tvbuff_t *tvb, proto_tree *tree, XDR *xdrs, int hf) +{ + goffset start; + proto_item *ti; + + start =3D xdr_getpos(xdrs); + if (hf =3D=3D -1) { + ti =3D proto_tree_add_item(tree, hf_libvirt_stream_hole, tvb, star= t, -1, ENC_NA); + } else { + header_field_info *hfinfo; + hfinfo =3D proto_registrar_get_nth(hf_libvirt_stream_hole); + ti =3D proto_tree_add_item(tree, hf, tvb, start, -1, ENC_NA); + proto_item_append_text(ti, " :: %s", hfinfo->name); + } + tree =3D proto_item_add_subtree(ti, ett_libvirt_stream_hole); + + hf =3D hf_libvirt_stream_hole_length; + if (!dissect_xdr_hyper(tvb, tree, xdrs, hf)) return FALSE; + + hf =3D hf_libvirt_stream_hole_flags; + if (!dissect_xdr_u_int(tvb, tree, xdrs, hf)) return FALSE; + + proto_item_set_len(ti, xdr_getpos(xdrs) - start); + return TRUE; +} + static void dissect_libvirt_payload(tvbuff_t *tvb, proto_tree *tree, guint32 prog, guint32 proc, guint32 type, guint32 = status) @@ -346,6 +377,8 @@ dissect_libvirt_payload(tvbuff_t *tvb, proto_tree *tree, dissect_libvirt_payload_xdr_data(tvb, tree, payload_length, status= , VIR_ERROR_MESSAGE_DISSECTOR); } else if (type =3D=3D VIR_NET_STREAM) { /* implicitly, status =3D=3D = VIR_NET_CONTINUE */ dissect_libvirt_stream(tvb, tree, payload_length); + } else if (type =3D=3D VIR_NET_STREAM_HOLE) { + dissect_libvirt_payload_xdr_data(tvb, tree, payload_length, status= , dissect_xdr_stream_hole); } else { goto unknown; } @@ -525,6 +558,24 @@ proto_register_libvirt(void) NULL, 0x0, NULL, HFILL} }, + { &hf_libvirt_stream_hole, + { "stream_hole", "libvirt.stream_hole", + FT_NONE, BASE_NONE, + NULL, 0x0, + NULL, HFILL} + }, + { &hf_libvirt_stream_hole_length, + { "length", "libvirt.stream_hole.length", + FT_INT64, BASE_DEC, + NULL, 0x0, + NULL, HFILL} + }, + { &hf_libvirt_stream_hole_flags, + { "flags", "libvirt.stream_hole.flags", + FT_UINT32, BASE_DEC, + NULL, 0x0, + NULL, HFILL} + }, { &hf_libvirt_unknown, { "unknown", "libvirt.unknown", FT_BYTES, BASE_NONE, @@ -535,6 +586,7 @@ proto_register_libvirt(void) =20 static gint *ett[] =3D { VIR_DYNAMIC_ETTSET + &ett_libvirt_stream_hole, &ett_libvirt }; =20 diff --git a/tools/wireshark/src/packet-libvirt.h b/tools/wireshark/src/pac= ket-libvirt.h index 5f99fdfae..9874a8cbf 100644 --- a/tools/wireshark/src/packet-libvirt.h +++ b/tools/wireshark/src/packet-libvirt.h @@ -53,6 +53,7 @@ enum vir_net_message_type { VIR_NET_STREAM =3D 3, VIR_NET_CALL_WITH_FDS =3D 4, VIR_NET_REPLY_WITH_FDS =3D 5, + VIR_NET_STREAM_HOLE =3D 6, }; =20 enum vir_net_message_status { @@ -76,6 +77,7 @@ static const value_string type_strings[] =3D { { VIR_NET_STREAM, "STREAM" }, { VIR_NET_CALL_WITH_FDS, "CALL_WITH_FDS" }, { VIR_NET_REPLY_WITH_FDS, "REPLY_WITH_FDS" }, + { VIR_NET_STREAM_HOLE, "STREAM_HOLE" }, { -1, NULL } }; =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944756333510.4997504227033; Tue, 16 May 2017 07:25:56 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 995B63024A3; Tue, 16 May 2017 14:25:24 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 464CC7F85C; Tue, 16 May 2017 14:25:16 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E99BF4BB74; Tue, 16 May 2017 14:25:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4XiE027846 for ; Tue, 16 May 2017 10:04:33 -0400 Received: by smtp.corp.redhat.com (Postfix) id 1050E860E6; Tue, 16 May 2017 14:04:33 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D56A6292C for ; Tue, 16 May 2017 14:04:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 995B63024A3 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 995B63024A3 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:56 +0200 Message-Id: <00c6da4082e50d9781ba684f24412b2ca358fea9.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 16/31] daemon: Introduce virNetServerProgramSendStreamHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 16 May 2017 14:25:48 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is just a helper function that takes in a length value, encodes it into XDR and sends to client. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/libvirt_remote.syms | 1 + src/rpc/virnetserverprogram.c | 35 +++++++++++++++++++++++++++++++++++ src/rpc/virnetserverprogram.h | 8 ++++++++ 3 files changed, 44 insertions(+) diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index ca1f3ac86..bb6a8d465 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -178,6 +178,7 @@ virNetServerProgramNew; virNetServerProgramSendReplyError; virNetServerProgramSendStreamData; virNetServerProgramSendStreamError; +virNetServerProgramSendStreamHole; virNetServerProgramUnknownError; =20 =20 diff --git a/src/rpc/virnetserverprogram.c b/src/rpc/virnetserverprogram.c index d1597f438..556c91605 100644 --- a/src/rpc/virnetserverprogram.c +++ b/src/rpc/virnetserverprogram.c @@ -548,6 +548,41 @@ int virNetServerProgramSendStreamData(virNetServerProg= ramPtr prog, } =20 =20 +int virNetServerProgramSendStreamHole(virNetServerProgramPtr prog, + virNetServerClientPtr client, + virNetMessagePtr msg, + int procedure, + unsigned int serial, + long long length, + unsigned int flags) +{ + virNetStreamHole data; + + VIR_DEBUG("client=3D%p msg=3D%p length=3D%lld", client, msg, length); + + memset(&data, 0, sizeof(data)); + data.length =3D length; + data.flags =3D flags; + + msg->header.prog =3D prog->program; + msg->header.vers =3D prog->version; + msg->header.proc =3D procedure; + msg->header.type =3D VIR_NET_STREAM_HOLE; + msg->header.serial =3D serial; + msg->header.status =3D VIR_NET_CONTINUE; + + if (virNetMessageEncodeHeader(msg) < 0) + return -1; + + if (virNetMessageEncodePayload(msg, + (xdrproc_t) xdr_virNetStreamHole, + &data) < 0) + return -1; + + return virNetServerClientSendMessage(client, msg); +} + + void virNetServerProgramDispose(void *obj ATTRIBUTE_UNUSED) { } diff --git a/src/rpc/virnetserverprogram.h b/src/rpc/virnetserverprogram.h index 531fca024..1731c9e1d 100644 --- a/src/rpc/virnetserverprogram.h +++ b/src/rpc/virnetserverprogram.h @@ -104,4 +104,12 @@ int virNetServerProgramSendStreamData(virNetServerProg= ramPtr prog, const char *data, size_t len); =20 +int virNetServerProgramSendStreamHole(virNetServerProgramPtr prog, + virNetServerClientPtr client, + virNetMessagePtr msg, + int procedure, + unsigned int serial, + long long length, + unsigned int flags); + #endif /* __VIR_NET_SERVER_PROGRAM_H__ */ --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943683175717.3016884217309; Tue, 16 May 2017 07:08:03 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DB8D28B968; Tue, 16 May 2017 14:07:22 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 975FA777CE; Tue, 16 May 2017 14:07:10 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C15D94BB74; Tue, 16 May 2017 14:06:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4XfK027856 for ; Tue, 16 May 2017 10:04:33 -0400 Received: by smtp.corp.redhat.com (Postfix) id DDFFB860F3; Tue, 16 May 2017 14:04:33 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 66EE7860F0 for ; Tue, 16 May 2017 14:04:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com DB8D28B968 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com DB8D28B968 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:57 +0200 Message-Id: <71258cbfbfd5ade10d7392aa066b5d7ed58e7591.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 17/31] virnetclientstream: Introduce virNetClientStreamSendHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 16 May 2017 14:08:00 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" While the previous commit implemented a helper for sending a STREAM_HOLE packet for daemon, this is a client's counterpart. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/libvirt_remote.syms | 1 + src/rpc/virnetclientstream.c | 54 ++++++++++++++++++++++++++++++++++++++++= ++++ src/rpc/virnetclientstream.h | 5 ++++ 3 files changed, 60 insertions(+) diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index bb6a8d465..186d2c622 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -53,6 +53,7 @@ virNetClientStreamNew; virNetClientStreamQueuePacket; virNetClientStreamRaiseError; virNetClientStreamRecvPacket; +virNetClientStreamSendHole; virNetClientStreamSendPacket; virNetClientStreamSetError; =20 diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index 4c27f308e..9005e6be9 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -429,6 +429,60 @@ int virNetClientStreamRecvPacket(virNetClientStreamPtr= st, } =20 =20 +int +virNetClientStreamSendHole(virNetClientStreamPtr st, + virNetClientPtr client, + long long length, + unsigned int flags) +{ + virNetMessagePtr msg =3D NULL; + virNetStreamHole data; + int ret =3D -1; + + VIR_DEBUG("st=3D%p length=3D%llu", st, length); + + if (!st->allowSkip) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("Skipping is not supported with this stream")); + return -1; + } + + memset(&data, 0, sizeof(data)); + data.length =3D length; + data.flags =3D flags; + + if (!(msg =3D virNetMessageNew(false))) + return -1; + + virObjectLock(st); + + msg->header.prog =3D virNetClientProgramGetProgram(st->prog); + msg->header.vers =3D virNetClientProgramGetVersion(st->prog); + msg->header.status =3D VIR_NET_CONTINUE; + msg->header.type =3D VIR_NET_STREAM_HOLE; + msg->header.serial =3D st->serial; + msg->header.proc =3D st->proc; + + virObjectUnlock(st); + + if (virNetMessageEncodeHeader(msg) < 0) + goto cleanup; + + if (virNetMessageEncodePayload(msg, + (xdrproc_t) xdr_virNetStreamHole, + &data) < 0) + goto cleanup; + + if (virNetClientSendNoReply(client, msg) < 0) + goto cleanup; + + ret =3D 0; + cleanup: + virNetMessageFree(msg); + return ret; +} + + int virNetClientStreamEventAddCallback(virNetClientStreamPtr st, int events, virNetClientStreamEventCallback cb, diff --git a/src/rpc/virnetclientstream.h b/src/rpc/virnetclientstream.h index f3bc0672b..c25c69bb1 100644 --- a/src/rpc/virnetclientstream.h +++ b/src/rpc/virnetclientstream.h @@ -61,6 +61,11 @@ int virNetClientStreamRecvPacket(virNetClientStreamPtr s= t, size_t nbytes, bool nonblock); =20 +int virNetClientStreamSendHole(virNetClientStreamPtr st, + virNetClientPtr client, + long long length, + unsigned int flags); + int virNetClientStreamEventAddCallback(virNetClientStreamPtr st, int events, virNetClientStreamEventCallback cb, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494943981305263.43315377211945; Tue, 16 May 2017 07:13:01 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B0F7212BE4; Tue, 16 May 2017 14:12:29 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4E27160A9C; Tue, 16 May 2017 14:12:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3E7E7180BAF9; Tue, 16 May 2017 14:12:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4dmM027891 for ; Tue, 16 May 2017 10:04:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id DF76A860F0; Tue, 16 May 2017 14:04:39 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 67B3386104 for ; Tue, 16 May 2017 14:04:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B0F7212BE4 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B0F7212BE4 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:58 +0200 Message-Id: <5d3cd87b7ce1086c095e3d80cd98073f8ad4a568.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 18/31] daemon: Implement VIR_NET_STREAM_HOLE handling X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 16 May 2017 14:12:57 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Basically, whenever the new type of stream packet arrives to the daemon call this function that decodes it and calls virStreamSendHole(). Otherwise a regular data stream packet has arrived and therefore continue its processing. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- daemon/stream.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++----= ---- 1 file changed, 69 insertions(+), 11 deletions(-) diff --git a/daemon/stream.c b/daemon/stream.c index f44c21278..57ddfe830 100644 --- a/daemon/stream.c +++ b/daemon/stream.c @@ -653,6 +653,52 @@ daemonStreamHandleAbort(virNetServerClientPtr client, } =20 =20 +static int +daemonStreamHandleHole(virNetServerClientPtr client, + daemonClientStream *stream, + virNetMessagePtr msg) +{ + int ret; + virNetStreamHole data; + + VIR_DEBUG("client=3D%p, stream=3D%p, proc=3D%d, serial=3D%u", + client, stream, msg->header.proc, msg->header.serial); + + /* Let's check if client plays nicely and advertised usage of + * sparse stream upfront. */ + if (!stream->allowSkip) { + virReportError(VIR_ERR_RPC, "%s", + _("Unexpected stream hole")); + return -1; + } + + if (virNetMessageDecodePayload(msg, + (xdrproc_t) xdr_virNetStreamHole, + &data) < 0) + return -1; + + ret =3D virStreamSendHole(stream->st, data.length, data.flags); + + if (ret < 0) { + virNetMessageError rerr; + + memset(&rerr, 0, sizeof(rerr)); + + VIR_INFO("Stream send hole failed"); + stream->closed =3D true; + virStreamEventRemoveCallback(stream->st); + virStreamAbort(stream->st); + + return virNetServerProgramSendReplyError(stream->prog, + client, + msg, + &rerr, + &msg->header); + } + + return 0; +} + =20 /* * Called when the stream is signalled has being able to accept @@ -671,19 +717,31 @@ daemonStreamHandleWrite(virNetServerClientPtr client, virNetMessagePtr msg =3D stream->rx; int ret; =20 - switch (msg->header.status) { - case VIR_NET_OK: - ret =3D daemonStreamHandleFinish(client, stream, msg); - break; + if (msg->header.type =3D=3D VIR_NET_STREAM_HOLE) { + /* Handle special case when the client sent us a hole. + * Otherwise just carry on with processing stream + * data. */ + ret =3D daemonStreamHandleHole(client, stream, msg); + } else if (msg->header.type =3D=3D VIR_NET_STREAM) { + switch (msg->header.status) { + case VIR_NET_OK: + ret =3D daemonStreamHandleFinish(client, stream, msg); + break; =20 - case VIR_NET_CONTINUE: - ret =3D daemonStreamHandleWriteData(client, stream, msg); - break; + case VIR_NET_CONTINUE: + ret =3D daemonStreamHandleWriteData(client, stream, msg); + break; =20 - case VIR_NET_ERROR: - default: - ret =3D daemonStreamHandleAbort(client, stream, msg); - break; + case VIR_NET_ERROR: + default: + ret =3D daemonStreamHandleAbort(client, stream, msg); + break; + } + } else { + virReportError(VIR_ERR_RPC, + _("Unexpected message type: %d"), + msg->header.type); + ret =3D -1; } =20 if (ret > 0) --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944187147486.6178494097071; Tue, 16 May 2017 07:16:27 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 67CC480492; Tue, 16 May 2017 14:15:51 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C1A1A7D97C; Tue, 16 May 2017 14:15:35 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1E29B180BAFA; Tue, 16 May 2017 14:15:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4eNY027897 for ; Tue, 16 May 2017 10:04:41 -0400 Received: by smtp.corp.redhat.com (Postfix) id B3A1B860E4; Tue, 16 May 2017 14:04:40 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C074860E0 for ; Tue, 16 May 2017 14:04:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 67CC480492 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 67CC480492 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:03:59 +0200 Message-Id: <9920c5b27004b45826f6458d340d2a4f08864dcb.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 19/31] virnetclientstream: Introduce virNetClientStreamHandleHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 16 May 2017 14:16:24 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This is a function that handles an incoming STREAM_HOLE packet. Even though it is not wired up yet, it will be soon. At the beginning do couple of checks whether server plays nicely and sent us a STREAM_HOLE packed only after we've enabled sparse streams. Then decodes the message payload to see how big the hole is and stores it in passed @length argument. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/rpc/virnetclientstream.c | 96 ++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 96 insertions(+) diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index 9005e6be9..2f4f92a96 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -55,6 +55,7 @@ struct _virNetClientStream { bool incomingEOF; =20 bool allowSkip; + long long holeLength; /* Size of incoming hole in stream. */ =20 virNetClientStreamEventCallback cb; void *cbOpaque; @@ -356,6 +357,101 @@ int virNetClientStreamSendPacket(virNetClientStreamPt= r st, return -1; } =20 + +static int +virNetClientStreamSetHole(virNetClientStreamPtr st, + long long length, + unsigned int flags) +{ + virCheckFlags(0, -1); + + /* Shouldn't happen, But it's better to safe than sorry. */ + if (st->holeLength) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("unprocessed hole of size %lld already in the que= ue"), + st->holeLength); + return -1; + } + + st->holeLength +=3D length; + return 0; +} + + +/** + * virNetClientStreamHandleHole: + * @client: client + * @st: stream + * + * Called whenever current message processed in the stream is + * VIR_NET_STREAM_HOLE. The stream @st is expected to be locked + * already. + * + * Returns: 0 on success, + * -1 otherwise. + */ +static int ATTRIBUTE_UNUSED +virNetClientStreamHandleHole(virNetClientPtr client, + virNetClientStreamPtr st) +{ + virNetMessagePtr msg; + virNetStreamHole data; + int ret =3D -1; + + VIR_DEBUG("client=3D%p st=3D%p", client, st); + + msg =3D st->rx; + memset(&data, 0, sizeof(data)); + + /* We should not be called unless there's VIR_NET_STREAM_HOLE + * message at the head of the list. But doesn't hurt to check */ + if (!msg) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("No message in the queue")); + goto cleanup; + } + + if (msg->header.type !=3D VIR_NET_STREAM_HOLE) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid message prog=3D%d type=3D%d serial=3D%u = proc=3D%d"), + msg->header.prog, + msg->header.type, + msg->header.serial, + msg->header.proc); + goto cleanup; + } + + /* Server should not send us VIR_NET_STREAM_HOLE unless we + * have requested so. But does not hurt to check ... */ + if (!st->allowSkip) { + virReportError(VIR_ERR_RPC, "%s", + _("Unexpected stream hole")); + goto cleanup; + } + + if (virNetMessageDecodePayload(msg, + (xdrproc_t) xdr_virNetStreamHole, + &data) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Malformed stream hole packet")); + goto cleanup; + } + + virNetMessageQueueServe(&st->rx); + virNetMessageFree(msg); + + if (virNetClientStreamSetHole(st, data.length, data.flags) < 0) + goto cleanup; + + ret =3D 0; + cleanup: + if (ret < 0) { + /* Abort stream? */ + } + return ret; +} + + int virNetClientStreamRecvPacket(virNetClientStreamPtr st, virNetClientPtr client, char *data, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494945182634734.7287404786432; Tue, 16 May 2017 07:33:02 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B0F68C04B923; Tue, 16 May 2017 14:32:56 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 41568757D7; Tue, 16 May 2017 14:32:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 98B03180BAF7; Tue, 16 May 2017 14:32:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4fu3027898 for ; Tue, 16 May 2017 10:04:42 -0400 Received: by smtp.corp.redhat.com (Postfix) id A0247860F9; Tue, 16 May 2017 14:04:41 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29FF5860FA for ; Tue, 16 May 2017 14:04:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B0F68C04B923 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B0F68C04B923 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:00 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 20/31] remote_driver: Implement virStreamSendHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 16 May 2017 14:32:59 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Now that we have RPC wrappers over VIR_NET_STREAM_HOLE we can start wiring them up. This commit wires up situation when a client wants to send a hole to daemon. To keep stream offsets synchronous, upon successful call on the daemon skip the same hole in local part of the stream. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/remote/remote_driver.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index aebdd47c9..ff5be6ebb 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -5683,6 +5683,37 @@ remoteStreamRecv(virStreamPtr st, return remoteStreamRecvFlags(st, data, nbytes, 0); } =20 + +static int +remoteStreamSendHole(virStreamPtr st, + long long length, + unsigned int flags) +{ + VIR_DEBUG("st=3D%p length=3D%lld flags=3D%x", + st, length, flags); + struct private_data *priv =3D st->conn->privateData; + virNetClientStreamPtr privst =3D st->privateData; + int rv; + + if (virNetClientStreamRaiseError(privst)) + return -1; + + remoteDriverLock(priv); + priv->localUses++; + remoteDriverUnlock(priv); + + rv =3D virNetClientStreamSendHole(privst, + priv->client, + length, + flags); + + remoteDriverLock(priv); + priv->localUses--; + remoteDriverUnlock(priv); + return rv; +} + + struct remoteStreamCallbackData { virStreamPtr st; virStreamEventCallback cb; @@ -5857,6 +5888,7 @@ static virStreamDriver remoteStreamDrv =3D { .streamRecv =3D remoteStreamRecv, .streamRecvFlags =3D remoteStreamRecvFlags, .streamSend =3D remoteStreamSend, + .streamSendHole =3D remoteStreamSendHole, .streamFinish =3D remoteStreamFinish, .streamAbort =3D remoteStreamAbort, .streamEventAddCallback =3D remoteStreamEventAddCallback, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 14949451392831015.0825715917007; Tue, 16 May 2017 07:32:19 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0543765985; Tue, 16 May 2017 14:31:35 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5610FAAEE7; Tue, 16 May 2017 14:31:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D8287180BAF6; Tue, 16 May 2017 14:30:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4g1x027908 for ; Tue, 16 May 2017 10:04:42 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7E4EE860E0; Tue, 16 May 2017 14:04:42 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 06031860F3 for ; Tue, 16 May 2017 14:04:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0543765985 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 0543765985 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:01 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 21/31] virNetClientStreamRecvPacket: Introduce @flags argument X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 16 May 2017 14:32:12 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/remote/remote_driver.c | 3 ++- src/rpc/virnetclientstream.c | 10 +++++++--- src/rpc/virnetclientstream.h | 3 ++- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index ff5be6ebb..63daec587 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -5665,7 +5665,8 @@ remoteStreamRecvFlags(virStreamPtr st, priv->client, data, nbytes, - (st->flags & VIR_STREAM_NONBLOCK)); + (st->flags & VIR_STREAM_NONBLOCK), + flags); =20 VIR_DEBUG("Done %d", rv); =20 diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index 2f4f92a96..4a3d843b1 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -456,13 +456,17 @@ int virNetClientStreamRecvPacket(virNetClientStreamPt= r st, virNetClientPtr client, char *data, size_t nbytes, - bool nonblock) + bool nonblock, + unsigned int flags) { int rv =3D -1; size_t want; =20 - VIR_DEBUG("st=3D%p client=3D%p data=3D%p nbytes=3D%zu nonblock=3D%d", - st, client, data, nbytes, nonblock); + VIR_DEBUG("st=3D%p client=3D%p data=3D%p nbytes=3D%zu nonblock=3D%d fl= ags=3D%x", + st, client, data, nbytes, nonblock, flags); + + virCheckFlags(0, -1); + virObjectLock(st); if (!st->rx && !st->incomingEOF) { virNetMessagePtr msg; diff --git a/src/rpc/virnetclientstream.h b/src/rpc/virnetclientstream.h index c25c69bb1..1774e5ac3 100644 --- a/src/rpc/virnetclientstream.h +++ b/src/rpc/virnetclientstream.h @@ -59,7 +59,8 @@ int virNetClientStreamRecvPacket(virNetClientStreamPtr st, virNetClientPtr client, char *data, size_t nbytes, - bool nonblock); + bool nonblock, + unsigned int flags); =20 int virNetClientStreamSendHole(virNetClientStreamPtr st, virNetClientPtr client, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 149494446718214.47725614588353; Tue, 16 May 2017 07:21:07 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5337DC04BD3E; Tue, 16 May 2017 14:19:54 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F2BCB87E2A; Tue, 16 May 2017 14:19:40 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 296A84A48C; Tue, 16 May 2017 14:19:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4hPQ027918 for ; Tue, 16 May 2017 10:04:43 -0400 Received: by smtp.corp.redhat.com (Postfix) id 516AC860F2; Tue, 16 May 2017 14:04:43 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB98B860E0 for ; Tue, 16 May 2017 14:04:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5337DC04BD3E Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 5337DC04BD3E From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:02 +0200 Message-Id: <822ea2b48fe5e6f8c55b44d7b927f1f76ad9d984.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 22/31] Introduce virNetClientStreamRecvHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 16 May 2017 14:20:47 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This function will fetch previously processed stream holes and return their sum. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/libvirt_remote.syms | 1 + src/rpc/virnetclientstream.c | 16 ++++++++++++++++ src/rpc/virnetclientstream.h | 4 ++++ 3 files changed, 21 insertions(+) diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index 186d2c622..61c20d530 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -52,6 +52,7 @@ virNetClientStreamMatches; virNetClientStreamNew; virNetClientStreamQueuePacket; virNetClientStreamRaiseError; +virNetClientStreamRecvHole; virNetClientStreamRecvPacket; virNetClientStreamSendHole; virNetClientStreamSendPacket; diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index 4a3d843b1..bf3922cb5 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -583,6 +583,22 @@ virNetClientStreamSendHole(virNetClientStreamPtr st, } =20 =20 +int virNetClientStreamRecvHole(virNetClientPtr client ATTRIBUTE_UNUSED, + virNetClientStreamPtr st, + long long *length) +{ + if (!st->allowSkip) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("Holes are not supported with this stream")); + return -1; + } + + *length =3D st->holeLength; + st->holeLength =3D 0; + return 0; +} + + int virNetClientStreamEventAddCallback(virNetClientStreamPtr st, int events, virNetClientStreamEventCallback cb, diff --git a/src/rpc/virnetclientstream.h b/src/rpc/virnetclientstream.h index 1774e5ac3..c4e01bf1c 100644 --- a/src/rpc/virnetclientstream.h +++ b/src/rpc/virnetclientstream.h @@ -67,6 +67,10 @@ int virNetClientStreamSendHole(virNetClientStreamPtr st, long long length, unsigned int flags); =20 +int virNetClientStreamRecvHole(virNetClientPtr client, + virNetClientStreamPtr st, + long long *length); + int virNetClientStreamEventAddCallback(virNetClientStreamPtr st, int events, virNetClientStreamEventCallback cb, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944740702324.29097935791253; Tue, 16 May 2017 07:25:40 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 764AD3F723; Tue, 16 May 2017 14:24:36 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B17DE80704; Tue, 16 May 2017 14:24:29 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 917CF180BAF2; Tue, 16 May 2017 14:24:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4j3l027936 for ; Tue, 16 May 2017 10:04:45 -0400 Received: by smtp.corp.redhat.com (Postfix) id 636CA860FB; Tue, 16 May 2017 14:04:44 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id E0797860F9 for ; Tue, 16 May 2017 14:04:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 764AD3F723 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 764AD3F723 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:03 +0200 Message-Id: <14d7d8685b9c2e17c74a7b5842ffaac3a321f9bc.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 23/31] remote: Implement virStreamRecvHole X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 16 May 2017 14:25:34 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/remote/remote_driver.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 63daec587..dc59034c3 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -5715,6 +5715,36 @@ remoteStreamSendHole(virStreamPtr st, } =20 =20 +static int +remoteStreamRecvHole(virStreamPtr st, + long long *length, + unsigned int flags) +{ + struct private_data *priv =3D st->conn->privateData; + virNetClientStreamPtr privst =3D st->privateData; + int rv; + + VIR_DEBUG("st=3D%p length=3D%p flags=3D%x", + st, length, flags); + + virCheckFlags(0, -1); + + if (virNetClientStreamRaiseError(privst)) + return -1; + + remoteDriverLock(priv); + priv->localUses++; + remoteDriverUnlock(priv); + + rv =3D virNetClientStreamRecvHole(priv->client, privst, length); + + remoteDriverLock(priv); + priv->localUses--; + remoteDriverUnlock(priv); + return rv; +} + + struct remoteStreamCallbackData { virStreamPtr st; virStreamEventCallback cb; @@ -5890,6 +5920,7 @@ static virStreamDriver remoteStreamDrv =3D { .streamRecvFlags =3D remoteStreamRecvFlags, .streamSend =3D remoteStreamSend, .streamSendHole =3D remoteStreamSendHole, + .streamRecvHole =3D remoteStreamRecvHole, .streamFinish =3D remoteStreamFinish, .streamAbort =3D remoteStreamAbort, .streamEventAddCallback =3D remoteStreamEventAddCallback, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494945174041474.7711836063372; Tue, 16 May 2017 07:32:54 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8361C65983; Tue, 16 May 2017 14:32:05 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 88B37AAF14; Tue, 16 May 2017 14:31:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4D72E180BAF6; Tue, 16 May 2017 14:31:44 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4jE7027931 for ; Tue, 16 May 2017 10:04:45 -0400 Received: by smtp.corp.redhat.com (Postfix) id 34AB5860F9; Tue, 16 May 2017 14:04:45 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id AFFC9860FC for ; Tue, 16 May 2017 14:04:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8361C65983 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 8361C65983 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:04 +0200 Message-Id: <2c00bb2cf8f8e76d584a573fe0e807bf7759aa19.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 24/31] virNetClientStream: Wire up VIR_NET_STREAM_HOLE X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 16 May 2017 14:32:52 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Whenever server sends a client stream packet (either regular with actual data or stream skip one) it is queued on @st->rx. So the list is a mixture of both types of stream packets. So now that we have all the helpers needed we can wire their processing up. But since virNetClientStreamRecvPacket doesn't support VIR_STREAM_RECV_STOP_AT_HOLE flag yet, let's turn all received skips into zeroes repeating requested times. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/rpc/virnetclientstream.c | 45 ++++++++++++++++++++++++++++++++++++++++= ++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index bf3922cb5..75ec3f57f 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -295,6 +295,8 @@ int virNetClientStreamQueuePacket(virNetClientStreamPtr= st, =20 virObjectLock(st); =20 + /* Don't distinguish VIR_NET_STREAM and VIR_NET_STREAM_SKIP + * here just yet. We want in order processing! */ virNetMessageQueuePush(&st->rx, tmp_msg); =20 virNetClientStreamEventTimerUpdate(st); @@ -390,7 +392,7 @@ virNetClientStreamSetHole(virNetClientStreamPtr st, * Returns: 0 on success, * -1 otherwise. */ -static int ATTRIBUTE_UNUSED +static int virNetClientStreamHandleHole(virNetClientPtr client, virNetClientStreamPtr st) { @@ -468,6 +470,8 @@ int virNetClientStreamRecvPacket(virNetClientStreamPtr = st, virCheckFlags(0, -1); =20 virObjectLock(st); + + reread: if (!st->rx && !st->incomingEOF) { virNetMessagePtr msg; int ret; @@ -499,8 +503,45 @@ int virNetClientStreamRecvPacket(virNetClientStreamPtr= st, } =20 VIR_DEBUG("After IO rx=3D%p", st->rx); + + if (st->rx && + st->rx->header.type =3D=3D VIR_NET_STREAM_HOLE && + st->holeLength =3D=3D 0) { + /* Handle skip sent to us by server. */ + + if (virNetClientStreamHandleHole(client, st) < 0) + goto cleanup; + } + + if (!st->rx && !st->incomingEOF && !st->holeLength) { + if (nonblock) { + VIR_DEBUG("Non-blocking mode and no data available"); + rv =3D -2; + goto cleanup; + } + + /* We have consumed all packets from incoming queue but those + * were only skip packets, no data. Read the stream again. */ + goto reread; + } + want =3D nbytes; - while (want && st->rx) { + + if (st->holeLength) { + /* Pretend holeLength zeroes was read from stream. */ + size_t len =3D want; + + if (len > st->holeLength) + len =3D st->holeLength; + + memset(data, 0, len); + st->holeLength -=3D len; + want -=3D len; + } + + while (want && + st->rx && + st->rx->header.type =3D=3D VIR_NET_STREAM) { virNetMessagePtr msg =3D st->rx; size_t len =3D want; =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494945183479713.7545908763185; Tue, 16 May 2017 07:33:03 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 05A9CC051662; Tue, 16 May 2017 14:32:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6CA847E5CE; Tue, 16 May 2017 14:32:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EEE834A490; Tue, 16 May 2017 14:32:56 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4lV7027974 for ; Tue, 16 May 2017 10:04:47 -0400 Received: by smtp.corp.redhat.com (Postfix) id 1C815860F0; Tue, 16 May 2017 14:04:47 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 995E2860F1 for ; Tue, 16 May 2017 14:04:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 05A9CC051662 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 05A9CC051662 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:05 +0200 Message-Id: <9574b9d9024cd3b9d5ac516034e10e6077dbb0e3.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 25/31] remote_driver: Implement VIR_STREAM_RECV_STOP_AT_HOLE X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 16 May 2017 14:33:00 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/remote/remote_driver.c | 2 +- src/rpc/virnetclientstream.c | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index dc59034c3..d27e96ffc 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -5652,7 +5652,7 @@ remoteStreamRecvFlags(virStreamPtr st, virNetClientStreamPtr privst =3D st->privateData; int rv; =20 - virCheckFlags(0, -1); + virCheckFlags(VIR_STREAM_RECV_STOP_AT_HOLE, -1); =20 if (virNetClientStreamRaiseError(privst)) return -1; diff --git a/src/rpc/virnetclientstream.c b/src/rpc/virnetclientstream.c index 75ec3f57f..e68d8f946 100644 --- a/src/rpc/virnetclientstream.c +++ b/src/rpc/virnetclientstream.c @@ -467,7 +467,7 @@ int virNetClientStreamRecvPacket(virNetClientStreamPtr = st, VIR_DEBUG("st=3D%p client=3D%p data=3D%p nbytes=3D%zu nonblock=3D%d fl= ags=3D%x", st, client, data, nbytes, nonblock, flags); =20 - virCheckFlags(0, -1); + virCheckFlags(VIR_STREAM_RECV_STOP_AT_HOLE, -1); =20 virObjectLock(st); =20 @@ -531,6 +531,13 @@ int virNetClientStreamRecvPacket(virNetClientStreamPtr= st, /* Pretend holeLength zeroes was read from stream. */ size_t len =3D want; =20 + /* Yes, pretend unless we are asked not to. */ + if (flags & VIR_STREAM_RECV_STOP_AT_HOLE) { + /* No error reporting here. Caller knows what they are doing. = */ + rv =3D -3; + goto cleanup; + } + if (len > st->holeLength) len =3D st->holeLength; =20 --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494945187329439.04687660900515; Tue, 16 May 2017 07:33:07 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9982D7F4B3; Tue, 16 May 2017 14:33:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4E9697ED99; Tue, 16 May 2017 14:33:01 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D238B180BAFD; Tue, 16 May 2017 14:33:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4mMO027982 for ; Tue, 16 May 2017 10:04:49 -0400 Received: by smtp.corp.redhat.com (Postfix) id 12782860EE; Tue, 16 May 2017 14:04:48 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E6D5860E4 for ; Tue, 16 May 2017 14:04:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 9982D7F4B3 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 9982D7F4B3 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:06 +0200 Message-Id: <78d56a6a920269c3669c9985a6ff36b2f79176cb.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 26/31] daemonStreamHandleRead: Wire up seekable stream X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 16 May 2017 14:33:03 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Whenever client is able to receive some data from stream daemonStreamHandleRead is called. But now the behaviour of this function needs to be changed a bit. Previously it just read data from underlying file (of chardev or whatever) and sent those through the stream to client. This model will not work any longer because it does not differentiate whether underlying file is in data or hole section. Therefore, at the beginning of this function add code that checks this situation and acts accordingly. So after the this, when wanting to send some data we always check whether we are not in a hole and if so, skip it an inform client about its size. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- daemon/stream.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 59 insertions(+) diff --git a/daemon/stream.c b/daemon/stream.c index 57ddfe830..284499912 100644 --- a/daemon/stream.c +++ b/daemon/stream.c @@ -29,6 +29,7 @@ #include "virlog.h" #include "virnetserverclient.h" #include "virerror.h" +#include "libvirt_internal.h" =20 #define VIR_FROM_THIS VIR_FROM_STREAMS =20 @@ -53,6 +54,7 @@ struct daemonClientStream { bool tx; =20 bool allowSkip; + size_t dataLen; /* How much data is there remaining until we see a hol= e */ =20 daemonClientStreamPtr next; }; @@ -796,6 +798,8 @@ daemonStreamHandleRead(virNetServerClientPtr client, size_t bufferLen =3D VIR_NET_MESSAGE_LEGACY_PAYLOAD_MAX; int ret =3D -1; int rv; + int inData =3D 0; + long long length =3D 0; =20 VIR_DEBUG("client=3D%p, stream=3D%p tx=3D%d closed=3D%d", client, stream, stream->tx, stream->closed); @@ -820,6 +824,58 @@ daemonStreamHandleRead(virNetServerClientPtr client, if (!(msg =3D virNetMessageNew(false))) goto cleanup; =20 + if (stream->allowSkip && !stream->dataLen) { + /* Handle skip. We want to send some data to the client. But we mi= ght + * be in a hole. Seek to next data. But if we are in data already,= just + * carry on. */ + + rv =3D virStreamInData(stream->st, &inData, &length); + VIR_DEBUG("rv=3D%d inData=3D%d length=3D%lld", rv, inData, length); + + if (rv < 0) { + if (virNetServerProgramSendStreamError(remoteProgram, + client, + msg, + &rerr, + stream->procedure, + stream->serial) < 0) + goto cleanup; + msg =3D NULL; + + /* We're done with this call */ + goto done; + } else { + if (!inData && length) { + stream->tx =3D false; + msg->cb =3D daemonStreamMessageFinished; + msg->opaque =3D stream; + stream->refs++; + if (virNetServerProgramSendStreamHole(remoteProgram, + client, + msg, + stream->procedure, + stream->serial, + length, + 0) < 0) + goto cleanup; + + msg =3D NULL; + + /* We have successfully sent stream skip to the other sid= e. + * To keep streams in sync seek locally too. */ + virStreamSendHole(stream->st, length, 0); + /* We're done with this call */ + goto done; + } + } + + stream->dataLen =3D length; + } + + if (stream->allowSkip && + bufferLen > stream->dataLen) + bufferLen =3D stream->dataLen; + rv =3D virStreamRecv(stream->st, buffer, bufferLen); if (rv =3D=3D -2) { /* Should never get this, since we're only called when we know @@ -834,6 +890,8 @@ daemonStreamHandleRead(virNetServerClientPtr client, goto cleanup; msg =3D NULL; } else { + stream->dataLen -=3D rv; + stream->tx =3D false; if (rv =3D=3D 0) stream->recvEOF =3D true; @@ -851,6 +909,7 @@ daemonStreamHandleRead(virNetServerClientPtr client, msg =3D NULL; } =20 + done: ret =3D 0; cleanup: VIR_FREE(buffer); --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944758742432.914630959208; Tue, 16 May 2017 07:25:58 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F19FE7E9EF; Tue, 16 May 2017 14:25:49 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 46461A471F; Tue, 16 May 2017 14:25:49 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 95299180BAF8; Tue, 16 May 2017 14:25:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4nRK027990 for ; Tue, 16 May 2017 10:04:49 -0400 Received: by smtp.corp.redhat.com (Postfix) id EEAE0860E4; Tue, 16 May 2017 14:04:48 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 70C2862926 for ; Tue, 16 May 2017 14:04:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com F19FE7E9EF Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com F19FE7E9EF From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:07 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 27/31] fdstream: Implement sparse stream X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 16 May 2017 14:25:55 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Basically, what is needed here is to introduce new message type for the messages passed between the event loop callbacks and the worker thread that does all the I/O. The idea is that instead of a queue of read buffers we will have a queue where "hole of size X" messages appear. That way the even loop callbacks can just check the head of the queue and see if the worker thread is in data or a hole section and how long the section is. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/storage/storage_util.c | 4 +- src/util/virfdstream.c | 239 ++++++++++++++++++++++++++++++++++++++++-= ---- src/util/virfdstream.h | 1 + 3 files changed, 220 insertions(+), 24 deletions(-) diff --git a/src/storage/storage_util.c b/src/storage/storage_util.c index 43f3561f8..908cad874 100644 --- a/src/storage/storage_util.c +++ b/src/storage/storage_util.c @@ -2427,7 +2427,7 @@ virStorageBackendVolUploadLocal(virConnectPtr conn AT= TRIBUTE_UNUSED, /* Not using O_CREAT because the file is required to already exist at * this point */ ret =3D virFDStreamOpenBlockDevice(stream, target_path, - offset, len, O_WRONLY); + offset, len, false, O_WRONLY); =20 cleanup: VIR_FREE(path); @@ -2465,7 +2465,7 @@ virStorageBackendVolDownloadLocal(virConnectPtr conn = ATTRIBUTE_UNUSED, } =20 ret =3D virFDStreamOpenBlockDevice(stream, target_path, - offset, len, O_RDONLY); + offset, len, false, O_RDONLY); =20 cleanup: VIR_FREE(path); diff --git a/src/util/virfdstream.c b/src/util/virfdstream.c index 4b42939e7..ba209025a 100644 --- a/src/util/virfdstream.c +++ b/src/util/virfdstream.c @@ -51,6 +51,7 @@ VIR_LOG_INIT("fdstream"); =20 typedef enum { VIR_FDSTREAM_MSG_TYPE_DATA, + VIR_FDSTREAM_MSG_TYPE_HOLE, } virFDStreamMsgType; =20 typedef struct _virFDStreamMsg virFDStreamMsg; @@ -66,6 +67,9 @@ struct _virFDStreamMsg { size_t len; size_t offset; } data; + struct { + long long len; + } hole; } stream; }; =20 @@ -198,6 +202,9 @@ virFDStreamMsgFree(virFDStreamMsgPtr msg) case VIR_FDSTREAM_MSG_TYPE_DATA: VIR_FREE(msg->stream.data.buf); break; + case VIR_FDSTREAM_MSG_TYPE_HOLE: + /* nada */ + break; } =20 VIR_FREE(msg); @@ -385,6 +392,7 @@ struct _virFDStreamThreadData { virStreamPtr st; size_t length; bool doRead; + bool sparse; int fdin; char *fdinname; int fdout; @@ -407,34 +415,68 @@ virFDStreamThreadDataFree(virFDStreamThreadDataPtr da= ta) =20 static ssize_t virFDStreamThreadDoRead(virFDStreamDataPtr fdst, + bool sparse, const int fdin, const int fdout, const char *fdinname, const char *fdoutname, + size_t *dataLen, size_t buflen) { virFDStreamMsgPtr msg =3D NULL; + int inData =3D 0; + long long sectionLen =3D 0; char *buf =3D NULL; ssize_t got; =20 + if (sparse && *dataLen =3D=3D 0) { + if (virFileInData(fdin, &inData, §ionLen) < 0) + goto error; + + if (inData) + *dataLen =3D sectionLen; + } + if (VIR_ALLOC(msg) < 0) goto error; =20 - if (VIR_ALLOC_N(buf, buflen) < 0) - goto error; - - if ((got =3D saferead(fdin, buf, buflen)) < 0) { - virReportSystemError(errno, - _("Unable to read %s"), - fdinname); - goto error; + if (sparse && *dataLen =3D=3D 0) { + msg->type =3D VIR_FDSTREAM_MSG_TYPE_HOLE; + msg->stream.hole.len =3D sectionLen; + got =3D sectionLen; + + /* HACK. The message queue is one directional. So caller + * cannot make us skip the hole. Do that for them instead. */ + if (sectionLen && + lseek(fdin, sectionLen, SEEK_CUR) =3D=3D (off_t) -1) { + virReportSystemError(errno, + _("unable to seek in %s"), + fdinname); + goto error; + } + } else { + if (sparse && + buflen > *dataLen) + buflen =3D *dataLen; + + if (VIR_ALLOC_N(buf, buflen) < 0) + goto error; + + if ((got =3D saferead(fdin, buf, buflen)) < 0) { + virReportSystemError(errno, + _("Unable to read %s"), + fdinname); + goto error; + } + + msg->type =3D VIR_FDSTREAM_MSG_TYPE_DATA; + msg->stream.data.buf =3D buf; + msg->stream.data.len =3D got; + buf =3D NULL; + if (sparse) + *dataLen -=3D got; } =20 - msg->type =3D VIR_FDSTREAM_MSG_TYPE_DATA; - msg->stream.data.buf =3D buf; - msg->stream.data.len =3D got; - buf =3D NULL; - virFDStreamMsgQueuePush(fdst, msg, fdout, fdoutname); msg =3D NULL; =20 @@ -449,6 +491,7 @@ virFDStreamThreadDoRead(virFDStreamDataPtr fdst, =20 static ssize_t virFDStreamThreadDoWrite(virFDStreamDataPtr fdst, + bool sparse, const int fdin, const int fdout, const char *fdinname, @@ -456,6 +499,7 @@ virFDStreamThreadDoWrite(virFDStreamDataPtr fdst, { ssize_t got; virFDStreamMsgPtr msg =3D fdst->msg; + off_t off; bool pop =3D false; =20 switch (msg->type) { @@ -474,6 +518,32 @@ virFDStreamThreadDoWrite(virFDStreamDataPtr fdst, =20 pop =3D msg->stream.data.offset =3D=3D msg->stream.data.len; break; + + case VIR_FDSTREAM_MSG_TYPE_HOLE: + if (!sparse) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("unexpected stream hole")); + return -1; + } + + got =3D msg->stream.hole.len; + off =3D lseek(fdout, got, SEEK_CUR); + if (off =3D=3D (off_t) -1) { + virReportSystemError(errno, + _("unable to seek in %s"), + fdoutname); + return -1; + } + + if (ftruncate(fdout, off) < 0) { + virReportSystemError(errno, + _("unable to truncate %s"), + fdoutname); + return -1; + } + + pop =3D true; + break; } =20 if (pop) { @@ -491,6 +561,7 @@ virFDStreamThread(void *opaque) virFDStreamThreadDataPtr data =3D opaque; virStreamPtr st =3D data->st; size_t length =3D data->length; + bool sparse =3D data->sparse; int fdin =3D data->fdin; char *fdinname =3D data->fdinname; int fdout =3D data->fdout; @@ -499,6 +570,7 @@ virFDStreamThread(void *opaque) bool doRead =3D fdst->threadDoRead; size_t buflen =3D 256 * 1024; size_t total =3D 0; + size_t dataLen =3D 0; =20 virObjectRef(fdst); virObjectLock(fdst); @@ -533,12 +605,12 @@ virFDStreamThread(void *opaque) } =20 if (doRead) - got =3D virFDStreamThreadDoRead(fdst, + got =3D virFDStreamThreadDoRead(fdst, sparse, fdin, fdout, fdinname, fdoutname, - buflen); + &dataLen, buflen); else - got =3D virFDStreamThreadDoWrite(fdst, + got =3D virFDStreamThreadDoWrite(fdst, sparse, fdin, fdout, fdinname, fdoutname); =20 @@ -809,6 +881,14 @@ static int virFDStreamRead(virStreamPtr st, char *byte= s, size_t nbytes) } } =20 + /* Shortcut, if the stream is in the trailing hole, + * return 0 immediately. */ + if (msg->type =3D=3D VIR_FDSTREAM_MSG_TYPE_HOLE && + msg->stream.hole.len =3D=3D 0) { + ret =3D 0; + goto cleanup; + } + if (msg->type !=3D VIR_FDSTREAM_MSG_TYPE_DATA) { /* Nope, nope, I'm outta here */ virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -859,11 +939,123 @@ static int virFDStreamRead(virStreamPtr st, char *by= tes, size_t nbytes) } =20 =20 +static int +virFDStreamSendHole(virStreamPtr st, + long long length, + unsigned int flags) +{ + virFDStreamDataPtr fdst =3D st->privateData; + virFDStreamMsgPtr msg =3D NULL; + off_t off; + int ret =3D -1; + + virCheckFlags(0, -1); + + virObjectLock(fdst); + if (fdst->length) { + if (length > fdst->length - fdst->offset) + length =3D fdst->length - fdst->offset; + fdst->offset +=3D length; + } + + if (fdst->thread) { + /* Things are a bit complicated here. But bear with me. If FDStrea= m is + * in a read mode, then if the message at the queue head is HOLE, = just + * pop it. The thread has lseek()-ed anyway. If however, the FDStr= eam + * is in write mode, then tell the thread to do the lseek() for us. + * Under no circumstances we can do the lseek() ourselves here. We + * might mess up file position for the thread. */ + if (fdst->threadDoRead) { + msg =3D fdst->msg; + if (msg->type !=3D VIR_FDSTREAM_MSG_TYPE_HOLE) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Invalid stream hole")); + goto cleanup; + } + + virFDStreamMsgQueuePop(fdst, fdst->fd, "pipe"); + } else { + if (VIR_ALLOC(msg) < 0) + goto cleanup; + + msg->type =3D VIR_FDSTREAM_MSG_TYPE_HOLE; + msg->stream.hole.len =3D length; + virFDStreamMsgQueuePush(fdst, msg, fdst->fd, "pipe"); + msg =3D NULL; + } + } else { + off =3D lseek(fdst->fd, length, SEEK_CUR); + if (off =3D=3D (off_t) -1) { + virReportSystemError(errno, "%s", + _("unable to seek")); + goto cleanup; + } + + if (ftruncate(fdst->fd, off) < 0) { + virReportSystemError(errno, "%s", + _("unable to truncate")); + goto cleanup; + } + } + + ret =3D 0; + cleanup: + virObjectUnlock(fdst); + virFDStreamMsgFree(msg); + return ret; +} + + +static int +virFDStreamInData(virStreamPtr st, + int *inData, + long long *length) +{ + virFDStreamDataPtr fdst =3D st->privateData; + int ret =3D -1; + + virObjectLock(fdst); + + if (fdst->thread) { + virFDStreamMsgPtr msg; + + while (!(msg =3D fdst->msg)) { + if (fdst->threadQuit) { + *inData =3D *length =3D 0; + ret =3D 0; + goto cleanup; + } else { + virObjectUnlock(fdst); + virCondSignal(&fdst->threadCond); + virObjectLock(fdst); + } + } + + if (msg->type =3D=3D VIR_FDSTREAM_MSG_TYPE_DATA) { + *inData =3D 1; + *length =3D msg->stream.data.len - msg->stream.data.offset; + } else { + *inData =3D 0; + *length =3D msg->stream.hole.len; + } + ret =3D 0; + } else { + ret =3D virFileInData(fdst->fd, inData, length); + } + + cleanup: + virObjectUnlock(fdst); + return ret; +} + + static virStreamDriver virFDStreamDrv =3D { .streamSend =3D virFDStreamWrite, .streamRecv =3D virFDStreamRead, .streamFinish =3D virFDStreamClose, .streamAbort =3D virFDStreamAbort, + .streamSendHole =3D virFDStreamSendHole, + .streamInData =3D virFDStreamInData, .streamEventAddCallback =3D virFDStreamAddCallback, .streamEventUpdateCallback =3D virFDStreamUpdateCallback, .streamEventRemoveCallback =3D virFDStreamRemoveCallback @@ -1004,7 +1196,8 @@ virFDStreamOpenFileInternal(virStreamPtr st, unsigned long long length, int oflags, int mode, - bool forceIOHelper) + bool forceIOHelper, + bool sparse) { int fd =3D -1; int pipefds[2] =3D { -1, -1 }; @@ -1071,6 +1264,7 @@ virFDStreamOpenFileInternal(virStreamPtr st, =20 threadData->st =3D virObjectRef(st); threadData->length =3D length; + threadData->sparse =3D sparse; =20 if ((oflags & O_ACCMODE) =3D=3D O_RDONLY) { threadData->fdin =3D fd; @@ -1120,7 +1314,7 @@ int virFDStreamOpenFile(virStreamPtr st, } return virFDStreamOpenFileInternal(st, path, offset, length, - oflags, 0, false); + oflags, 0, false, false); } =20 int virFDStreamCreateFile(virStreamPtr st, @@ -1133,7 +1327,7 @@ int virFDStreamCreateFile(virStreamPtr st, return virFDStreamOpenFileInternal(st, path, offset, length, oflags | O_CREAT, mode, - false); + false, false); } =20 #ifdef HAVE_CFMAKERAW @@ -1149,7 +1343,7 @@ int virFDStreamOpenPTY(virStreamPtr st, if (virFDStreamOpenFileInternal(st, path, offset, length, oflags | O_CREAT, 0, - false) < 0) + false, false) < 0) return -1; =20 fdst =3D st->privateData; @@ -1186,7 +1380,7 @@ int virFDStreamOpenPTY(virStreamPtr st, return virFDStreamOpenFileInternal(st, path, offset, length, oflags | O_CREAT, 0, - false); + false, false); } #endif /* !HAVE_CFMAKERAW */ =20 @@ -1194,11 +1388,12 @@ int virFDStreamOpenBlockDevice(virStreamPtr st, const char *path, unsigned long long offset, unsigned long long length, + bool sparse, int oflags) { return virFDStreamOpenFileInternal(st, path, offset, length, - oflags, 0, true); + oflags, 0, true, sparse); } =20 int virFDStreamSetInternalCloseCb(virStreamPtr st, diff --git a/src/util/virfdstream.h b/src/util/virfdstream.h index 34c4c3fc6..887c991d6 100644 --- a/src/util/virfdstream.h +++ b/src/util/virfdstream.h @@ -59,6 +59,7 @@ int virFDStreamOpenBlockDevice(virStreamPtr st, const char *path, unsigned long long offset, unsigned long long length, + bool sparse, int oflags); =20 int virFDStreamSetInternalCloseCb(virStreamPtr st, --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 149494540787775.68269884118808; Tue, 16 May 2017 07:36:47 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D225BAECC3; Tue, 16 May 2017 14:35:55 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D983F77CB4; Tue, 16 May 2017 14:35:45 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 039C7180BAF3; Tue, 16 May 2017 14:35:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4oa9027999 for ; Tue, 16 May 2017 10:04:50 -0400 Received: by smtp.corp.redhat.com (Postfix) id DAFFF860E0; Tue, 16 May 2017 14:04:49 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 62C9D860E4 for ; Tue, 16 May 2017 14:04:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D225BAECC3 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D225BAECC3 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:08 +0200 Message-Id: <2d2db7a061c9bb5dcbeaa30e55ac6665eb2ebb08.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 28/31] gendispatch: Introduce @sparseflag for our calls X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 16 May 2017 14:36:32 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Now, not all APIs are going to support sparse streams. To some it makes no sense at all, e.g. virDomainOpenConsole() or virDomainOpenChannel(). To others, we will need a special flag to indicate that client wants to enable sparse streams. Instead of having to write RPC dispatchers by hand we can just annotate in our .x files that a certain flag to certain RPC call enables this feature. For instance: /** * @generate: both * @readstream: 1 * @sparseflag: VIR_SPARSE_STREAM * @acl: storage_vol:data_read */ REMOTE_PROC_DOMAIN_SOME_API =3D XXX, Therefore, whenever client calls virDomainSomeAPI(.., VIR_SPARSE_STREAM); daemon will mark that down and send stream skips when possible. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- src/rpc/gendispatch.pl | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 98625983a..def88d4f9 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -281,6 +281,13 @@ while () { $calls{$name}->{streamflag} =3D "none"; } =20 + if (exists $opts{sparseflag}) { + die "\@sparseflag requires stream" unless $calls{$name}->{stre= amflag} ne "none"; + $calls{$name}->{sparseflag} =3D $opts{sparseflag}; + } else { + $calls{$name}->{sparseflag} =3D "none"; + } + $calls{$name}->{acl} =3D $opts{acl}; $calls{$name}->{aclfilter} =3D $opts{aclfilter}; =20 @@ -982,6 +989,11 @@ elsif ($mode eq "server") { if ($call->{streamflag} ne "none") { print " virStreamPtr st =3D NULL;\n"; print " daemonClientStreamPtr stream =3D NULL;\n"; + if ($call->{sparseflag} ne "none") { + print " const bool sparse =3D args->flags & $call->{spa= rseflag};\n" + } else { + print " const bool sparse =3D false;\n"; + } } =20 print "\n"; @@ -1024,7 +1036,7 @@ elsif ($mode eq "server") { print " if (!(st =3D virStreamNew(priv->conn, VIR_STREAM_NO= NBLOCK)))\n"; print " goto cleanup;\n"; print "\n"; - print " if (!(stream =3D daemonCreateClientStream(client, s= t, remoteProgram, &msg->header, false)))\n"; + print " if (!(stream =3D daemonCreateClientStream(client, s= t, remoteProgram, &msg->header, sparse)))\n"; print " goto cleanup;\n"; print "\n"; } @@ -1727,6 +1739,11 @@ elsif ($mode eq "client") { =20 if ($call->{streamflag} ne "none") { print " virNetClientStreamPtr netst =3D NULL;\n"; + if ($call->{sparseflag} ne "none") { + print " const bool sparse =3D flags & $call->{sparsefla= g};\n" + } else { + print " const bool sparse =3D false;\n"; + } } =20 print "\n"; @@ -1738,7 +1755,7 @@ elsif ($mode eq "client") { =20 if ($call->{streamflag} ne "none") { print "\n"; - print " if (!(netst =3D virNetClientStreamNew(st, priv->rem= oteProgram, $call->{constname}, priv->counter, false)))\n"; + print " if (!(netst =3D virNetClientStreamNew(st, priv->rem= oteProgram, $call->{constname}, priv->counter, sparse)))\n"; print " goto done;\n"; print "\n"; print " if (virNetClientAddStream(priv->client, netst) < 0)= {\n"; --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944760822344.42124311212126; Tue, 16 May 2017 07:26:00 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 23419C0E7244; Tue, 16 May 2017 14:25:55 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AECE5AC1FE; Tue, 16 May 2017 14:25:54 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5462B4A48C; Tue, 16 May 2017 14:25:54 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4oXY028005 for ; Tue, 16 May 2017 10:04:50 -0400 Received: by smtp.corp.redhat.com (Postfix) id AA328860E4; Tue, 16 May 2017 14:04:50 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 32808860E0 for ; Tue, 16 May 2017 14:04:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 23419C0E7244 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 23419C0E7244 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:09 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 29/31] Introduce virStorageVol{Download, Upload}Flags X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 16 May 2017 14:25:57 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" These flags to APIs will tell if caller wants to use sparse stream for storage transfer. At the same time, it's safe to enable them in storage driver frontend and rely on our backends checking the flags. This way we can enable specific flags only on some specific backends, e.g. enable VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM for filesystem backend but not iSCSI backend. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- include/libvirt/libvirt-storage.h | 9 +++++++++ src/libvirt-storage.c | 4 ++-- src/remote/remote_protocol.x | 2 ++ src/storage/storage_driver.c | 4 ++-- src/storage/storage_util.c | 10 ++++++---- 5 files changed, 21 insertions(+), 8 deletions(-) diff --git a/include/libvirt/libvirt-storage.h b/include/libvirt/libvirt-st= orage.h index 45ec72065..4517f713c 100644 --- a/include/libvirt/libvirt-storage.h +++ b/include/libvirt/libvirt-storage.h @@ -346,11 +346,20 @@ virStorageVolPtr virStorageVolCreateXMLFrom = (virStoragePoolPtr pool, const char *xmlde= sc, virStorageVolPtr = clonevol, unsigned int flag= s); + +typedef enum { + VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM =3D 1 << 0, /* Use sparse strea= m */ +} virStorageVolDownloadFlags; + int virStorageVolDownload (virStorageVolPtr = vol, virStreamPtr stre= am, unsigned long lon= g offset, unsigned long lon= g length, unsigned int flag= s); +typedef enum { + VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM =3D 1 << 0, /* Use sparse stream= */ +} virStorageVolUploadFlags; + int virStorageVolUpload (virStorageVolPtr = vol, virStreamPtr stre= am, unsigned long lon= g offset, diff --git a/src/libvirt-storage.c b/src/libvirt-storage.c index 05eec8a9d..64202998b 100644 --- a/src/libvirt-storage.c +++ b/src/libvirt-storage.c @@ -1549,7 +1549,7 @@ virStorageVolCreateXMLFrom(virStoragePoolPtr pool, * @stream: stream to use as output * @offset: position in @vol to start reading from * @length: limit on amount of data to download - * @flags: extra flags; not used yet, so callers should always pass 0 + * @flags: bitwise-OR of virStorageVolDownloadFlags * * Download the content of the volume as a stream. If @length * is zero, then the remaining contents of the volume after @@ -1613,7 +1613,7 @@ virStorageVolDownload(virStorageVolPtr vol, * @stream: stream to use as input * @offset: position to start writing to * @length: limit on amount of data to upload - * @flags: extra flags; not used yet, so callers should always pass 0 + * @flags: bitwise-OR of virStorageVolUploadFlags * * Upload new content to the volume from a stream. This call * will fail if @offset + @length exceeds the size of the diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 87b2bd365..25e62a181 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -4896,6 +4896,7 @@ enum remote_procedure { /** * @generate: both * @writestream: 1 + * @sparseflag: VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM * @acl: storage_vol:data_write */ REMOTE_PROC_STORAGE_VOL_UPLOAD =3D 208, @@ -4903,6 +4904,7 @@ enum remote_procedure { /** * @generate: both * @readstream: 1 + * @sparseflag: VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM * @acl: storage_vol:data_read */ REMOTE_PROC_STORAGE_VOL_DOWNLOAD =3D 209, diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index 2103ed11d..1b0d776c7 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -2117,7 +2117,7 @@ storageVolDownload(virStorageVolPtr obj, virStorageVolDefPtr vol =3D NULL; int ret =3D -1; =20 - virCheckFlags(0, -1); + virCheckFlags(VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM, -1); =20 if (!(vol =3D virStorageVolDefFromVol(obj, &pool, &backend))) return -1; @@ -2285,7 +2285,7 @@ storageVolUpload(virStorageVolPtr obj, virStorageVolStreamInfoPtr cbdata =3D NULL; int ret =3D -1; =20 - virCheckFlags(0, -1); + virCheckFlags(VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM, -1); =20 if (!(vol =3D virStorageVolDefFromVol(obj, &pool, &backend))) return -1; diff --git a/src/storage/storage_util.c b/src/storage/storage_util.c index 908cad874..493c651b7 100644 --- a/src/storage/storage_util.c +++ b/src/storage/storage_util.c @@ -2401,8 +2401,9 @@ virStorageBackendVolUploadLocal(virConnectPtr conn AT= TRIBUTE_UNUSED, char *target_path =3D vol->target.path; int ret =3D -1; int has_snap =3D 0; + bool sparse =3D flags & VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM; =20 - virCheckFlags(0, -1); + virCheckFlags(VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM, -1); /* if volume has target format VIR_STORAGE_FILE_PLOOP * we need to restore DiskDescriptor.xml, according to * new contents of volume. This operation will be perfomed @@ -2427,7 +2428,7 @@ virStorageBackendVolUploadLocal(virConnectPtr conn AT= TRIBUTE_UNUSED, /* Not using O_CREAT because the file is required to already exist at * this point */ ret =3D virFDStreamOpenBlockDevice(stream, target_path, - offset, len, false, O_WRONLY); + offset, len, sparse, O_WRONLY); =20 cleanup: VIR_FREE(path); @@ -2447,8 +2448,9 @@ virStorageBackendVolDownloadLocal(virConnectPtr conn = ATTRIBUTE_UNUSED, char *target_path =3D vol->target.path; int ret =3D -1; int has_snap =3D 0; + bool sparse =3D flags & VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM; =20 - virCheckFlags(0, -1); + virCheckFlags(VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM, -1); if (vol->target.format =3D=3D VIR_STORAGE_FILE_PLOOP) { has_snap =3D storageBackendPloopHasSnapshots(vol->target.path); if (has_snap < 0) { @@ -2465,7 +2467,7 @@ virStorageBackendVolDownloadLocal(virConnectPtr conn = ATTRIBUTE_UNUSED, } =20 ret =3D virFDStreamOpenBlockDevice(stream, target_path, - offset, len, false, O_RDONLY); + offset, len, sparse, O_RDONLY); =20 cleanup: VIR_FREE(path); --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494944913196314.7263031807254; Tue, 16 May 2017 07:28:33 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A1D7197830; Tue, 16 May 2017 14:27:48 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7D69A7EF6C; Tue, 16 May 2017 14:27:38 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2B328180BAF2; Tue, 16 May 2017 14:27:24 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4pEA028018 for ; Tue, 16 May 2017 10:04:51 -0400 Received: by smtp.corp.redhat.com (Postfix) id 8E767860F8; Tue, 16 May 2017 14:04:51 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 16B16860E0 for ; Tue, 16 May 2017 14:04:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A1D7197830 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A1D7197830 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:10 +0200 Message-Id: In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 30/31] virsh: Implement sparse stream to vol-download X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 16 May 2017 14:28:31 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The command grew new --sparse switch that does nothing more than enables the sparse streams feature for this command. Among with the switch new helper function is introduced: virshStreamSkip(). This is the callback that is called whenever daemon sends us a hole. In the callback we reflect the hole in underlying file by seeking as many bytes as told. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- tools/virsh-util.c | 18 ++++++++++++++++++ tools/virsh-util.h | 5 +++++ tools/virsh-volume.c | 12 ++++++++++-- tools/virsh.pod | 3 ++- 4 files changed, 35 insertions(+), 3 deletions(-) diff --git a/tools/virsh-util.c b/tools/virsh-util.c index 4b86e29cb..198625bdb 100644 --- a/tools/virsh-util.c +++ b/tools/virsh-util.c @@ -153,6 +153,24 @@ virshStreamSink(virStreamPtr st ATTRIBUTE_UNUSED, } =20 =20 +int +virshStreamSkip(virStreamPtr st ATTRIBUTE_UNUSED, + long long offset, + void *opaque) +{ + int *fd =3D opaque; + off_t cur; + + if ((cur =3D lseek(*fd, offset, SEEK_CUR)) =3D=3D (off_t) -1) + return -1; + + if (ftruncate(*fd, cur) < 0) + return -1; + + return 0; +} + + void virshDomainFree(virDomainPtr dom) { diff --git a/tools/virsh-util.h b/tools/virsh-util.h index 64cef23c0..0aba247f6 100644 --- a/tools/virsh-util.h +++ b/tools/virsh-util.h @@ -57,6 +57,11 @@ virshStreamSink(virStreamPtr st, size_t nbytes, void *opaque); =20 +int +virshStreamSkip(virStreamPtr st, + long long offset, + void *opaque); + int virshDomainGetXMLFromDom(vshControl *ctl, virDomainPtr dom, diff --git a/tools/virsh-volume.c b/tools/virsh-volume.c index 66fe70ea7..3d19b745e 100644 --- a/tools/virsh-volume.c +++ b/tools/virsh-volume.c @@ -763,6 +763,10 @@ static const vshCmdOptDef opts_vol_download[] =3D { .type =3D VSH_OT_INT, .help =3D N_("amount of data to download") }, + {.name =3D "sparse", + .type =3D VSH_OT_BOOL, + .help =3D N_("preserve sparseness of volume") + }, {.name =3D NULL} }; =20 @@ -778,6 +782,7 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) unsigned long long offset =3D 0, length =3D 0; bool created =3D false; virshControlPtr priv =3D ctl->privData; + unsigned int flags =3D 0; =20 if (vshCommandOptULongLong(ctl, cmd, "offset", &offset) < 0) return false; @@ -791,6 +796,9 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) if (vshCommandOptStringReq(ctl, cmd, "file", &file) < 0) goto cleanup; =20 + if (vshCommandOptBool(cmd, "sparse")) + flags |=3D VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM; + if ((fd =3D open(file, O_WRONLY|O_CREAT|O_EXCL, 0666)) < 0) { if (errno !=3D EEXIST || (fd =3D open(file, O_WRONLY|O_TRUNC, 0666)) < 0) { @@ -806,12 +814,12 @@ cmdVolDownload(vshControl *ctl, const vshCmd *cmd) goto cleanup; } =20 - if (virStorageVolDownload(vol, st, offset, length, 0) < 0) { + if (virStorageVolDownload(vol, st, offset, length, flags) < 0) { vshError(ctl, _("cannot download from volume %s"), name); goto cleanup; } =20 - if (virStreamRecvAll(st, virshStreamSink, &fd) < 0) { + if (virStreamSparseRecvAll(st, virshStreamSink, virshStreamSkip, &fd) = < 0) { vshError(ctl, _("cannot receive data from volume %s"), name); goto cleanup; } diff --git a/tools/virsh.pod b/tools/virsh.pod index 727acf6e6..dcaa0c170 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -3942,12 +3942,13 @@ regarding possible target volume and pool changes a= s a result of the pool refresh when the upload is attempted. =20 =3Ditem B [I<--pool> I] [I<--offset> I] -[I<--length> I] I I +[I<--length> I] [I<--sparse>] I I =20 Download the contents of a storage volume to I. I<--pool> I is the name or UUID of the storage pool the volu= me is in. I is the name or key or path of the volume to dow= nload. +If I<--sparse> is specified, this command will preserve volume sparseness. I<--offset> is the position in the storage volume at which to start reading the data. The value must be 0 or larger. I<--length> is an upper bound of the amount of data to be downloaded. A negative value is interpreted as --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 00:09:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1494945667763764.7047082552136; Tue, 16 May 2017 07:41:07 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B222E2D9FF8; Tue, 16 May 2017 14:40:18 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 147EF183E3; Tue, 16 May 2017 14:40:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id ED6584BB7F; Tue, 16 May 2017 14:39:44 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v4GE4q5E028028 for ; Tue, 16 May 2017 10:04:52 -0400 Received: by smtp.corp.redhat.com (Postfix) id 5E03B860E0; Tue, 16 May 2017 14:04:52 +0000 (UTC) Received: from moe.brq.redhat.com (dhcp129-131.brq.redhat.com [10.34.129.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id DA23262926 for ; Tue, 16 May 2017 14:04:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B222E2D9FF8 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B222E2D9FF8 From: Michal Privoznik To: libvir-list@redhat.com Date: Tue, 16 May 2017 16:04:11 +0200 Message-Id: <14ba6ce85934cc2cd53286c71f0805c63ed76719.1494943031.git.mprivozn@redhat.com> In-Reply-To: References: In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 31/31] virsh: Implement sparse stream to vol-upload X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 16 May 2017 14:41:06 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Similarly to previous commit, implement sparse streams feature for vol-upload. This is, however, slightly different approach, because we must implement a function that will tell us whether we are in a data section or in a hole. But there's no magic hidden in here. Signed-off-by: Michal Privoznik Reviewed-by: John Ferlan --- tools/virsh-util.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh-util.h | 24 ++++++++++++++++++++++++ tools/virsh-volume.c | 38 +++++++++++++++++++++++++------------- tools/virsh.pod | 3 ++- 4 files changed, 98 insertions(+), 14 deletions(-) diff --git a/tools/virsh-util.c b/tools/virsh-util.c index 198625bdb..44be3ad64 100644 --- a/tools/virsh-util.c +++ b/tools/virsh-util.c @@ -153,6 +153,35 @@ virshStreamSink(virStreamPtr st ATTRIBUTE_UNUSED, } =20 =20 +int +virshStreamSource(virStreamPtr st ATTRIBUTE_UNUSED, + char *bytes, + size_t nbytes, + void *opaque) +{ + virshStreamCallbackDataPtr cbData =3D opaque; + int fd =3D cbData->fd; + + return saferead(fd, bytes, nbytes); +} + + +int +virshStreamSourceSkip(virStreamPtr st ATTRIBUTE_UNUSED, + long long offset, + void *opaque) +{ + virshStreamCallbackDataPtr cbData =3D opaque; + int fd =3D cbData->fd; + off_t cur; + + if ((cur =3D lseek(fd, offset, SEEK_CUR)) =3D=3D (off_t) -1) + return -1; + + return 0; +} + + int virshStreamSkip(virStreamPtr st ATTRIBUTE_UNUSED, long long offset, @@ -171,6 +200,24 @@ virshStreamSkip(virStreamPtr st ATTRIBUTE_UNUSED, } =20 =20 +int +virshStreamInData(virStreamPtr st ATTRIBUTE_UNUSED, + int *inData, + long long *offset, + void *opaque) +{ + virshStreamCallbackDataPtr cbData =3D opaque; + vshControl *ctl =3D cbData->ctl; + int fd =3D cbData->fd; + int ret; + + if ((ret =3D virFileInData(fd, inData, offset)) < 0) + vshError(ctl, "%s", _("Unable to get current position in stream")); + + return ret; +} + + void virshDomainFree(virDomainPtr dom) { diff --git a/tools/virsh-util.h b/tools/virsh-util.h index 0aba247f6..9a0af3513 100644 --- a/tools/virsh-util.h +++ b/tools/virsh-util.h @@ -57,11 +57,35 @@ virshStreamSink(virStreamPtr st, size_t nbytes, void *opaque); =20 +typedef struct _virshStreamCallbackData virshStreamCallbackData; +typedef virshStreamCallbackData *virshStreamCallbackDataPtr; +struct _virshStreamCallbackData { + vshControl *ctl; + int fd; +}; + +int +virshStreamSource(virStreamPtr st, + char *bytes, + size_t nbytes, + void *opaque); + +int +virshStreamSourceSkip(virStreamPtr st, + long long offset, + void *opaque); + int virshStreamSkip(virStreamPtr st, long long offset, void *opaque); =20 +int +virshStreamInData(virStreamPtr st, + int *inData, + long long *offset, + void *opaque); + int virshDomainGetXMLFromDom(vshControl *ctl, virDomainPtr dom, diff --git a/tools/virsh-volume.c b/tools/virsh-volume.c index 3d19b745e..0736bdcdb 100644 --- a/tools/virsh-volume.c +++ b/tools/virsh-volume.c @@ -660,18 +660,13 @@ static const vshCmdOptDef opts_vol_upload[] =3D { .type =3D VSH_OT_INT, .help =3D N_("amount of data to upload") }, + {.name =3D "sparse", + .type =3D VSH_OT_BOOL, + .help =3D N_("preserve sparseness of volume") + }, {.name =3D NULL} }; =20 -static int -cmdVolUploadSource(virStreamPtr st ATTRIBUTE_UNUSED, - char *bytes, size_t nbytes, void *opaque) -{ - int *fd =3D opaque; - - return saferead(*fd, bytes, nbytes); -} - static bool cmdVolUpload(vshControl *ctl, const vshCmd *cmd) { @@ -683,6 +678,8 @@ cmdVolUpload(vshControl *ctl, const vshCmd *cmd) const char *name =3D NULL; unsigned long long offset =3D 0, length =3D 0; virshControlPtr priv =3D ctl->privData; + unsigned int flags =3D 0; + virshStreamCallbackData cbData; =20 if (vshCommandOptULongLong(ctl, cmd, "offset", &offset) < 0) return false; @@ -701,19 +698,34 @@ cmdVolUpload(vshControl *ctl, const vshCmd *cmd) goto cleanup; } =20 + cbData.ctl =3D ctl; + cbData.fd =3D fd; + + if (vshCommandOptBool(cmd, "sparse")) + flags |=3D VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM; + if (!(st =3D virStreamNew(priv->conn, 0))) { vshError(ctl, _("cannot create a new stream")); goto cleanup; } =20 - if (virStorageVolUpload(vol, st, offset, length, 0) < 0) { + if (virStorageVolUpload(vol, st, offset, length, flags) < 0) { vshError(ctl, _("cannot upload to volume %s"), name); goto cleanup; } =20 - if (virStreamSendAll(st, cmdVolUploadSource, &fd) < 0) { - vshError(ctl, _("cannot send data to volume %s"), name); - goto cleanup; + if (flags & VIR_STORAGE_VOL_UPLOAD_SPARSE_STREAM) { + if (virStreamSparseSendAll(st, virshStreamSource, + virshStreamInData, + virshStreamSourceSkip, &cbData) < 0) { + vshError(ctl, _("cannot send data to volume %s"), name); + goto cleanup; + } + } else { + if (virStreamSendAll(st, virshStreamSource, &cbData) < 0) { + vshError(ctl, _("cannot send data to volume %s"), name); + goto cleanup; + } } =20 if (VIR_CLOSE(fd) < 0) { diff --git a/tools/virsh.pod b/tools/virsh.pod index dcaa0c170..42769170d 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -3924,13 +3924,14 @@ the storage volume should be deleted as well. Not a= ll storage drivers support this option, presently only rbd. =20 =3Ditem B [I<--pool> I] [I<--offset> I] -[I<--length> I] I I +[I<--length> I] [I<--sparse>] I I =20 Upload the contents of I to a storage volume. I<--pool> I is the name or UUID of the storage pool the volu= me is in. I is the name or key or path of the volume where = the file will be uploaded. +If I<--sparse> is specified, this command will preserve volume sparseness. I<--offset> is the position in the storage volume at which to start writing the data. The value must be 0 or larger. I<--length> is an upper bound of the amount of data to be uploaded. A negative value is interpreted --=20 2.13.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list