From nobody Fri Dec 19 16:06:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592241066; cv=none; d=zohomail.com; s=zohoarc; b=G5nh4qK/tJuHMKnMnPdRz06E8hLwCEzodifpB6WcsUwISK9Xqq55ImteDAR9k6Wvg6O+1uwG4uDhEaOUMKsKjWt8g1/8/QCkCduY6bbDh7JtLoVJIdJCaqVLlkIF4zeOblzCIiGa0vKmtMVNpSdm3teLX4J4zgjItU6DXVTm/vI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592241066; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GB0774F8iPtjHmhI5oNT6mJYgpHpXE65Om0d2jJU2Ao=; b=hOMYQgTqdtgY51J2+v7cyiT+sSCiCy/VMGhTYRyWeV2TLZPT4nkzk4mz/AQ7qfGe4ba3lstkQdWF2ZuvQNg4K3kDw7Couq0dHTfrPsV97TMJi077nsPTkOdS7rK+6oMq1b34SpUMYTOSFAkqsdNk+EyTjV+YhvSBD7qb/PXNYlU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1592241066480951.0023913560264; Mon, 15 Jun 2020 10:11:06 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-458-B0mTpyVqPYO_XXMD2eDM1w-1; Mon, 15 Jun 2020 13:11:02 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 11E90872FEE; Mon, 15 Jun 2020 17:10:57 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E923A768AE; Mon, 15 Jun 2020 17:10:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id BC554833CF; Mon, 15 Jun 2020 17:10:56 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05FHAs25004912 for ; Mon, 15 Jun 2020 13:10:54 -0400 Received: by smtp.corp.redhat.com (Postfix) id A8018100238C; Mon, 15 Jun 2020 17:10:54 +0000 (UTC) Received: from speedmetal.redhat.com (unknown [10.40.208.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 212671001901 for ; Mon, 15 Jun 2020 17:10:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592241065; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=GB0774F8iPtjHmhI5oNT6mJYgpHpXE65Om0d2jJU2Ao=; b=H9rFiT4T+N6r/YUZdI0LB3vWzMaMRBoJaKfDB6Ga9kjHca46D0DkfIPra5P6AAe6bApMna 995uRI9nAyBXhJebcEJKDkvag6c5XmZuGbvo+iJJF+arNOP72cnkSw9etKx0nl6bkmKcBV CYngmhPnYI2GVxeabtETZP0J73zrrvs= X-MC-Unique: B0mTpyVqPYO_XXMD2eDM1w-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 23/32] qemu: backup: Rewrite backup bitmap handling to the new bitmap semantics Date: Mon, 15 Jun 2020 19:10:10 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Reuse qemuBlockGetBitmapMergeActions which allows to remove the ad-hoc iplementatio of bitmap merging for backup. The new approach is simpler and also more robust in case some of the bitmaps break as they remove the dependancy on the whole chain of bitmaps working. The new approach also allows backups if a snapshot is created outside of libvirt. Additionally the code is greatly simplified. Signed-off-by: Peter Krempa Reviewed-by: Eric Blake --- src/qemu/qemu_backup.c | 220 +++--------------- src/qemu/qemu_backup.h | 12 +- tests/qemublocktest.c | 86 ++----- .../backupmerge/empty-out.json | 4 +- 4 files changed, 63 insertions(+), 259 deletions(-) diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c index 495da86ff0..7c337846dc 100644 --- a/src/qemu/qemu_backup.c +++ b/src/qemu/qemu_backup.c @@ -173,199 +173,58 @@ qemuBackupDiskDataCleanup(virDomainObjPtr vm, } -/** - * qemuBackupBeginCollectIncrementalCheckpoints: - * @vm: domain object - * @incrFrom: name of checkpoint representing starting point of incrementa= l backup - * - * Returns a NULL terminated list of pointers to checkpoint definitions in - * chronological order starting from the 'current' checkpoint until reachi= ng - * @incrFrom. - */ -static virDomainMomentDefPtr * -qemuBackupBeginCollectIncrementalCheckpoints(virDomainObjPtr vm, - const char *incrFrom) -{ - virDomainMomentObjPtr n =3D virDomainCheckpointGetCurrent(vm->checkpoi= nts); - g_autofree virDomainMomentDefPtr *incr =3D NULL; - size_t nincr =3D 0; - - while (n) { - virDomainMomentDefPtr def =3D n->def; - - if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0) - return NULL; - - if (STREQ(def->name, incrFrom)) { - def =3D NULL; - if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0) - return NULL; - - return g_steal_pointer(&incr); - } - - if (!n->def->parent_name) - break; - - n =3D virDomainCheckpointFindByName(vm->checkpoints, n->def->paren= t_name); - } - - virReportError(VIR_ERR_OPERATION_INVALID, - _("could not locate checkpoint '%s' for incremental bac= kup"), - incrFrom); - return NULL; -} - - -static int -qemuBackupGetBitmapMergeRange(virStorageSourcePtr from, - const char *bitmapname, - virJSONValuePtr *actions, - virStorageSourcePtr *to, - const char *diskdst, - virHashTablePtr blockNamedNodeData) +int +qemuBackupDiskPrepareOneBitmapsChain(virStorageSourcePtr backingChain, + virStorageSourcePtr targetsrc, + const char *targetbitmap, + const char *incremental, + virJSONValuePtr actions, + virHashTablePtr blockNamedNodeData) { - g_autoptr(virJSONValue) ret =3D virJSONValueNewArray(); - virStorageSourcePtr tmpsrc =3D NULL; - virStorageSourcePtr n; - bool foundbitmap =3D false; + g_autoptr(virJSONValue) tmpactions =3D NULL; - for (n =3D from; virStorageSourceIsBacking(n); n =3D n->backingStore) { - qemuBlockNamedNodeDataBitmapPtr bitmap =3D NULL; - - if (!(bitmap =3D qemuBlockNamedNodeDataGetBitmapByName(blockNamedN= odeData, - n, - bitmapname))) - break; - - foundbitmap =3D true; - - if (bitmap->inconsistent) { - virReportError(VIR_ERR_INVALID_ARG, - _("bitmap '%s' for image '%s%u' is inconsistent= "), - bitmap->name, diskdst, n->id); - return -1; - } - - if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(ret, - n->nodeformat, - bitmapname) <= 0) - return -1; - - tmpsrc =3D n; - } - - if (!foundbitmap) { - virReportError(VIR_ERR_INVALID_ARG, - _("failed to find bitmap '%s' in image '%s%u'"), - bitmapname, diskdst, from->id); + if (qemuBlockGetBitmapMergeActions(backingChain, NULL, targetsrc, + incremental, targetbitmap, NULL, + &tmpactions, + blockNamedNodeData) < 0) return -1; - } - *actions =3D g_steal_pointer(&ret); - *to =3D tmpsrc; + if (tmpactions && + virJSONValueArrayConcat(actions, tmpactions) < 0) + return -1; return 0; } -virJSONValuePtr -qemuBackupDiskPrepareOneBitmapsChain(virDomainMomentDefPtr *incremental, - virStorageSourcePtr backingChain, - virHashTablePtr blockNamedNodeData, - const char *diskdst) -{ - g_autoptr(virJSONValue) ret =3D NULL; - size_t incridx =3D 0; - virStorageSourcePtr n =3D backingChain; - - ret =3D virJSONValueNewArray(); - - for (incridx =3D 0; incremental[incridx]; incridx++) { - g_autoptr(virJSONValue) tmp =3D virJSONValueNewArray(); - virStorageSourcePtr tmpsrc =3D NULL; - virDomainCheckpointDefPtr chkdef =3D (virDomainCheckpointDefPtr) i= ncremental[incridx]; - bool checkpoint_has_disk =3D false; - size_t i; - - for (i =3D 0; i < chkdef->ndisks; i++) { - if (STRNEQ_NULLABLE(diskdst, chkdef->disks[i].name)) - continue; - - if (chkdef->disks[i].type =3D=3D VIR_DOMAIN_CHECKPOINT_TYPE_BI= TMAP) - checkpoint_has_disk =3D true; - - break; - } - - if (!checkpoint_has_disk) { - if (!incremental[incridx + 1]) { - virReportError(VIR_ERR_INVALID_ARG, - _("disk '%s' not found in checkpoint '%s'"), - diskdst, incremental[incridx]->name); - return NULL; - } - - continue; - } - - if (qemuBackupGetBitmapMergeRange(n, incremental[incridx]->name, - &tmp, &tmpsrc, diskdst, - blockNamedNodeData) < 0) - return NULL; - - if (virJSONValueArrayConcat(ret, tmp) < 0) - return NULL; - - n =3D tmpsrc; - } - - return g_steal_pointer(&ret); -} - - static int qemuBackupDiskPrepareOneBitmaps(struct qemuBackupDiskData *dd, virJSONValuePtr actions, - virDomainMomentDefPtr *incremental, virHashTablePtr blockNamedNodeData) { - g_autoptr(virJSONValue) mergebitmaps =3D NULL; - g_autoptr(virJSONValue) mergebitmapsstore =3D NULL; - - if (!(mergebitmaps =3D qemuBackupDiskPrepareOneBitmapsChain(incrementa= l, - dd->domdisk-= >src, - blockNamedNo= deData, - dd->domdisk-= >dst))) - return -1; - - if (!(mergebitmapsstore =3D virJSONValueCopy(mergebitmaps))) - return -1; - - if (qemuMonitorTransactionBitmapAdd(actions, - dd->domdisk->src->nodeformat, - dd->incrementalBitmap, - false, - true, 0) < 0) - return -1; - - if (qemuMonitorTransactionBitmapMerge(actions, - dd->domdisk->src->nodeformat, - dd->incrementalBitmap, - &mergebitmaps) < 0) + if (!qemuBlockBitmapChainIsValid(dd->domdisk->src, + dd->backupdisk->incremental, + blockNamedNodeData)) { + virReportError(VIR_ERR_INVALID_ARG, + _("missing or broken bitmap '%s' for disk '%s'"), + dd->backupdisk->incremental, dd->domdisk->dst); return -1; + } - if (qemuMonitorTransactionBitmapAdd(actions, - dd->store->nodeformat, - dd->incrementalBitmap, - false, - true, 0) < 0) + if (qemuBackupDiskPrepareOneBitmapsChain(dd->domdisk->src, + dd->domdisk->src, + dd->incrementalBitmap, + dd->backupdisk->incremental, + actions, + blockNamedNodeData) < 0) return -1; - if (qemuMonitorTransactionBitmapMerge(actions, - dd->store->nodeformat, - dd->incrementalBitmap, - &mergebitmapsstore) < 0) + if (qemuBackupDiskPrepareOneBitmapsChain(dd->domdisk->src, + dd->store, + dd->incrementalBitmap, + dd->backupdisk->incremental, + actions, + blockNamedNodeData) < 0) return -1; return 0; @@ -382,7 +241,6 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm, virQEMUDriverConfigPtr cfg) { qemuDomainObjPrivatePtr priv =3D vm->privateData; - g_autofree virDomainMomentDefPtr *incremental =3D NULL; /* set data structure */ dd->backupdisk =3D backupdisk; @@ -420,16 +278,12 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm, return -1; if (dd->backupdisk->incremental) { - if (!(incremental =3D qemuBackupBeginCollectIncrementalCheckpoints= (vm, dd->backupdisk->incremental))) - return -1; - if (dd->backupdisk->exportbitmap) dd->incrementalBitmap =3D g_strdup(dd->backupdisk->exportbitma= p); else dd->incrementalBitmap =3D g_strdup_printf("backup-%s", dd->dom= disk->dst); - if (qemuBackupDiskPrepareOneBitmaps(dd, actions, incremental, - blockNamedNodeData) < 0) + if (qemuBackupDiskPrepareOneBitmaps(dd, actions, blockNamedNodeDat= a) < 0) return -1; } @@ -883,8 +737,8 @@ qemuBackupBegin(virDomainObjPtr vm, if (!(blockNamedNodeData =3D qemuBlockGetNamedNodeData(vm, QEMU_ASYNC_= JOB_BACKUP))) goto endjob; - if ((ndd =3D qemuBackupDiskPrepareData(vm, def, blockNamedNodeData, - actions, cfg, &dd)) <=3D 0) { + if ((ndd =3D qemuBackupDiskPrepareData(vm, def, blockNamedNodeData, ac= tions, + cfg, &dd)) <=3D 0) { if (ndd =3D=3D 0) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("no disks selected for backup")); diff --git a/src/qemu/qemu_backup.h b/src/qemu/qemu_backup.h index b19c3bf1c9..075fde709b 100644 --- a/src/qemu/qemu_backup.h +++ b/src/qemu/qemu_backup.h @@ -53,8 +53,10 @@ qemuBackupGetJobInfoStats(virQEMUDriverPtr driver, qemuDomainJobInfoPtr jobInfo); /* exported for testing */ -virJSONValuePtr -qemuBackupDiskPrepareOneBitmapsChain(virDomainMomentDefPtr *incremental, - virStorageSourcePtr backingChain, - virHashTablePtr blockNamedNodeData, - const char *diskdst); +int +qemuBackupDiskPrepareOneBitmapsChain(virStorageSourcePtr backingChain, + virStorageSourcePtr targetsrc, + const char *targetbitmap, + const char *incremental, + virJSONValuePtr actions, + virHashTablePtr blockNamedNodeData); diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c index 9e142a8851..30662f6f79 100644 --- a/tests/qemublocktest.c +++ b/tests/qemublocktest.c @@ -716,65 +716,6 @@ testQemuBitmapGetFakeChainEntry(virStorageSourcePtr sr= c, } -typedef virDomainMomentDefPtr testMomentList; - -static void -testMomentListFree(testMomentList *list) -{ - testMomentList *tmp =3D list; - - if (!list) - return; - - while (*tmp) { - virObjectUnref(*tmp); - tmp++; - } - - g_free(list); -} - -G_DEFINE_AUTOPTR_CLEANUP_FUNC(testMomentList, testMomentListFree); - -static virDomainMomentDefPtr -testQemuBackupGetIncrementalMoment(const char *name) -{ - virDomainCheckpointDefPtr checkpoint =3D NULL; - - if (!(checkpoint =3D virDomainCheckpointDefNew())) - abort(); - - checkpoint->disks =3D g_new0(virDomainCheckpointDiskDef, 1); - checkpoint->ndisks =3D 1; - - checkpoint->disks[0].name =3D g_strdup("testdisk"); - checkpoint->disks[0].type =3D VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP; - - checkpoint->parent.name =3D g_strdup(name); - - return (virDomainMomentDefPtr) checkpoint; -} - - -static virDomainMomentDefPtr * -testQemuBackupGetIncremental(const char *incFrom) -{ - const char *checkpoints[] =3D {"current", "d", "c", "b", "a"}; - virDomainMomentDefPtr *incr; - size_t i; - - incr =3D g_new0(virDomainMomentDefPtr, G_N_ELEMENTS(checkpoints) + 1); - - for (i =3D 0; i < G_N_ELEMENTS(checkpoints); i++) { - incr[i] =3D testQemuBackupGetIncrementalMoment(checkpoints[i]); - - if (STREQ(incFrom, checkpoints[i])) - break; - } - - return incr; -} - static const char *backupDataPrefix =3D "qemublocktestdata/backupmerge/"; struct testQemuBackupIncrementalBitmapCalculateData { @@ -791,10 +732,10 @@ testQemuBackupIncrementalBitmapCalculate(const void *= opaque) const struct testQemuBackupIncrementalBitmapCalculateData *data =3D op= aque; g_autoptr(virJSONValue) nodedatajson =3D NULL; g_autoptr(virHashTable) nodedata =3D NULL; - g_autoptr(virJSONValue) mergebitmaps =3D NULL; - g_autofree char *actual =3D NULL; + g_autoptr(virJSONValue) actions =3D virJSONValueNewArray(); g_autofree char *expectpath =3D NULL; - g_autoptr(testMomentList) incremental =3D NULL; + g_autoptr(virStorageSource) target =3D NULL; + g_auto(virBuffer) buf =3D VIR_BUFFER_INITIALIZER; expectpath =3D g_strdup_printf("%s/%s%s-out.json", abs_srcdir, backupDataPrefix, data->name); @@ -808,19 +749,24 @@ testQemuBackupIncrementalBitmapCalculate(const void *= opaque) return -1; } - incremental =3D testQemuBackupGetIncremental(data->incremental); + if (!(target =3D virStorageSourceNew())) + return -1; - if ((mergebitmaps =3D qemuBackupDiskPrepareOneBitmapsChain(incremental, - data->chain, - nodedata, - "testdisk")))= { - if (!(actual =3D virJSONValueToString(mergebitmaps, true))) + target->nodeformat =3D g_strdup_printf("target_node"); + + if (qemuBackupDiskPrepareOneBitmapsChain(data->chain, + target, + "target-bitmap-name", + data->incremental, + actions, + nodedata) >=3D 0) { + if (virJSONValueToBuffer(actions, &buf, true) < 0) return -1; } else { - actual =3D g_strdup("NULL\n"); + virBufferAddLit(&buf, "NULL\n"); } - return virTestCompareToFile(actual, expectpath); + return virTestCompareToFile(virBufferCurrentContent(&buf), expectpath); } diff --git a/tests/qemublocktestdata/backupmerge/empty-out.json b/tests/qem= ublocktestdata/backupmerge/empty-out.json index 7951defec1..41b42e677b 100644 --- a/tests/qemublocktestdata/backupmerge/empty-out.json +++ b/tests/qemublocktestdata/backupmerge/empty-out.json @@ -1 +1,3 @@ -NULL +[ + +] --=20 2.26.2