From nobody Sun Feb 8 22:06:34 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562471813; cv=none; d=zoho.com; s=zohoarc; b=PKPNnTbpBoOWygnRtTKtdExTYfBaMWiZcA8LFjVPr3BLCZKAG6uvsgE3o/u+ZrifhDoauaocUNoNbBKLMXpRgWyKmKvzvWUCIp+POYSsvOXPlRgnGMZKQbOsh2lSNI5yKGhnAYbAJ/R0zD75KqZQAI6uuFtFqN3O4bXVfKwOhwo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562471813; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ptaL/U/fXR/dRww7LXVCpeEod0UHAtQpFJOTgySwl6A=; b=j7WN3jqZFt+EDe6Q//LYkPXUnEpTKNrhDT+2K8+VWXRON4NFLoosD78kxi9bJghf04Zimw7Vpgu3CSuoQTBV01vNWt9LiTNojlkzsbK9XRjlDO6hnRq/kl31BIyatXIBBZt1RnY4HvxB3qgjx+36efiPchj804OPOJZb2Jtr1EQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1562471813882639.2157128855881; Sat, 6 Jul 2019 20:56:53 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E842C3082A9B; Sun, 7 Jul 2019 03:56:43 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BEE1851F06; Sun, 7 Jul 2019 03:56:43 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 87CF8206D4; Sun, 7 Jul 2019 03:56:43 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x673uGKw006178 for ; Sat, 6 Jul 2019 23:56:16 -0400 Received: by smtp.corp.redhat.com (Postfix) id B3750413A; Sun, 7 Jul 2019 03:56:16 +0000 (UTC) Received: from blue.redhat.com (ovpn-116-78.phx2.redhat.com [10.3.116.78]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33B6018A86; Sun, 7 Jul 2019 03:56:16 +0000 (UTC) From: Eric Blake To: libvir-list@redhat.com Date: Sat, 6 Jul 2019 22:56:01 -0500 Message-Id: <20190707035613.25754-2-eblake@redhat.com> In-Reply-To: <20190707035613.25754-1-eblake@redhat.com> References: <20190707035613.25754-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Cc: nsoffer@redhat.com, eshenitz@redhat.com, pkrempa@redhat.com Subject: [libvirt] [PATCH v9 01/13] backup: Document new XML for checkpoints X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Sun, 07 Jul 2019 03:56:44 +0000 (UTC) Content-Type: text/plain; charset="utf-8" Prepare for new checkpoint APIs by describing the XML that will represent a checkpoint. The checkpoint XML is modeled heavily after virDomainSnapshotPtr. See the docs for more details. Add testsuite coverage for some minimal uses of the XML (bare minimum, the sample from html, and a full dumpxml). Although use of the REDEFINE flag will require the subelement to be present, it is easier for most of the tests to provide counterpart output produced with the NO_DOMAIN flag (particularly since synthesizing a valid during testing is not trivial). Signed-off-by: Eric Blake --- docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatcheckpoint.html.in | 218 ++++++++++++++++++ docs/formatsnapshot.html.in | 4 +- docs/index.html.in | 3 +- docs/schemas/domaincheckpoint.rng | 95 ++++++++ libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + tests/Makefile.am | 2 + tests/qemudomaincheckpointxml2xmlin/empty.xml | 1 + .../qemudomaincheckpointxml2xmlin/sample.xml | 7 + tests/qemudomaincheckpointxml2xmlin/size.xml | 4 + .../qemudomaincheckpointxml2xmlout/empty.xml | 8 + .../redefine.xml | 64 +++++ .../qemudomaincheckpointxml2xmlout/sample.xml | 13 ++ tests/qemudomaincheckpointxml2xmlout/size.xml | 12 + tests/virschematest.c | 2 + 17 files changed, 437 insertions(+), 3 deletions(-) create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 tests/qemudomaincheckpointxml2xmlin/empty.xml create mode 100644 tests/qemudomaincheckpointxml2xmlin/sample.xml create mode 100644 tests/qemudomaincheckpointxml2xmlin/size.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/empty.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/redefine.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/sample.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/size.xml diff --git a/docs/docs.html.in b/docs/docs.html.in index c8674e1457..55cbab5f15 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -81,7 +81,8 @@ storage pool capabilities, node devices, secrets, - snapshots + snapshots, + checkpoints
URI format
The URI formats used for connecting to libvirt
diff --git a/docs/format.html.in b/docs/format.html.in index 640a9957ee..20f9ef3348 100644 --- a/docs/format.html.in +++ b/docs/format.html.in @@ -25,6 +25,7 @@
  • Node devices
  • Secrets
  • Snapshots
    • +
  • Checkpoints

    • Command line validation

    diff --git a/docs/formatcheckpoint.html.in b/docs/formatcheckpoint.html.in new file mode 100644 index 0000000000..ef5f8a826b --- /dev/null +++ b/docs/formatcheckpoint.html.in @@ -0,0 +1,218 @@ + + + + +

    Checkpoint XML format

    + +
      + +

      Checkpoint XML

      + +

      + One method of capturing domain disk backups is via the use of + incremental backups. Right now, incremental backups are only + supported for the QEMU hypervisor when using qcow2 disks at the + active layer; if other disk formats are in use, capturing disk + backups requires different libvirt APIs. +

      +

      + Libvirt is able to facilitate incremental backups by tracking + disk checkpoints, which are points in time against which it is + easy to compute which portion of the disk has changed. Given a + full backup (a backup created from the creation of the disk to a + given point in time), coupled with the creation of a disk + checkpoint at that time, and an incremental backup (a backup + created from just the dirty portion of the disk between the + first checkpoint and the second backup operation), it is + possible to do an offline reconstruction of the state of the + disk at the time of the second backup without having to copy as + much data as a second full backup would require. Future API + additions will make it possible to create checkpoints in + conjunction with a backup + via virDomainBackupBegin() or with an external + snapshot via virDomainSnapshotCreateXML2; but for + now, libvirt exposes enough support to create disk checkpoints + independently from a backup operation + via virDomainCheckpointCreateXML(). +

      +

      + Attributes of libvirt checkpoints are stored as child elements + of the domaincheckpoint element. At checkpoint + creation time, normally only + the name, description, + and disks elements are settable. The rest of the + fields are ignored on creation and will be filled in by libvirt + in for informational purposes + by virDomainCheckpointGetXMLDesc(). However, when + redefining a checkpoint, with + the VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE flag + of virDomainCheckpointCreateXML(), all of the XML + fields described here are relevant on input, even the fields + that are normally described as readonly for output. +

      +

      + A checkpoint is marked current if the hypervisor is actively + updating the checkpoint as guest operations modify the disk + after the point where the checkpoint was created. When tracking + multiple checkpoints, a hypervisor may choose to either have all + relevant checkpoints be current (a guest write updates the + associated tracking for each checkpoint), or to have a single + checkpoint be current (older checkpoints stop tracking changes + when a newer one is created, and describing all changes since an + older checkpoint then involves libvirt automatically combining + the changes from the chain of checkpoints from the current back + to the checkpoint in question; the QEMU hypervisor uses this + method). Checkpoints are maintained in a hierarchy to facilitate + hypervisors that use this latter method, where libvirt + automatically updates the parent-child relationship as well as + which checkpoint is current when creating or deleting + checkpoints. For now, the creation of checkpoints is not + possible when external snapshots exist, but this may be added in + the future. +

      +

      + The top-level domaincheckpoint element may contain + the following elements: +

      +
      +
      name
      +
      The optional name for this checkpoint. If the name is + omitted, libvirt will create a name based on the time of the + creation. +
      +
      description
      +
      An optional human-readable description of the checkpoint. + If the description is omitted when initially creating the + checkpoint, then this field will be empty. +
      +
      disks
      +
      On input, this is an optional listing of specific + instructions for disk checkpoints; it is needed when making a + checkpoint on only a subset of the disks associated with a + domain. In particular, since QEMU checkpoints require qcow2 + disks, this element may be needed on input for excluding guest + disks that are not in qcow2 format. If the entire element was + omitted on input, then all disks participate in the + checkpoint, otherwise, only the disks explicitly listed which + do not also use checkpoint=3D'no' will + participate. On output, this is the checkpoint state of each + of the domain's disks. +
      +
      disk
      +
      This sub-element describes the checkpoint properties of + a specific disk with the following attributes: +
      +
      name
      +
      A mandatory attribute which must match either + the <target dev=3D'name'/> or an + unambiguous <source file=3D'name'/> + of one of + the disk + devices specified for the domain at the time of + the checkpoint.
      +
      checkpoint
      +
      An optional attribute; possible values + are no when the disk does not participate + in this checkpoint; or bitmap if the disk + will track all changes since the creation of this + checkpoint via a bitmap.
      +
      bitmap
      +
      The attribute bitmap is only valid + if checkpoint=3D'bitmap'; it describes the + name of the tracking bitmap (defaulting to the + checkpoint name).
      +
      size
      +
      The attribute size is ignored on input; + on output, it is only present if + the VIR_DOMAIN_CHECKPOINT_XML_SIZE flag + was used to perform a dynamic query of the estimated + size in bytes of the changes made since the checkpoint + was created.
      +
      +
      +
      +
      +
      creationTime
      +
      A readonly representation of the time this checkpoint was + created. The time is specified in seconds since the Epoch, + UTC (i.e. Unix time). +
      +
      parent
      +
      An optional readonly representation of the parent of this + checkpoint. If present, this element contains exactly one + child element, name. +
      +
      current
      +
      A readonly integer, 1 if this checkpoint is current (that + is, actively tracking guest changes) or 0 if the checkpoint is + inactive because a child checkpoint is now tracking changes. +
      +
      domain
      +
      A readonly representation of the + inactive domain configuration + at the time the checkpoint was created. This element may be + omitted for output brevity by supplying + the VIR_DOMAIN_CHECKPOINT_XML_NO_DOMAIN flag, but + the resulting XML is no longer viable for use with + the VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE flag + of virDomainCheckpointCreateXML(). The domain + will have security-sensitive information omitted unless the + flag VIR_DOMAIN_CHECKPOINT_XML_SECURE is provided + on a read-write connection. +
      +
      + +

      Examples

      + +

      Using this XML to create a checkpoint of just vda on a qemu + domain with two disks and a prior checkpoint:

      + 
      +<domaincheckpoint>
+  <description>Completion of updates after OS install</descriptio=
n>
+  <disks>
+    <disk name=3D'vda' checkpoint=3D'bitmap'/>
+    <disk name=3D'vdb' checkpoint=3D'no'/>
+  </disks>
+</domaincheckpoint>
      + +

      will result in XML similar to this from + virDomainCheckpointGetXMLDesc():

      + 
      +<domaincheckpoint>
+  <name>1525889631</name>
+  <description>Completion of updates after OS install</descriptio=
n>
+  <parent>
+    <name>1525111885</name>
+  </parent>
+  <creationTime>1525889631</creationTime>
+  <disks>
+    <disk name=3D'vda' checkpoint=3D'bitmap' bitmap=3D'1525889631'/>
+    <disk name=3D'vdb' checkpoint=3D'no'/>
+  </disks>
+  <domain type=3D'qemu'>
+    <name>fedora</name>
+    <uuid>93a5c045-6457-2c09-e56c-927cdf34e178</uuid>
+    <memory>1048576</memory>
+    ...
+    <devices>
+      <disk type=3D'file' device=3D'disk'>
+        <driver name=3D'qemu' type=3D'qcow2'/>
+        <source file=3D'/path/to/file1'/>
+        <target dev=3D'vda' bus=3D'virtio'/>
+      </disk>
+      <disk type=3D'file' device=3D'disk' snapshot=3D'external'>
+        <driver name=3D'qemu' type=3D'raw'/>
+        <source file=3D'/path/to/file2'/>
+        <target dev=3D'vdb' bus=3D'virtio'/>
+      </disk>
+      ...
+    </devices>
+  </domain>
+</domaincheckpoint>
      + +

      With that checkpoint created, the qcow2 image is now tracking + all changes that occur in the image since the checkpoint via + the persistent bitmap named 1525889631. +

      + + diff --git a/docs/formatsnapshot.html.in b/docs/formatsnapshot.html.in index 92cc566467..2bfb69cf49 100644 --- a/docs/formatsnapshot.html.in +++ b/docs/formatsnapshot.html.in @@ -91,7 +91,9 @@ sets that snapshot as current, and the prior current snapshot is the parent of the new snapshot. Branches in the hierarchy can be formed by reverting to a snapshot with a child, then creating - another snapshot. + another snapshot. For now, the creation of external snapshots + when checkpoints exist is forbidden, although future work will + make it possible to integrate these two concepts.

      The top-level domainsnapshot element may contain diff --git a/docs/index.html.in b/docs/index.html.in index 2bd647f8dd..7d0ab650e3 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -58,7 +58,8 @@ storage pool capabilities, node devices, secrets, - snapshots + snapshots, + checkpoints

      Wiki
      Read further community contributed content
      diff --git a/docs/schemas/domaincheckpoint.rng b/docs/schemas/domaincheckpo= int.rng new file mode 100644 index 0000000000..ceb5436aa3 --- /dev/null +++ b/docs/schemas/domaincheckpoint.rng @@ -0,0 +1,95 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 0 + 1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + no + + + + + bitmap + + + + + + + + + + + + + + + + + + diff --git a/libvirt.spec.in b/libvirt.spec.in index d54f58f1d4..dece22688d 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1780,6 +1780,7 @@ exit 0 %{_datadir}/libvirt/schemas/cputypes.rng %{_datadir}/libvirt/schemas/domain.rng %{_datadir}/libvirt/schemas/domaincaps.rng +%{_datadir}/libvirt/schemas/domaincheckpoint.rng %{_datadir}/libvirt/schemas/domaincommon.rng %{_datadir}/libvirt/schemas/domainsnapshot.rng %{_datadir}/libvirt/schemas/interface.rng diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index ad6d6e952e..df89d9f8b3 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -232,6 +232,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-gue= sts.sh %{mingw32_datadir}/libvirt/schemas/cputypes.rng %{mingw32_datadir}/libvirt/schemas/domain.rng %{mingw32_datadir}/libvirt/schemas/domaincaps.rng +%{mingw32_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw32_datadir}/libvirt/schemas/domaincommon.rng %{mingw32_datadir}/libvirt/schemas/domainsnapshot.rng %{mingw32_datadir}/libvirt/schemas/interface.rng @@ -321,6 +322,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-gue= sts.sh %{mingw64_datadir}/libvirt/schemas/cputypes.rng %{mingw64_datadir}/libvirt/schemas/domain.rng %{mingw64_datadir}/libvirt/schemas/domaincaps.rng +%{mingw64_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw64_datadir}/libvirt/schemas/domaincommon.rng %{mingw64_datadir}/libvirt/schemas/domainsnapshot.rng %{mingw64_datadir}/libvirt/schemas/interface.rng diff --git a/tests/Makefile.am b/tests/Makefile.am index e57d0da58a..66c6ad911e 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -110,6 +110,8 @@ EXTRA_DIST =3D \ qemublocktestdata \ qemucapabilitiesdata \ qemucaps2xmloutdata \ + qemudomaincheckpointxml2xmlin \ + qemudomaincheckpointxml2xmlout \ qemudomainsnapshotxml2xmlin \ qemudomainsnapshotxml2xmlout \ qemuhotplugtestcpus \ diff --git a/tests/qemudomaincheckpointxml2xmlin/empty.xml b/tests/qemudoma= incheckpointxml2xmlin/empty.xml new file mode 100644 index 0000000000..dc36449142 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/empty.xml @@ -0,0 +1 @@ + diff --git a/tests/qemudomaincheckpointxml2xmlin/sample.xml b/tests/qemudom= aincheckpointxml2xmlin/sample.xml new file mode 100644 index 0000000000..70ed964e1e --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/sample.xml @@ -0,0 +1,7 @@ + + Completion of updates after OS install + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlin/size.xml b/tests/qemudomai= ncheckpointxml2xmlin/size.xml new file mode 100644 index 0000000000..99ce84c7a0 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/size.xml @@ -0,0 +1,4 @@ + + c2 + something fun + diff --git a/tests/qemudomaincheckpointxml2xmlout/empty.xml b/tests/qemudom= aincheckpointxml2xmlout/empty.xml new file mode 100644 index 0000000000..ff49bef17f --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/empty.xml @@ -0,0 +1,8 @@ + + 1525889631 + 1525889631 + 0 + + + + diff --git a/tests/qemudomaincheckpointxml2xmlout/redefine.xml b/tests/qemu= domaincheckpointxml2xmlout/redefine.xml new file mode 100644 index 0000000000..44310c2e1d --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/redefine.xml @@ -0,0 +1,64 @@ + + c1 + 1553647812 + 1 + + + + + + + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-i686 + + + + +
      + + + + + + +
      + + + + + +
      + + + + + +
      + + +
      + + +
      + + + + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlout/sample.xml b/tests/qemudo= maincheckpointxml2xmlout/sample.xml new file mode 100644 index 0000000000..913c26d2b9 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/sample.xml @@ -0,0 +1,13 @@ + + 1525889631 + Completion of updates after OS install + + 1525111885 + + 1525889631 + 0 + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlout/size.xml b/tests/qemudoma= incheckpointxml2xmlout/size.xml new file mode 100644 index 0000000000..17d884983d --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/size.xml @@ -0,0 +1,12 @@ + + c2 + something fun + + 1525111885 + + 1553648510 + 1 + + + + diff --git a/tests/virschematest.c b/tests/virschematest.c index b7c2f7cfaa..8f5101df21 100644 --- a/tests/virschematest.c +++ b/tests/virschematest.c @@ -222,6 +222,8 @@ mymain(void) "genericxml2xmloutdata", "xlconfigdata", "libxlxml2domconf= igdata", "qemuhotplugtestdomains"); DO_TEST_DIR("domaincaps.rng", "domaincapsschemadata"); + DO_TEST_DIR("domaincheckpoint.rng", "qemudomaincheckpointxml2xmlin", + "qemudomaincheckpointxml2xmlout"); DO_TEST_DIR("domainsnapshot.rng", "qemudomainsnapshotxml2xmlin", "qemudomainsnapshotxml2xmlout"); DO_TEST_DIR("interface.rng", "interfaceschemadata"); --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list