From nobody Fri Dec 19 17:10:50 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1574804440; cv=none; d=zohomail.com; s=zohoarc; b=etZhDh+d5ioFk3amUAMNJKYDgBaKMdQk6/QsyALfke81+zSzkF6Pqz6bCd0B0kmaLwvcH2O4nAt1dZW//djLMoEEYXkSya10z5NMfHCfguuB8zvwj8aM20/XR5Sg9kUCTng4VOT/qckXJ+vyBbgW+CF5oD+5IediZHBL/tEej6g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1574804440; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8gJIvtfRQI1/O/RCSxrDDf/V2U05lC/qmQrK4FL6mTo=; b=WDuRkS2s7sa9mvN8Gu8pP/59NL23Aec6vyRHKrEUWziG7j+K3nKC4Gh4/SmkMBuE+twmhU8udAf6+v6uSHd3yAailA0cxjo9FNY9OsvVxpa6pgnzGUiNKLR4gPwHJm4jy9IVmDEIpNrYHJprYWAgU/gQkly2ZlzljcUaYDfkUmc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1574804440855376.9671132437428; Tue, 26 Nov 2019 13:40:40 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-138-cmsIZNHcN8-UP0YvENtJTg-1; Tue, 26 Nov 2019 16:40:36 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 960FBDBAD; Tue, 26 Nov 2019 21:40:30 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5E6A219481; Tue, 26 Nov 2019 21:40:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1E7DB18089D6; Tue, 26 Nov 2019 21:40:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xAQLeD8A027821 for ; Tue, 26 Nov 2019 16:40:13 -0500 Received: by smtp.corp.redhat.com (Postfix) id C4E74600CA; Tue, 26 Nov 2019 21:40:13 +0000 (UTC) Received: from angien.redhat.com (unknown [10.43.2.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2454E600C8 for ; Tue, 26 Nov 2019 21:40:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574804438; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=8gJIvtfRQI1/O/RCSxrDDf/V2U05lC/qmQrK4FL6mTo=; b=Sh6vLS7qQp/EGMyaPQh5OdnVSC9bQtRTOJklq9mN36P6eg/Q0QP2kNlnohV1/skmochXxR j1fC4nz+c3sAcOiwoygS8+KZ8jq35pu0Prt1wgKeaTaE8JipFz2gRZQr56ouQV4Ks4N975 Z6SF2YhQtsw7XNQuUkMwFuPLOrDtXtw= From: Peter Krempa To: libvir-list@redhat.com Date: Tue, 26 Nov 2019 22:39:49 +0100 Message-Id: <3b8cba586ff7f3ce61c896a16cf0ab5a342a321d.1574803735.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 03/21] backup: Document new XML for backups X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: cmsIZNHcN8-UP0YvENtJTg-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" From: Eric Blake Prepare for new backup APIs by describing the XML that will represent a backup. The XML resembles snapshots and checkpoints in being able to select actions for a set of disks, but has other differences. It can support both push model (the hypervisor does the backup directly into the destination file) and pull model (the hypervisor exposes an access port for a third party to grab what is necessary). Add testsuite coverage for some minimal uses of the XML. The element within tries to model the same elements as a under , but sharing the RNG grammar proved to be hairy. That is in part because while use to describe a host resource in use by the guest, a backup job is using a host resource that is not visible to the guest: a push backup action is instead describing a (which ultimately could be a remote network resource, but for simplicity the RNG just validates a local file for now), and a pull backup action is instead describing a temporary local file (which probably should not be a remote resource). A future refactoring may thus introduce some way to parameterize RNG to accept ... so that the name of the subelement can be for domain, or or as needed for backups. Future patches may improve this area of code. Signed-off-by: Eric Blake Signed-off-by: Peter Krempa --- docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 161 ++++++++++++++ docs/formatcheckpoint.html.in | 12 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 214 +++++++++++++++++++ libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + tests/Makefile.am | 3 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/virschematest.c | 2 + 16 files changed, 438 insertions(+), 8 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml diff --git a/docs/docs.html.in b/docs/docs.html.in index 268c16f3b3..f8a949bb53 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -82,7 +82,8 @@ node devices, secrets, snapshots, - checkpoints + checkpoints, + backup jobs
URI format
The URI formats used for connecting to libvirt
diff --git a/docs/format.html.in b/docs/format.html.in index 3be2237663..d013528fe0 100644 --- a/docs/format.html.in +++ b/docs/format.html.in @@ -27,6 +27,7 @@
  • Secrets
  • Snapshots
  • Checkpoints
    • +
  • Backup jobs

    • Command line validation

    diff --git a/docs/formatbackup.html.in b/docs/formatbackup.html.in new file mode 100644 index 0000000000..b417331dfe --- /dev/null +++ b/docs/formatbackup.html.in @@ -0,0 +1,161 @@ + + + + +

    Backup XML format

    + +
      + +

      Backup XML

      + +

      + Creating a backup, whether full or incremental, is done + via virDomainBackupBegin(), which takes an XML + description of the actions to perform, as well as an optional + second XML document describing a + checkpoint to create at the same point in time. See + also a comparison between + the various state capture APIs. +

      +

      + There are two general modes for backups: a push mode (where the + hypervisor writes out the data to the destination file, which + may be local or remote), and a pull mode (where the hypervisor + creates an NBD server that a third-party client can then read as + needed, and which requires the use of temporary storage, + typically local, until the backup is complete). +

      +

      + The instructions for beginning a backup job are provided as + attributes and elements of the + top-level domainbackup element. This element + includes an optional attribute mode which can be + either "push" or "pull" (default + push). virDomainBackupGetXMLDesc() can be used to + see the actual values selected for elements omitted during + creation (for example, learning which port the NBD server is + using in the pull model or what file names libvirt generated + when none were supplied). The following child elements and attributes + are supported: +

      +
      +
      incremental
      +
      An optional element giving the name of an existing + checkpoint of the domain, which will be used to make this + backup an incremental one. In the push model, only changes + since the named checkpoint are written to the destination. In + the pull model, the NBD server uses the + NBD_OPT_SET_META_CONTEXT extension to advertise to the client + which portions of the export contain changes since the named + checkpoint. If omitted, a full backup is performed. +
      +
      server
      +
      Present only for a pull mode backup. Contains the same + attributes as + the protocol + element of a disk attached via NBD in the domain (such as + transport, socket, name, port, or tls), necessary to set up an + NBD server that exposes the content of each disk at the time + the backup is started. +
      +
      disks
      +
      An optional listing of instructions for disks participating + in the backup (if omitted, all disks participate and libvirt + attempts to generate filenames by appending the current + timestamp as a suffix). If the entire element was omitted on + input, then all disks participate in the backup, otherwise, + only the disks explicitly listed which do not also + use backup=3D'no' will participate. On output, this + is the state of each of the domain's disk in relation to the + backup operation. +
      +
      disk
      +
      This sub-element describes the backup properties of a + specific disk, with the following attributes and child + elements: +
      +
      name
      +
      A mandatory attribute which must match + the <target dev=3D'name'/> + of one of + the disk + devices specified for the domain at the time of + the checkpoint.
      +
      backup
      +
      Setting this attribute to yes(default) spec= ifies + that the disk should take part in the backup and using + no excludes the disk from the backup.
      +
      type
      +
      A mandatory attribute to describe the type of the + disk, except when backup=3D'no' is + used. Valid values include file, + block, or network. + Similar to a disk declaration for a domain, the choice of = type + controls what additional sub-elements are needed to descri= be + the destination (such as protocol for a + network destination).
      +
      target
      +
      Valid only for push mode backups, this is the + primary sub-element that describes the file name of + the backup destination, similar to + the source sub-element of a domain + disk. An optional sub-element driver can + also be used, with an attribute type to + specify a destination format different from + qcow2.
      +
      scratch
      +
      Valid only for pull mode backups, this is the + primary sub-element that describes the file name of + the local scratch file to be used in facilitating the + backup, and is similar to the source + sub-element of a domain disk.
      +
      +
      +
      +
      +
      + +

      Examples

      + +

      Use virDomainBackupBegin() to perform a full + backup using push mode. The example lets libvirt pick the + destination and format for 'vda', fully specifies that we want a + raw backup of 'vdb', and omits 'vdc' from the operation. +

      + 
      +<domainbackup>
+  <disks/>
+    <disk name=3D'vda' backup=3D'yes'/>
+    <disk name=3D'vdb' type=3D'file'>
+      <target file=3D'/path/to/vdb.backup'/>
+      <driver type=3D'raw'/>
+    </disk>
+    <disk name=3D'vdc' backup=3D'no'/>
+  </disks/>
+</domainbackup>
+
      + +

      If the previous full backup also passed a parameter describing + checkpoint XML that resulted + in a checkpoint named 1525889631, we can make + another call to virDomainBackupBegin() to perform + an incremental backup of just the data changed since that + checkpoint, this time using the following XML to start a pull + model export of the 'vda' and 'vdb' disks, where a third-party + NBD client connecting to '/path/to/server' completes the backup + (omitting 'vdc' from the explicit list has the same effect as + the backup=3D'no' from the previous example): +

      + 
      +<domainbackup mode=3D"pull">
+  <incremental>1525889631</incremental>
+  <server transport=3D"unix" socket=3D"/path/to/server"/>
+  <disks/>
+    <disk name=3D'vda' backup=3D'yes' type=3D'file'>
+      <scratch file=3D'/path/to/file1.scratch'/>
+    </disk>
+  </disks/>
+</domainbackup>
+
      + + diff --git a/docs/formatcheckpoint.html.in b/docs/formatcheckpoint.html.in index 044bbfe4b0..ee56194523 100644 --- a/docs/formatcheckpoint.html.in +++ b/docs/formatcheckpoint.html.in @@ -28,12 +28,12 @@ first checkpoint and the second backup operation), it is possible to do an offline reconstruction of the state of the disk at the time of the second backup without having to copy as - much data as a second full backup would require. Future API - additions will make it possible to create checkpoints in - conjunction with a backup - via virDomainBackupBegin() or with an external - snapshot via virDomainSnapshotCreateXML2; but for - now, libvirt exposes enough support to create disk checkpoints + much data as a second full backup would require. Most disk + checkpoints are created in conjunction with a backup + via virDomainBackupBegin(), although a future API + addition of virDomainSnapshotCreateXML2() will also + make this possible when creating external snapshots; however, + libvirt also exposes enough support to create disk checkpoints independently from a backup operation via virDomainCheckpointCreateXML() since 5.6.0. Likewise, the creation of checkpoints when diff --git a/docs/index.html.in b/docs/index.html.in index 7d0ab650e3..26e8406917 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -59,7 +59,8 @@ node devices, secrets, snapshots, - checkpoints + checkpoints, + backup jobs
      Wiki
      Read further community contributed content
      diff --git a/docs/schemas/domainbackup.rng b/docs/schemas/domainbackup.rng new file mode 100644 index 0000000000..cb6e12b2a7 --- /dev/null +++ b/docs/schemas/domainbackup.rng @@ -0,0 +1,214 @@ + + + + + + + + + + + + + + + + + + + + + + push + + + + + + + pull + + + + + + + + tcp + + + + + + + + + + + + + + + + + + unix + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + begin + inprogress + ready + + + + + + + + + + + + + + + + + + + on + + + + + + no + + + + + + + + file + + + + + + + + + + + + + + + + + block + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + file + + + + + + + + + + + + + block + + + + + + + + + + + + + + + + + diff --git a/libvirt.spec.in b/libvirt.spec.in index a6219da604..40d7f91451 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1884,6 +1884,7 @@ exit 0 %{_datadir}/libvirt/schemas/capability.rng %{_datadir}/libvirt/schemas/cputypes.rng %{_datadir}/libvirt/schemas/domain.rng +%{_datadir}/libvirt/schemas/domainbackup.rng %{_datadir}/libvirt/schemas/domaincaps.rng %{_datadir}/libvirt/schemas/domaincheckpoint.rng %{_datadir}/libvirt/schemas/domaincommon.rng diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index c29f3eeed2..453570fc3c 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -233,6 +233,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-gue= sts.sh %{mingw32_datadir}/libvirt/schemas/capability.rng %{mingw32_datadir}/libvirt/schemas/cputypes.rng %{mingw32_datadir}/libvirt/schemas/domain.rng +%{mingw32_datadir}/libvirt/schemas/domainbackup.rng %{mingw32_datadir}/libvirt/schemas/domaincaps.rng %{mingw32_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw32_datadir}/libvirt/schemas/domaincommon.rng @@ -324,6 +325,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-gue= sts.sh %{mingw64_datadir}/libvirt/schemas/capability.rng %{mingw64_datadir}/libvirt/schemas/cputypes.rng %{mingw64_datadir}/libvirt/schemas/domain.rng +%{mingw64_datadir}/libvirt/schemas/domainbackup.rng %{mingw64_datadir}/libvirt/schemas/domaincaps.rng %{mingw64_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw64_datadir}/libvirt/schemas/domaincommon.rng diff --git a/tests/Makefile.am b/tests/Makefile.am index e009de830c..775526c358 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -91,6 +91,9 @@ EXTRA_DIST =3D \ commanddata \ cputestdata \ domaincapsdata \ + domainbackupxml2xmlin \ + domainbackupxml2xmlout \ + domaincapsschemadata \ domainconfdata \ domainschemadata \ fchostdata \ diff --git a/tests/domainbackupxml2xmlin/backup-pull.xml b/tests/domainback= upxml2xmlin/backup-pull.xml new file mode 100644 index 0000000000..2ce5cd6711 --- /dev/null +++ b/tests/domainbackupxml2xmlin/backup-pull.xml @@ -0,0 +1,9 @@ + + 1525889631 + + + + + + + diff --git a/tests/domainbackupxml2xmlin/backup-push.xml b/tests/domainback= upxml2xmlin/backup-push.xml new file mode 100644 index 0000000000..1b7d3061fd --- /dev/null +++ b/tests/domainbackupxml2xmlin/backup-push.xml @@ -0,0 +1,9 @@ + + 1525889631 + + + + + + + diff --git a/tests/domainbackupxml2xmlin/empty.xml b/tests/domainbackupxml2= xmlin/empty.xml new file mode 100644 index 0000000000..7ed511f97b --- /dev/null +++ b/tests/domainbackupxml2xmlin/empty.xml @@ -0,0 +1 @@ + diff --git a/tests/domainbackupxml2xmlout/backup-pull.xml b/tests/domainbac= kupxml2xmlout/backup-pull.xml new file mode 100644 index 0000000000..2ce5cd6711 --- /dev/null +++ b/tests/domainbackupxml2xmlout/backup-pull.xml @@ -0,0 +1,9 @@ + + 1525889631 + + + + + + + diff --git a/tests/domainbackupxml2xmlout/backup-push.xml b/tests/domainbac= kupxml2xmlout/backup-push.xml new file mode 100644 index 0000000000..1b7d3061fd --- /dev/null +++ b/tests/domainbackupxml2xmlout/backup-push.xml @@ -0,0 +1,9 @@ + + 1525889631 + + + + + + + diff --git a/tests/domainbackupxml2xmlout/empty.xml b/tests/domainbackupxml= 2xmlout/empty.xml new file mode 100644 index 0000000000..13600fbb1c --- /dev/null +++ b/tests/domainbackupxml2xmlout/empty.xml @@ -0,0 +1,7 @@ + + + + + + + diff --git a/tests/virschematest.c b/tests/virschematest.c index df50ef1717..e4a440afb0 100644 --- a/tests/virschematest.c +++ b/tests/virschematest.c @@ -205,6 +205,8 @@ mymain(void) "genericxml2xmloutdata", "xlconfigdata", "libxlxml2domconf= igdata", "qemuhotplugtestdomains"); DO_TEST_DIR("domaincaps.rng", "domaincapsdata"); + DO_TEST_DIR("domainbackup.rng", "domainbackupxml2xmlin", + "domainbackupxml2xmlout"); DO_TEST_DIR("domaincheckpoint.rng", "qemudomaincheckpointxml2xmlin", "qemudomaincheckpointxml2xmlout"); DO_TEST_DIR("domainsnapshot.rng", "qemudomainsnapshotxml2xmlin", --=20 2.23.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list