[libvirt] [PATCH v7 00/23] Incremental backups

Eric Blake posted 23 patches 5 years ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20190327100734.10225-1-eblake@redhat.com
Test syntax-check passed
There is a newer version of this series
include/libvirt/libvirt-domain-checkpoint.h   |  161 ++
include/libvirt/libvirt-domain-snapshot.h     |    8 +-
include/libvirt/libvirt-domain.h              |   47 +-
include/libvirt/libvirt.h                     |    5 +-
src/access/viraccessperm.h                    |    8 +-
src/conf/backup_conf.h                        |   96 +
src/conf/checkpoint_conf.h                    |  103 ++
src/conf/domain_conf.h                        |    2 +
src/conf/virconftypes.h                       |   12 +
src/conf/virdomaincheckpointobjlist.h         |   74 +
src/conf/virdomainmomentobjlist.h             |    9 +-
src/conf/virdomainobjlist.h                   |    9 +-
src/conf/virdomainsnapshotobjlist.h           |    4 +-
src/driver-hypervisor.h                       |   81 +-
src/qemu/qemu_block.h                         |    3 +
src/qemu/qemu_blockjob.h                      |    1 +
src/qemu/qemu_capabilities.h                  |    4 +
src/qemu/qemu_conf.h                          |    2 +
src/qemu/qemu_domain.h                        |   39 +-
src/qemu/qemu_monitor.h                       |   21 +-
src/qemu/qemu_monitor_json.h                  |   21 +-
tools/virsh-checkpoint.h                      |   29 +
tools/virsh-completer.h                       |    4 +
tools/virsh-util.h                            |    3 +
tools/virsh.h                                 |    1 +
docs/Makefile.am                              |    3 +
docs/apibuild.py                              |    2 +
docs/docs.html.in                             |   10 +-
docs/domainstatecapture.html.in               |  315 ++++
docs/format.html.in                           |    2 +
docs/formatbackup.html.in                     |  184 ++
docs/formatcheckpoint.html.in                 |  204 +++
docs/formatsnapshot.html.in                   |    2 +
docs/index.html.in                            |    4 +-
docs/schemas/domainbackup.rng                 |  219 +++
docs/schemas/domaincheckpoint.rng             |   87 +
examples/object-events/event-test.c           |    3 +
libvirt.spec.in                               |    3 +
mingw-libvirt.spec.in                         |    6 +
po/POTFILES                                   |    3 +
src/Makefile.am                               |    2 +
src/access/viraccessperm.c                    |    5 +-
src/conf/Makefile.inc.am                      |    6 +
src/conf/backup_conf.c                        |  544 ++++++
src/conf/checkpoint_conf.c                    |  639 +++++++
src/conf/domain_conf.c                        |    8 +-
src/conf/snapshot_conf.c                      |    2 +-
src/conf/virdomaincheckpointobjlist.c         |  222 +++
src/conf/virdomainmomentobjlist.c             |   49 +-
src/conf/virdomainobjlist.c                   |   13 +-
src/conf/virdomainsnapshotobjlist.c           |   11 +-
src/libvirt-domain-checkpoint.c               |  750 ++++++++
src/libvirt-domain-snapshot.c                 |   89 +
src/libvirt-domain.c                          |  239 ++-
src/libvirt_private.syms                      |   33 +-
src/libvirt_public.syms                       |   20 +
src/qemu/qemu_block.c                         |   12 +
src/qemu/qemu_capabilities.c                  |    6 +
src/qemu/qemu_conf.c                          |    5 +
src/qemu/qemu_domain.c                        |  249 ++-
src/qemu/qemu_driver.c                        | 1609 ++++++++++++++++-
src/qemu/qemu_migration.c                     |    2 +-
src/qemu/qemu_monitor.c                       |   65 +-
src/qemu/qemu_monitor_json.c                  |  201 +-
src/qemu/qemu_process.c                       |    7 +
src/remote/remote_daemon_dispatch.c           |   22 +-
src/remote/remote_driver.c                    |   34 +-
src/remote/remote_protocol.x                  |  259 ++-
src/remote_protocol-structs                   |  139 ++
src/rpc/gendispatch.pl                        |   34 +-
src/test/test_driver.c                        |    7 +-
tests/Makefile.am                             |   13 +-
tests/domainbackupxml2xmlin/backup-pull.xml   |    9 +
tests/domainbackupxml2xmlin/backup-push.xml   |    9 +
tests/domainbackupxml2xmlin/empty.xml         |    1 +
tests/domainbackupxml2xmlout/backup-pull.xml  |    9 +
tests/domainbackupxml2xmlout/backup-push.xml  |    9 +
tests/domainbackupxml2xmlout/empty.xml        |    7 +
tests/domaincheckpointxml2xmlin/empty.xml     |    1 +
tests/domaincheckpointxml2xmlin/redefine.xml  |   63 +
tests/domaincheckpointxml2xmlin/sample.xml    |    7 +
tests/domaincheckpointxml2xmlin/size.xml      |    4 +
tests/domaincheckpointxml2xmlout/empty.xml    |    7 +
tests/domaincheckpointxml2xmlout/sample.xml   |   12 +
tests/domaincheckpointxml2xmlout/size.xml     |   11 +
tests/domaincheckpointxml2xmltest.c           |  212 +++
.../caps_4.0.0.riscv32.xml                    |    2 +
.../caps_4.0.0.riscv64.xml                    |    2 +
.../caps_4.0.0.x86_64.xml                     |    2 +
tests/qemumonitorjsontest.c                   |    2 +-
tests/virschematest.c                         |    4 +
tools/Makefile.am                             |    3 +-
tools/virsh-checkpoint.c                      | 1370 ++++++++++++++
tools/virsh-completer.c                       |   53 +-
tools/virsh-domain-monitor.c                  |   25 +-
tools/virsh-domain.c                          |  270 ++-
tools/virsh-snapshot.c                        |   37 +-
tools/virsh-util.c                            |   11 +
tools/virsh.c                                 |    2 +
tools/virsh.pod                               |  302 +++-
100 files changed, 9373 insertions(+), 163 deletions(-)
create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
create mode 100644 src/conf/backup_conf.h
create mode 100644 src/conf/checkpoint_conf.h
create mode 100644 src/conf/virdomaincheckpointobjlist.h
create mode 100644 tools/virsh-checkpoint.h
create mode 100644 docs/domainstatecapture.html.in
create mode 100644 docs/formatbackup.html.in
create mode 100644 docs/formatcheckpoint.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 docs/schemas/domaincheckpoint.rng
create mode 100644 src/conf/backup_conf.c
create mode 100644 src/conf/checkpoint_conf.c
create mode 100644 src/conf/virdomaincheckpointobjlist.c
create mode 100644 src/libvirt-domain-checkpoint.c
create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlin/empty.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/redefine.xml
create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
create mode 100644 tests/domaincheckpointxml2xmlin/size.xml
create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
create mode 100644 tests/domaincheckpointxml2xmlout/size.xml
create mode 100644 tests/domaincheckpointxml2xmltest.c
create mode 100644 tools/virsh-checkpoint.c
[libvirt] [PATCH v7 00/23] Incremental backups
Posted by Eric Blake 5 years ago
I've finished rebasing my earlier v4/v5 patches on top of cleanups
earlier in the series (v6 didn't include the second half of the
series). I've pushed a tag backup-v7 to both my libvirt.git and
libvirt-python.git repos to match:
https://repo.or.cz/libvirt/ericb.git/shortlog/refs/tags/backup-v7
https://repo.or.cz/libvirt-python/ericb.git/shortlog/refs/tags/backup-v7

Here's hoping we're happy enough with the API, including the fact that
rudimentary operation of pull mode backups work with qemu 4.0-rc1, for
this to make it into the 5.2 release.

Diffs from v6
- couple more early easy cleanups
- add Dan's R-b where appropriate
- drop any attempt at virDomainGetJobIds(), instead:
- document that job id 0 always works for the current backup job (if
there is just one), which is just fine for now because qemu 4.0 doesn't
permit parallel jobs (and we can do full job id support for a later release)
- split some more patches; finish moving backup code to a new backup_conf.c
and not piggybacked onto checkpoint_conf.c

Eric Blake (23):
  snapshot: Drop pointless function virDomainMomentIsCurrentName
  snapshot: Allow NULL to virDomainSnapshotObjGetDef
  snapshot: Refactor qemu to utilize virDomainMoment more
  backup: Document new XML for checkpoints
  backup: Document new XML for backups
  backup: Introduce virDomainCheckpoint APIs
  backup: Introduce virDomainBackup APIs
  backup: Document nuances between different state capture APIs
  backup: Parse and output checkpoint XML
  backup: Allow for lists of checkpoint objects
  squash to checkpoint list
  backup: Add new domain:checkpoint access control
  backup: Implement backup APIs for remote driver
  backup: Parse and output backup XML
  backup: Implement virsh support for checkpoints
  backup: Implement virsh support for backup
  backup: Add new qemu monitor interactions
  backup: qemu: Implement metadata tracking for checkpoint APIs
  backup: Wire up qemu checkpoint commands over QMP
  backup: qemu: Implement framework for backup job APIs
  backup: Wire up qemu full pull backup commands over QMP
  backup: qemu: Wire up qemu full push backup commands over QMP
  backup: implement qemu incremental pull backup

 include/libvirt/libvirt-domain-checkpoint.h   |  161 ++
 include/libvirt/libvirt-domain-snapshot.h     |    8 +-
 include/libvirt/libvirt-domain.h              |   47 +-
 include/libvirt/libvirt.h                     |    5 +-
 src/access/viraccessperm.h                    |    8 +-
 src/conf/backup_conf.h                        |   96 +
 src/conf/checkpoint_conf.h                    |  103 ++
 src/conf/domain_conf.h                        |    2 +
 src/conf/virconftypes.h                       |   12 +
 src/conf/virdomaincheckpointobjlist.h         |   74 +
 src/conf/virdomainmomentobjlist.h             |    9 +-
 src/conf/virdomainobjlist.h                   |    9 +-
 src/conf/virdomainsnapshotobjlist.h           |    4 +-
 src/driver-hypervisor.h                       |   81 +-
 src/qemu/qemu_block.h                         |    3 +
 src/qemu/qemu_blockjob.h                      |    1 +
 src/qemu/qemu_capabilities.h                  |    4 +
 src/qemu/qemu_conf.h                          |    2 +
 src/qemu/qemu_domain.h                        |   39 +-
 src/qemu/qemu_monitor.h                       |   21 +-
 src/qemu/qemu_monitor_json.h                  |   21 +-
 tools/virsh-checkpoint.h                      |   29 +
 tools/virsh-completer.h                       |    4 +
 tools/virsh-util.h                            |    3 +
 tools/virsh.h                                 |    1 +
 docs/Makefile.am                              |    3 +
 docs/apibuild.py                              |    2 +
 docs/docs.html.in                             |   10 +-
 docs/domainstatecapture.html.in               |  315 ++++
 docs/format.html.in                           |    2 +
 docs/formatbackup.html.in                     |  184 ++
 docs/formatcheckpoint.html.in                 |  204 +++
 docs/formatsnapshot.html.in                   |    2 +
 docs/index.html.in                            |    4 +-
 docs/schemas/domainbackup.rng                 |  219 +++
 docs/schemas/domaincheckpoint.rng             |   87 +
 examples/object-events/event-test.c           |    3 +
 libvirt.spec.in                               |    3 +
 mingw-libvirt.spec.in                         |    6 +
 po/POTFILES                                   |    3 +
 src/Makefile.am                               |    2 +
 src/access/viraccessperm.c                    |    5 +-
 src/conf/Makefile.inc.am                      |    6 +
 src/conf/backup_conf.c                        |  544 ++++++
 src/conf/checkpoint_conf.c                    |  639 +++++++
 src/conf/domain_conf.c                        |    8 +-
 src/conf/snapshot_conf.c                      |    2 +-
 src/conf/virdomaincheckpointobjlist.c         |  222 +++
 src/conf/virdomainmomentobjlist.c             |   49 +-
 src/conf/virdomainobjlist.c                   |   13 +-
 src/conf/virdomainsnapshotobjlist.c           |   11 +-
 src/libvirt-domain-checkpoint.c               |  750 ++++++++
 src/libvirt-domain-snapshot.c                 |   89 +
 src/libvirt-domain.c                          |  239 ++-
 src/libvirt_private.syms                      |   33 +-
 src/libvirt_public.syms                       |   20 +
 src/qemu/qemu_block.c                         |   12 +
 src/qemu/qemu_capabilities.c                  |    6 +
 src/qemu/qemu_conf.c                          |    5 +
 src/qemu/qemu_domain.c                        |  249 ++-
 src/qemu/qemu_driver.c                        | 1609 ++++++++++++++++-
 src/qemu/qemu_migration.c                     |    2 +-
 src/qemu/qemu_monitor.c                       |   65 +-
 src/qemu/qemu_monitor_json.c                  |  201 +-
 src/qemu/qemu_process.c                       |    7 +
 src/remote/remote_daemon_dispatch.c           |   22 +-
 src/remote/remote_driver.c                    |   34 +-
 src/remote/remote_protocol.x                  |  259 ++-
 src/remote_protocol-structs                   |  139 ++
 src/rpc/gendispatch.pl                        |   34 +-
 src/test/test_driver.c                        |    7 +-
 tests/Makefile.am                             |   13 +-
 tests/domainbackupxml2xmlin/backup-pull.xml   |    9 +
 tests/domainbackupxml2xmlin/backup-push.xml   |    9 +
 tests/domainbackupxml2xmlin/empty.xml         |    1 +
 tests/domainbackupxml2xmlout/backup-pull.xml  |    9 +
 tests/domainbackupxml2xmlout/backup-push.xml  |    9 +
 tests/domainbackupxml2xmlout/empty.xml        |    7 +
 tests/domaincheckpointxml2xmlin/empty.xml     |    1 +
 tests/domaincheckpointxml2xmlin/redefine.xml  |   63 +
 tests/domaincheckpointxml2xmlin/sample.xml    |    7 +
 tests/domaincheckpointxml2xmlin/size.xml      |    4 +
 tests/domaincheckpointxml2xmlout/empty.xml    |    7 +
 tests/domaincheckpointxml2xmlout/sample.xml   |   12 +
 tests/domaincheckpointxml2xmlout/size.xml     |   11 +
 tests/domaincheckpointxml2xmltest.c           |  212 +++
 .../caps_4.0.0.riscv32.xml                    |    2 +
 .../caps_4.0.0.riscv64.xml                    |    2 +
 .../caps_4.0.0.x86_64.xml                     |    2 +
 tests/qemumonitorjsontest.c                   |    2 +-
 tests/virschematest.c                         |    4 +
 tools/Makefile.am                             |    3 +-
 tools/virsh-checkpoint.c                      | 1370 ++++++++++++++
 tools/virsh-completer.c                       |   53 +-
 tools/virsh-domain-monitor.c                  |   25 +-
 tools/virsh-domain.c                          |  270 ++-
 tools/virsh-snapshot.c                        |   37 +-
 tools/virsh-util.c                            |   11 +
 tools/virsh.c                                 |    2 +
 tools/virsh.pod                               |  302 +++-
 100 files changed, 9373 insertions(+), 163 deletions(-)
 create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
 create mode 100644 src/conf/backup_conf.h
 create mode 100644 src/conf/checkpoint_conf.h
 create mode 100644 src/conf/virdomaincheckpointobjlist.h
 create mode 100644 tools/virsh-checkpoint.h
 create mode 100644 docs/domainstatecapture.html.in
 create mode 100644 docs/formatbackup.html.in
 create mode 100644 docs/formatcheckpoint.html.in
 create mode 100644 docs/schemas/domainbackup.rng
 create mode 100644 docs/schemas/domaincheckpoint.rng
 create mode 100644 src/conf/backup_conf.c
 create mode 100644 src/conf/checkpoint_conf.c
 create mode 100644 src/conf/virdomaincheckpointobjlist.c
 create mode 100644 src/libvirt-domain-checkpoint.c
 create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlin/empty.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlout/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/redefine.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/size.xml
 create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
 create mode 100644 tests/domaincheckpointxml2xmlout/size.xml
 create mode 100644 tests/domaincheckpointxml2xmltest.c
 create mode 100644 tools/virsh-checkpoint.c

-- 
2.20.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v7 00/23] Incremental backups
Posted by Daniel P. Berrangé 5 years ago
On Wed, Mar 27, 2019 at 05:07:11AM -0500, Eric Blake wrote:
> I've finished rebasing my earlier v4/v5 patches on top of cleanups
> earlier in the series (v6 didn't include the second half of the
> series). I've pushed a tag backup-v7 to both my libvirt.git and
> libvirt-python.git repos to match:
> https://repo.or.cz/libvirt/ericb.git/shortlog/refs/tags/backup-v7
> https://repo.or.cz/libvirt-python/ericb.git/shortlog/refs/tags/backup-v7
> 
> Here's hoping we're happy enough with the API, including the fact that
> rudimentary operation of pull mode backups work with qemu 4.0-rc1, for
> this to make it into the 5.2 release.

I think we need to accept that we've missed the boat for 5.2 now I'm
afraid. While I'm fine with the API design, it feels contrary to our
norms to only push the API and not the implementation. We've seen
some destablization of build from the preparatory patches, so we
shouldn't push our luck by adding more at the last minute. Even if it
merges, it won't be actually usable so there's no reason to rush this
from an upstream POV.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list