[libvirt] [PATCH v6 0/9] Incremental backups, focus on API

Eric Blake posted 9 patches 5 years ago
Test syntax-check passed
Failed in applying to current master (apply log)
include/libvirt/virterror.h                  |   7 +-
src/util/virerror.c                          |  12 +-
include/libvirt/libvirt-domain-checkpoint.h  | 161 ++++
include/libvirt/libvirt-domain-snapshot.h    |   7 +-
include/libvirt/libvirt-domain.h             |  59 +-
include/libvirt/libvirt.h                    |   3 +-
src/conf/snapshot_conf.h                     |   7 -
src/conf/virdomainmomentobjlist.h            |   5 +-
src/conf/virdomainsnapshotobjlist.h          |  13 -
src/datatypes.h                              |  33 +-
src/driver-hypervisor.h                      |  87 ++-
src/qemu/qemu_blockjob.h                     |   1 +
docs/Makefile.am                             |   3 +
docs/apibuild.py                             |   2 +
docs/docs.html.in                            |  10 +-
docs/domainstatecapture.html.in              | 313 ++++++++
docs/format.html.in                          |   2 +
docs/formatbackup.html.in                    | 181 +++++
docs/formatcheckpoint.html.in                | 204 +++++
docs/formatsnapshot.html.in                  |  82 +-
docs/index.html.in                           |   4 +-
docs/schemas/domainbackup.rng                | 219 ++++++
docs/schemas/domaincheckpoint.rng            |  94 +++
examples/object-events/event-test.c          |   3 +
libvirt.spec.in                              |   3 +
mingw-libvirt.spec.in                        |   6 +
po/POTFILES                                  |   1 +
src/Makefile.am                              |   2 +
src/conf/domain_conf.c                       |   2 +-
src/conf/snapshot_conf.c                     |   2 +-
src/conf/virdomainsnapshotobjlist.c          | 166 ----
src/datatypes.c                              |  63 +-
src/libvirt-domain-checkpoint.c              | 749 +++++++++++++++++++
src/libvirt-domain-snapshot.c                | 311 ++++----
src/libvirt-domain.c                         | 289 ++++++-
src/libvirt_private.syms                     |   4 +-
src/libvirt_public.syms                      |  21 +
tests/Makefile.am                            |   4 +
tests/domainbackupxml2xmlin/backup-pull.xml  |   9 +
tests/domainbackupxml2xmlin/backup-push.xml  |   9 +
tests/domainbackupxml2xmlin/empty.xml        |   1 +
tests/domainbackupxml2xmlout/backup-pull.xml |   9 +
tests/domainbackupxml2xmlout/backup-push.xml |   9 +
tests/domainbackupxml2xmlout/empty.xml       |   7 +
tests/domaincheckpointxml2xmlin/empty.xml    |   1 +
tests/domaincheckpointxml2xmlin/sample.xml   |   7 +
tests/domaincheckpointxml2xmlout/empty.xml   |  10 +
tests/domaincheckpointxml2xmlout/sample.xml  |  16 +
tests/virschematest.c                        |   4 +
tools/virsh-domain.c                         |   8 +-
50 files changed, 2794 insertions(+), 431 deletions(-)
create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
create mode 100644 docs/domainstatecapture.html.in
create mode 100644 docs/formatbackup.html.in
create mode 100644 docs/formatcheckpoint.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 docs/schemas/domaincheckpoint.rng
create mode 100644 src/libvirt-domain-checkpoint.c
create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlin/empty.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
[libvirt] [PATCH v6 0/9] Incremental backups, focus on API
Posted by Eric Blake 5 years ago
I'm fairly confident that these API are ready to go (that is, I've got
qemu code in the wings to implement these API for the qemu driver, as
demonstrated at last KVM forum, and it shouldn't be too hard to add
support in the test driver to get some 'make check' coverage similar
to what I recently added for snapshots). I'm hoping the APIs make it
in for 5.2, even if I'm still dealing with review churn on the later
parts of my v5 series (there has been a lot of rebasing from earlier
review comments, so v5 is currently still the most recent version that
I was able to run demos with, although I hope to post the rest of v6
soon).

The biggest diffs compared to v4/v5 are addressing many of John
Ferlan's grammar suggestions, removing some TODO items from the public
API (although implementing those changes, such as the new
VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE, is still on my todo list),
splitting some large patches (checkpoints and backups are now
separated), moving the backup APIs into libvirt-domain.c instead of
piggybacking on libvirt-domain-checkpoint.c, and adding 2 more API
(virDomainSnapshotCreateXML2 to create a checkpoint at the same time
as a snapshot; virDomainListJobIds to learn the job ids of all
currently running backup jobs [the initial implementation will only
ever have job id 1 due to qemu 4.0 limitations, but the API was
designed with future expansion in mind]).  I've also got some
preliminary patches to snapshot in this posting based on things
noticed while working checkpoints.

001/9:[down] 'Revert "snapshot: Add virDomainSnapshotObjListParse"'
002/9:[down] 'Revert "snapshot: Add virDomainSnapshotObjListFormat"'
003/9:[down] 'snapshot: Various doc tweaks'
004/9:[0005] [FC] 'backup: Introduce virDomainCheckpointPtr'
005/9:[down] 'backup: Document new XML for checkpoints'
006/9:[0562] [FC] 'backup: Document new XML for backups'
007/9:[0173] [FC] 'backup: Introduce virDomainCheckpoint APIs'
008/9:[0296] [FC] 'backup: Introduce virDomainBackup APIs'
009/9:[0039] [FC] 'backup: Document nuances between different state capture APIs'

Eric Blake (9):
  Revert "snapshot: Add virDomainSnapshotObjListParse"
  Revert "snapshot: Add virDomainSnapshotObjListFormat"
  snapshot: Various doc tweaks
  backup: Introduce virDomainCheckpointPtr
  backup: Document new XML for checkpoints
  backup: Document new XML for backups
  backup: Introduce virDomainCheckpoint APIs
  backup: Introduce virDomainBackup APIs
  backup: Document nuances between different state capture APIs

 include/libvirt/virterror.h                  |   7 +-
 src/util/virerror.c                          |  12 +-
 include/libvirt/libvirt-domain-checkpoint.h  | 161 ++++
 include/libvirt/libvirt-domain-snapshot.h    |   7 +-
 include/libvirt/libvirt-domain.h             |  59 +-
 include/libvirt/libvirt.h                    |   3 +-
 src/conf/snapshot_conf.h                     |   7 -
 src/conf/virdomainmomentobjlist.h            |   5 +-
 src/conf/virdomainsnapshotobjlist.h          |  13 -
 src/datatypes.h                              |  33 +-
 src/driver-hypervisor.h                      |  87 ++-
 src/qemu/qemu_blockjob.h                     |   1 +
 docs/Makefile.am                             |   3 +
 docs/apibuild.py                             |   2 +
 docs/docs.html.in                            |  10 +-
 docs/domainstatecapture.html.in              | 313 ++++++++
 docs/format.html.in                          |   2 +
 docs/formatbackup.html.in                    | 181 +++++
 docs/formatcheckpoint.html.in                | 204 +++++
 docs/formatsnapshot.html.in                  |  82 +-
 docs/index.html.in                           |   4 +-
 docs/schemas/domainbackup.rng                | 219 ++++++
 docs/schemas/domaincheckpoint.rng            |  94 +++
 examples/object-events/event-test.c          |   3 +
 libvirt.spec.in                              |   3 +
 mingw-libvirt.spec.in                        |   6 +
 po/POTFILES                                  |   1 +
 src/Makefile.am                              |   2 +
 src/conf/domain_conf.c                       |   2 +-
 src/conf/snapshot_conf.c                     |   2 +-
 src/conf/virdomainsnapshotobjlist.c          | 166 ----
 src/datatypes.c                              |  63 +-
 src/libvirt-domain-checkpoint.c              | 749 +++++++++++++++++++
 src/libvirt-domain-snapshot.c                | 311 ++++----
 src/libvirt-domain.c                         | 289 ++++++-
 src/libvirt_private.syms                     |   4 +-
 src/libvirt_public.syms                      |  21 +
 tests/Makefile.am                            |   4 +
 tests/domainbackupxml2xmlin/backup-pull.xml  |   9 +
 tests/domainbackupxml2xmlin/backup-push.xml  |   9 +
 tests/domainbackupxml2xmlin/empty.xml        |   1 +
 tests/domainbackupxml2xmlout/backup-pull.xml |   9 +
 tests/domainbackupxml2xmlout/backup-push.xml |   9 +
 tests/domainbackupxml2xmlout/empty.xml       |   7 +
 tests/domaincheckpointxml2xmlin/empty.xml    |   1 +
 tests/domaincheckpointxml2xmlin/sample.xml   |   7 +
 tests/domaincheckpointxml2xmlout/empty.xml   |  10 +
 tests/domaincheckpointxml2xmlout/sample.xml  |  16 +
 tests/virschematest.c                        |   4 +
 tools/virsh-domain.c                         |   8 +-
 50 files changed, 2794 insertions(+), 431 deletions(-)
 create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
 create mode 100644 docs/domainstatecapture.html.in
 create mode 100644 docs/formatbackup.html.in
 create mode 100644 docs/formatcheckpoint.html.in
 create mode 100644 docs/schemas/domainbackup.rng
 create mode 100644 docs/schemas/domaincheckpoint.rng
 create mode 100644 src/libvirt-domain-checkpoint.c
 create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlin/empty.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlout/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
 create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml

-- 
2.20.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v6 0/9] Incremental backups, focus on API
Posted by Peter Krempa 5 years ago
On Tue, Mar 26, 2019 at 01:13:44 -0500, Eric Blake wrote:
> I'm fairly confident that these API are ready to go (that is, I've got
> qemu code in the wings to implement these API for the qemu driver, as
> demonstrated at last KVM forum, and it shouldn't be too hard to add
> support in the test driver to get some 'make check' coverage similar
> to what I recently added for snapshots). I'm hoping the APIs make it
> in for 5.2, even if I'm still dealing with review churn on the later
> parts of my v5 series (there has been a lot of rebasing from earlier
> review comments, so v5 is currently still the most recent version that
> I was able to run demos with, although I hope to post the rest of v6
> soon).

I'm still not persuaded that pushing any API without implementation is a
good idea. (No, test driver implementation does not count). If the
consensus of others is that it's good to go then go ahead, but I want to
voice this concern here.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v6 0/9] Incremental backups, focus on API
Posted by Eric Blake 5 years ago
On 3/26/19 12:41 PM, Peter Krempa wrote:
> On Tue, Mar 26, 2019 at 01:13:44 -0500, Eric Blake wrote:
>> I'm fairly confident that these API are ready to go (that is, I've got
>> qemu code in the wings to implement these API for the qemu driver, as
>> demonstrated at last KVM forum, and it shouldn't be too hard to add
>> support in the test driver to get some 'make check' coverage similar
>> to what I recently added for snapshots). I'm hoping the APIs make it
>> in for 5.2, even if I'm still dealing with review churn on the later
>> parts of my v5 series (there has been a lot of rebasing from earlier
>> review comments, so v5 is currently still the most recent version that
>> I was able to run demos with, although I hope to post the rest of v6
>> soon).
> 
> I'm still not persuaded that pushing any API without implementation is a
> good idea. (No, test driver implementation does not count). If the
> consensus of others is that it's good to go then go ahead, but I want to
> voice this concern here.

I do have the qemu implementation for checkpoints fully tested, as well
as working for pull mode backups (the push mode backups is still demo
quality, but as of v5, I was able to get a push mode backup started even
if the events weren't wired up correctly).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v6 0/9] Incremental backups, focus on API
Posted by Peter Krempa 5 years ago
On Tue, Mar 26, 2019 at 13:14:48 -0500, Eric Blake wrote:
> On 3/26/19 12:41 PM, Peter Krempa wrote:
> > On Tue, Mar 26, 2019 at 01:13:44 -0500, Eric Blake wrote:
> >> I'm fairly confident that these API are ready to go (that is, I've got
> >> qemu code in the wings to implement these API for the qemu driver, as
> >> demonstrated at last KVM forum, and it shouldn't be too hard to add
> >> support in the test driver to get some 'make check' coverage similar
> >> to what I recently added for snapshots). I'm hoping the APIs make it
> >> in for 5.2, even if I'm still dealing with review churn on the later
> >> parts of my v5 series (there has been a lot of rebasing from earlier
> >> review comments, so v5 is currently still the most recent version that
> >> I was able to run demos with, although I hope to post the rest of v6
> >> soon).
> > 
> > I'm still not persuaded that pushing any API without implementation is a
> > good idea. (No, test driver implementation does not count). If the
> > consensus of others is that it's good to go then go ahead, but I want to
> > voice this concern here.
> 
> I do have the qemu implementation for checkpoints fully tested, as well
> as working for pull mode backups (the push mode backups is still demo
> quality, but as of v5, I was able to get a push mode backup started even
> if the events weren't wired up correctly).

That is okay. If there is a working subset with a real driver I don't
object. I just don't see any value in pushing API without a real
implementation from upstream point of view.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list