[libvirt] [PATCH v4 00/20] Incremental Backup API additions

Eric Blake posted 20 patches 11 weeks ago
Test syntax-check passed
Failed in applying to current master (apply log)
include/libvirt/virterror.h                   |    7 +-
src/util/virerror.c                           |   12 +-
include/libvirt/libvirt-domain-checkpoint.h   |  176 ++
include/libvirt/libvirt-domain-snapshot.h     |    2 +-
include/libvirt/libvirt-domain.h              |   17 +-
include/libvirt/libvirt.h                     |    3 +-
src/access/viraccessperm.h                    |    8 +-
src/conf/checkpoint_conf.h                    |  215 +++
src/conf/domain_conf.h                        |   16 +-
src/datatypes.h                               |   31 +-
src/driver-hypervisor.h                       |   74 +-
src/qemu/qemu_block.h                         |    3 +
src/qemu/qemu_blockjob.h                      |    1 +
src/qemu/qemu_capabilities.h                  |    2 +
src/qemu/qemu_conf.h                          |    2 +
src/qemu/qemu_domain.h                        |   29 +-
src/qemu/qemu_monitor.h                       |   21 +-
src/qemu/qemu_monitor_json.h                  |   19 +-
tools/virsh-checkpoint.h                      |   29 +
tools/virsh-completer.h                       |    4 +
tools/virsh-util.h                            |    3 +
tools/virsh.h                                 |    1 +
docs/Makefile.am                              |    3 +
docs/apibuild.py                              |    2 +
docs/docs.html.in                             |    9 +-
docs/domainstatecapture.html.in               |  314 ++++
docs/format.html.in                           |    1 +
docs/formatcheckpoint.html.in                 |  291 +++
docs/formatsnapshot.html.in                   |   33 +-
docs/index.html.in                            |    3 +-
docs/schemas/domainbackup.rng                 |  185 ++
docs/schemas/domaincheckpoint.rng             |   94 +
examples/object-events/event-test.c           |    3 +
libvirt.spec.in                               |    3 +
mingw-libvirt.spec.in                         |    6 +
po/POTFILES                                   |    3 +
src/Makefile.am                               |    2 +
src/access/viraccessperm.c                    |    5 +-
src/conf/Makefile.inc.am                      |    2 +
src/conf/checkpoint_conf.c                    | 1533 ++++++++++++++++
src/conf/domain_conf.c                        |   52 +-
src/conf/snapshot_conf.c                      |    4 +-
src/datatypes.c                               |   62 +-
src/libvirt-domain-checkpoint.c               |  925 ++++++++++
src/libvirt-domain-snapshot.c                 |    7 +-
src/libvirt-domain.c                          |    8 +-
src/libvirt_private.syms                      |   30 +
src/libvirt_public.syms                       |   23 +
src/qemu/qemu_block.c                         |   12 +
src/qemu/qemu_capabilities.c                  |    4 +
src/qemu/qemu_conf.c                          |    5 +
src/qemu/qemu_domain.c                        |  217 ++-
src/qemu/qemu_driver.c                        | 1552 +++++++++++++++-
src/qemu/qemu_migration.c                     |    2 +-
src/qemu/qemu_monitor.c                       |   65 +-
src/qemu/qemu_monitor_json.c                  |  197 +-
src/qemu/qemu_process.c                       |    7 +
src/remote/remote_daemon_dispatch.c           |   22 +-
src/remote/remote_driver.c                    |   33 +-
src/remote/remote_protocol.x                  |  238 ++-
src/remote_protocol-structs                   |  129 ++
src/rpc/gendispatch.pl                        |   32 +-
tests/Makefile.am                             |   15 +-
tests/domainbackupxml2xmlin/backup-pull.xml   |    9 +
tests/domainbackupxml2xmlin/backup-push.xml   |    9 +
tests/domainbackupxml2xmlin/empty.xml         |    1 +
tests/domainbackupxml2xmlout/backup-pull.xml  |    9 +
tests/domainbackupxml2xmlout/backup-push.xml  |    9 +
tests/domainbackupxml2xmlout/empty.xml        |    7 +
tests/domaincheckpointxml2xmlin/empty.xml     |    1 +
tests/domaincheckpointxml2xmlin/sample.xml    |    7 +
tests/domaincheckpointxml2xmlout/empty.xml    |   10 +
tests/domaincheckpointxml2xmlout/sample.xml   |   16 +
tests/domaincheckpointxml2xmltest.c           |  231 +++
.../caps_4.0.0.riscv32.xml                    |    2 +
.../caps_4.0.0.riscv64.xml                    |    2 +
tests/qemumonitorjsontest.c                   |    2 +-
tests/virschematest.c                         |    4 +
tools/Makefile.am                             |    3 +-
tools/virsh-checkpoint.c                      | 1578 +++++++++++++++++
tools/virsh-completer.c                       |   52 +-
tools/virsh-domain.c                          |    8 +-
tools/virsh-snapshot.c                        |    2 +-
tools/virsh-util.c                            |   11 +
tools/virsh.c                                 |    2 +
tools/virsh.pod                               |   14 +-
86 files changed, 8674 insertions(+), 123 deletions(-)
create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
create mode 100644 src/conf/checkpoint_conf.h
create mode 100644 tools/virsh-checkpoint.h
create mode 100644 docs/domainstatecapture.html.in
create mode 100644 docs/formatcheckpoint.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 docs/schemas/domaincheckpoint.rng
create mode 100644 src/conf/checkpoint_conf.c
create mode 100644 src/libvirt-domain-checkpoint.c
create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlin/empty.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
create mode 100644 tests/domaincheckpointxml2xmltest.c
create mode 100644 tools/virsh-checkpoint.c

[libvirt] [PATCH v4 00/20] Incremental Backup API additions

Posted by Eric Blake 11 weeks ago
The following is the latest version of my API proposal for
incremental backups, and follows along from the demo I presented
as part of my KVM Forum 2018 talk:
https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat

The patches are also available via https://repo.or.cz/libvirt/ericb.git
at the tag backup-v4.

Integration with external snapshots is still not included in these
patches, although I have been playing more with that locally, and
I still haven't gotten a working demonstration of a push-mode
incremental backup.

At this point, I'm posting mainly because there have been enough
changes in qemu (the backup APIs are now stable! and in qemu.git
in time for 4.0) and libvirt (several releases and code cleanups
in between, such that the v3 no longer applies easily), while I
continue to work locally on more features, addressing the review
comments I got on v3, and remove the 'wip' tag on several of the
later patches.

Among other things, I still haven't determined how we want to
integrate checkpoints with external snapshots; we could either have:

virDomainSnapshotCreateXML("<domainsnapshot>...") # existing
virDomainBackupBegin("<domainbackup>...", "<domaincheckpoint>...") # this series
virDomainSnapshotCheckpointCreateXML("<domainsnapshot>...", "<domaincheckpoint>...") # new

or to make checkpoint creation part of the snapshot and backup XML, as in:

virDomainSnapshotCreateXML("<domainsnapshot><domaincheckpoint>...</domainsnapshot>") # extension of existing
virDomainBackupBegin("<domainbackup><domaincheckpoint>...</domainbackup>") # tweak to this series

I'm also planning to add an API to query which job ids are currently
running (right now, the code only supports one job at a time, and
always calls it job 1, but that is not future-friendly) and an update
to the redefine XML command to make it easier to redefine multiple
checkpoints in one API call.

Since v3:

001/20:[----] [--] 'snapshots: Avoid term 'checkpoint' for full system snapshot'
002/20:[----] [--] 'domain_conf: Expose virDomainStorageNetworkParseHost'
003/20:[----] [--] 'qemu: Allow optional export name during NBD export'
004/20:[----] [--] 'backup: Document nuances between different state capture APIs'
005/20:[0032] [FC] 'backup: Introduce virDomainCheckpointPtr'
006/20:[----] [--] 'backup: Document new XML for backups'
007/20:[0013] [FC] 'backup: Introduce virDomainCheckpoint APIs'
008/20:[0017] [FC] 'backup: Introduce virDomainBackup APIs'
009/20:[----] [--] 'backup: Add new domain:checkpoint access control'
010/20:[0104] [FC] 'backup: Implement backup APIs for remote driver'
011/20:[0034] [FC] 'wip: backup: Parse and output checkpoint XML'
012/20:[0013] [FC] 'wip: backup: Parse and output backup XML'
013/20:[0006] [FC] 'backup: Implement virsh support for checkpoints'
014/20:[----] [--] 'wip: backup: virsh support for backup'
015/20:[0126] [FC] 'backup: Add new qemu monitor interactions'
016/20:[0072] [FC] 'backup: qemu: Implement metadata tracking for checkpoint APIs'
017/20:[0069] [FC] 'wip: backup: Wire up qemu checkpoint commands over QMP'
018/20:[0018] [FC] 'wip: backup: qemu: Implement framework for backup job APIs'
019/20:[0045] [FC] 'backup: Wire up qemu full pull backup commands over QMP'
020/20:[0051] [FC] 'backup: implement qemu incremental pull backup'

Eric Blake (20):
  snapshots: Avoid term 'checkpoint' for full system snapshot
  domain_conf: Expose virDomainStorageNetworkParseHost
  qemu: Allow optional export name during NBD export
  backup: Document nuances between different state capture APIs
  backup: Introduce virDomainCheckpointPtr
  backup: Document new XML for backups
  backup: Introduce virDomainCheckpoint APIs
  backup: Introduce virDomainBackup APIs
  backup: Add new domain:checkpoint access control
  backup: Implement backup APIs for remote driver
  wip: backup: Parse and output checkpoint XML
  wip: backup: Parse and output backup XML
  backup: Implement virsh support for checkpoints
  wip: backup: virsh support for backup
  backup: Add new qemu monitor interactions
  backup: qemu: Implement metadata tracking for checkpoint APIs
  wip: backup: Wire up qemu checkpoint commands over QMP
  wip: backup: qemu: Implement framework for backup job APIs
  backup: Wire up qemu full pull backup commands over QMP
  backup: implement qemu incremental pull backup

 include/libvirt/virterror.h                   |    7 +-
 src/util/virerror.c                           |   12 +-
 include/libvirt/libvirt-domain-checkpoint.h   |  176 ++
 include/libvirt/libvirt-domain-snapshot.h     |    2 +-
 include/libvirt/libvirt-domain.h              |   17 +-
 include/libvirt/libvirt.h                     |    3 +-
 src/access/viraccessperm.h                    |    8 +-
 src/conf/checkpoint_conf.h                    |  215 +++
 src/conf/domain_conf.h                        |   16 +-
 src/datatypes.h                               |   31 +-
 src/driver-hypervisor.h                       |   74 +-
 src/qemu/qemu_block.h                         |    3 +
 src/qemu/qemu_blockjob.h                      |    1 +
 src/qemu/qemu_capabilities.h                  |    2 +
 src/qemu/qemu_conf.h                          |    2 +
 src/qemu/qemu_domain.h                        |   29 +-
 src/qemu/qemu_monitor.h                       |   21 +-
 src/qemu/qemu_monitor_json.h                  |   19 +-
 tools/virsh-checkpoint.h                      |   29 +
 tools/virsh-completer.h                       |    4 +
 tools/virsh-util.h                            |    3 +
 tools/virsh.h                                 |    1 +
 docs/Makefile.am                              |    3 +
 docs/apibuild.py                              |    2 +
 docs/docs.html.in                             |    9 +-
 docs/domainstatecapture.html.in               |  314 ++++
 docs/format.html.in                           |    1 +
 docs/formatcheckpoint.html.in                 |  291 +++
 docs/formatsnapshot.html.in                   |   33 +-
 docs/index.html.in                            |    3 +-
 docs/schemas/domainbackup.rng                 |  185 ++
 docs/schemas/domaincheckpoint.rng             |   94 +
 examples/object-events/event-test.c           |    3 +
 libvirt.spec.in                               |    3 +
 mingw-libvirt.spec.in                         |    6 +
 po/POTFILES                                   |    3 +
 src/Makefile.am                               |    2 +
 src/access/viraccessperm.c                    |    5 +-
 src/conf/Makefile.inc.am                      |    2 +
 src/conf/checkpoint_conf.c                    | 1533 ++++++++++++++++
 src/conf/domain_conf.c                        |   52 +-
 src/conf/snapshot_conf.c                      |    4 +-
 src/datatypes.c                               |   62 +-
 src/libvirt-domain-checkpoint.c               |  925 ++++++++++
 src/libvirt-domain-snapshot.c                 |    7 +-
 src/libvirt-domain.c                          |    8 +-
 src/libvirt_private.syms                      |   30 +
 src/libvirt_public.syms                       |   23 +
 src/qemu/qemu_block.c                         |   12 +
 src/qemu/qemu_capabilities.c                  |    4 +
 src/qemu/qemu_conf.c                          |    5 +
 src/qemu/qemu_domain.c                        |  217 ++-
 src/qemu/qemu_driver.c                        | 1552 +++++++++++++++-
 src/qemu/qemu_migration.c                     |    2 +-
 src/qemu/qemu_monitor.c                       |   65 +-
 src/qemu/qemu_monitor_json.c                  |  197 +-
 src/qemu/qemu_process.c                       |    7 +
 src/remote/remote_daemon_dispatch.c           |   22 +-
 src/remote/remote_driver.c                    |   33 +-
 src/remote/remote_protocol.x                  |  238 ++-
 src/remote_protocol-structs                   |  129 ++
 src/rpc/gendispatch.pl                        |   32 +-
 tests/Makefile.am                             |   15 +-
 tests/domainbackupxml2xmlin/backup-pull.xml   |    9 +
 tests/domainbackupxml2xmlin/backup-push.xml   |    9 +
 tests/domainbackupxml2xmlin/empty.xml         |    1 +
 tests/domainbackupxml2xmlout/backup-pull.xml  |    9 +
 tests/domainbackupxml2xmlout/backup-push.xml  |    9 +
 tests/domainbackupxml2xmlout/empty.xml        |    7 +
 tests/domaincheckpointxml2xmlin/empty.xml     |    1 +
 tests/domaincheckpointxml2xmlin/sample.xml    |    7 +
 tests/domaincheckpointxml2xmlout/empty.xml    |   10 +
 tests/domaincheckpointxml2xmlout/sample.xml   |   16 +
 tests/domaincheckpointxml2xmltest.c           |  231 +++
 .../caps_4.0.0.riscv32.xml                    |    2 +
 .../caps_4.0.0.riscv64.xml                    |    2 +
 tests/qemumonitorjsontest.c                   |    2 +-
 tests/virschematest.c                         |    4 +
 tools/Makefile.am                             |    3 +-
 tools/virsh-checkpoint.c                      | 1578 +++++++++++++++++
 tools/virsh-completer.c                       |   52 +-
 tools/virsh-domain.c                          |    8 +-
 tools/virsh-snapshot.c                        |    2 +-
 tools/virsh-util.c                            |   11 +
 tools/virsh.c                                 |    2 +
 tools/virsh.pod                               |   14 +-
 86 files changed, 8674 insertions(+), 123 deletions(-)
 create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
 create mode 100644 src/conf/checkpoint_conf.h
 create mode 100644 tools/virsh-checkpoint.h
 create mode 100644 docs/domainstatecapture.html.in
 create mode 100644 docs/formatcheckpoint.html.in
 create mode 100644 docs/schemas/domainbackup.rng
 create mode 100644 docs/schemas/domaincheckpoint.rng
 create mode 100644 src/conf/checkpoint_conf.c
 create mode 100644 src/libvirt-domain-checkpoint.c
 create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlin/empty.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlout/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
 create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
 create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
 create mode 100644 tests/domaincheckpointxml2xmltest.c
 create mode 100644 tools/virsh-checkpoint.c

-- 
2.20.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 00/20] Incremental Backup API additions

Posted by Kashyap Chamarthy 9 weeks ago
On Wed, Feb 06, 2019 at 01:17:58PM -0600, Eric Blake wrote:
> The following is the latest version of my API proposal for
> incremental backups, and follows along from the demo I presented
> as part of my KVM Forum 2018 talk:
> https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat
> 
> The patches are also available via https://repo.or.cz/libvirt/ericb.git
> at the tag backup-v4.
> 
> Integration with external snapshots is still not included in these
> patches, although I have been playing more with that locally, and
> I still haven't gotten a working demonstration of a push-mode
> incremental backup.
> 
> At this point, I'm posting mainly because there have been enough
> changes in qemu (the backup APIs are now stable! and in qemu.git
> in time for 4.0) and libvirt (several releases and code cleanups
> in between, such that the v3 no longer applies easily), while I
> continue to work locally on more features, addressing the review
> comments I got on v3, and remove the 'wip' tag on several of the
> later patches.
> 
> Among other things, I still haven't determined how we want to
> integrate checkpoints with external snapshots; we could either have:
> 
> virDomainSnapshotCreateXML("<domainsnapshot>...") # existing
> virDomainBackupBegin("<domainbackup>...", "<domaincheckpoint>...") # this series
> virDomainSnapshotCheckpointCreateXML("<domainsnapshot>...", "<domaincheckpoint>...") # new

A slightly related question: when these new APIs (thanks for working on
them!) are merged, am I right in assuming that they should be able to
"replace" the existing (and provide additional):

    virDomainBlockRebase(); and 
    virDomainBlockCopy()

... _provided_ that an application is adjusted to using libvirt that is
new enough to drive QEMU's '-blockdev', QMP `blockdev-add` et al?

Or is that (the new APIs being backward-compatible with blockRebase()
and blockCopy()) an explicit non-goal?

I'm only this out of curiosity.

> or to make checkpoint creation part of the snapshot and backup XML, as in:
> 
> virDomainSnapshotCreateXML("<domainsnapshot><domaincheckpoint>...</domainsnapshot>") # extension of existing
> virDomainBackupBegin("<domainbackup><domaincheckpoint>...</domainbackup>") # tweak to this series
> 
> I'm also planning to add an API to query which job ids are currently
> running (right now, the code only supports one job at a time, and
> always calls it job 1, but that is not future-friendly) and an update
> to the redefine XML command to make it easier to redefine multiple
> checkpoints in one API call.

[...]


-- 
/kashyap

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 00/20] Incremental Backup API additions

Posted by Peter Krempa 9 weeks ago
On Mon, Feb 18, 2019 at 17:16:41 +0100, Kashyap Chamarthy wrote:
> On Wed, Feb 06, 2019 at 01:17:58PM -0600, Eric Blake wrote:
> > The following is the latest version of my API proposal for
> > incremental backups, and follows along from the demo I presented
> > as part of my KVM Forum 2018 talk:
> > https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat
> > 
> > The patches are also available via https://repo.or.cz/libvirt/ericb.git
> > at the tag backup-v4.
> > 
> > Integration with external snapshots is still not included in these
> > patches, although I have been playing more with that locally, and
> > I still haven't gotten a working demonstration of a push-mode
> > incremental backup.
> > 
> > At this point, I'm posting mainly because there have been enough
> > changes in qemu (the backup APIs are now stable! and in qemu.git
> > in time for 4.0) and libvirt (several releases and code cleanups
> > in between, such that the v3 no longer applies easily), while I
> > continue to work locally on more features, addressing the review
> > comments I got on v3, and remove the 'wip' tag on several of the
> > later patches.
> > 
> > Among other things, I still haven't determined how we want to
> > integrate checkpoints with external snapshots; we could either have:
> > 
> > virDomainSnapshotCreateXML("<domainsnapshot>...") # existing
> > virDomainBackupBegin("<domainbackup>...", "<domaincheckpoint>...") # this series
> > virDomainSnapshotCheckpointCreateXML("<domainsnapshot>...", "<domaincheckpoint>...") # new
> 
> A slightly related question: when these new APIs (thanks for working on
> them!) are merged, am I right in assuming that they should be able to
> "replace" the existing (and provide additional):
> 
>     virDomainBlockRebase(); and 
>     virDomainBlockCopy()
> 
> ... _provided_ that an application is adjusted to using libvirt that is
> new enough to drive QEMU's '-blockdev', QMP `blockdev-add` et al?
> 
> Or is that (the new APIs being backward-compatible with blockRebase()
> and blockCopy()) an explicit non-goal?
> 
> I'm only this out of curiosity.

The checkpoints are really orthogonal to the backing chain/shapshot
managemet.

Checkpoints don't really store any data but rather provide a way to
determine which blocks were changed and thus need backup. Also one point
is that checkpoints don't allow (or I did not notice it in the proposal)
to capture memory state along with the disk state.

In the ideal world of snapshots when deletion/revertion was implemented
we'd never expose the virDomainBlockCommit and virDomainBlockPull APIs
including the multi-use backdoor virDomainBlockRebase() which should
have never existed and users would do equivalent operations using the
snapshot APIs.

virDomainBlockCopy is useful on it's own though but badly combines with
snapshots. This will need some fixing.

With the new checkpoint APIs the situation is even more "fun" as
modification of the backing chain involves in some cases changes to the
bitmaps. Ideally these would do "the right thing (TM)" during snapshot
deletion/reversion. Given that at this time we don't support snapshot
deletion/reversion for external snapshots we can use the excuse that
snapshot management is not implemented so that checkpoints don't need
to be modified.

Since virDomainBlockCommit/virDomainBlockPull are basically a backdoor to do
snapshot merging which is not actually recorded in the snapshot XML the
same damage can happen to the checkpoints.

I'm not sure how we want to deal with that at this point though.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 00/20] Incremental Backup API additions

Posted by Eric Blake 9 weeks ago
On 2/18/19 10:34 AM, Peter Krempa wrote:

>>> Among other things, I still haven't determined how we want to
>>> integrate checkpoints with external snapshots; we could either have:
>>>
>>> virDomainSnapshotCreateXML("<domainsnapshot>...") # existing
>>> virDomainBackupBegin("<domainbackup>...", "<domaincheckpoint>...") # this series
>>> virDomainSnapshotCheckpointCreateXML("<domainsnapshot>...", "<domaincheckpoint>...") # new
>>
>> A slightly related question: when these new APIs (thanks for working on
>> them!) are merged, am I right in assuming that they should be able to
>> "replace" the existing (and provide additional):
>>
>>     virDomainBlockRebase(); and 
>>     virDomainBlockCopy()
>>
>> ... _provided_ that an application is adjusted to using libvirt that is
>> new enough to drive QEMU's '-blockdev', QMP `blockdev-add` et al?
>>
>> Or is that (the new APIs being backward-compatible with blockRebase()
>> and blockCopy()) an explicit non-goal?
>>
>> I'm only this out of curiosity.
> 
> The checkpoints are really orthogonal to the backing chain/shapshot
> managemet.

Indeed.

Think of virDomainBlockRebase() as syntactic sugar (it really is a
wrapper around either virDomainBlockPull() streaming operation, or
around virDomainBlockRebase() for a mirror operation).  Given your
question grouping it with virDomainBlockCopy(), I'm assuming you are
only asking about the latter.

The biggest difference between virDomainBlockCopy() and
virDomainBackupBegin() is point-in-time: with mirroring, you start the
job up front, but you do not have a valid backup image until you cancel
the job; but since both files have the same content (once the job has
hit the sync phase), cancellation gives you a choice of whether to stay
with the old file (the mirror image is the backup) or to pivot to the
new file (the original file is now the backup).  On the other hand, with
virDomainBackupBegin(), you select the point-in-time at the point where
you start the job, and the backup copy is created independently of the
running image so there is no pivoting possible.

virDomainBackupBegin() will work even without checkpoints (in which
case, it is a full image backup); but their main power is that when used
WITH checkpoints, the backup operation can be done with much less effort
than a full copy.  Ideally, we want external snapshots to also be points
where checkpoints can be created; John Snow and I have had some ideas
about what is needed, but our focus is first getting the API working
without worrying about external snapshots, while ensuring that the XML
has enough flexibility to add in those improvements later without
needing more API.

Another difference: virDomainBlockCopy() operates on only one block
device at a time (you have to issue multiple calls if your guest has
multiple disks, although the calls can at least be run in parallel).
But virDomainBackupBegin() operates on the entire domain at once, with
your choice of granularity on which disk(s) to have involved (similar to
how virDomainSnapshotCreateXML() lets you choose which disks to snapshot).

> 
> Checkpoints don't really store any data but rather provide a way to
> determine which blocks were changed and thus need backup. Also one point
> is that checkpoints don't allow (or I did not notice it in the proposal)
> to capture memory state along with the disk state.

Correct - checkpoints ONLY track which portions of the disk have
changed.  In fact, when taking a differential backup, you can really
only request all changes occurring between one point in the past and the
present.  In particular, if you have Check1 and Check2, you can request
the backup of all sectors touched between Check1 and the present, or all
sectors touched between Check2 and the present, but you cannot request
the backup of only the sectors touched between Check1 and Check2 (at
least not through the libvirt API), because those sectors may have
changed again between Check2 and the present, and the bitmaps only
requires which sectors have changed, and NOT what the data was at the
time of Check2's creation.


> In the ideal world of snapshots when deletion/revertion was implemented
> we'd never expose the virDomainBlockCommit and virDomainBlockPull APIs
> including the multi-use backdoor virDomainBlockRebase() which should
> have never existed and users would do equivalent operations using the
> snapshot APIs.
> 
> virDomainBlockCopy is useful on it's own though but badly combines with
> snapshots. This will need some fixing.

Indeed, and that's true regardless of whether the backup API goes in
(although the backup API probably compounds the issue on the number of
corner cases we have to think about; the conservative approach is that
at least in the beginning, you won't be able to run a BlockCopy and a
BackupBegin job at the same time).

> 
> With the new checkpoint APIs the situation is even more "fun" as
> modification of the backing chain involves in some cases changes to the
> bitmaps. Ideally these would do "the right thing (TM)" during snapshot
> deletion/reversion. Given that at this time we don't support snapshot
> deletion/reversion for external snapshots we can use the excuse that
> snapshot management is not implemented so that checkpoints don't need
> to be modified.

Somewhat correct - but we DO have to think about how we plan for the API
to grow in the future when we eventually DO fix snapshot
deletion/reversion.  Hence my question - would we rather have the
creation of a checkpoint at the same time as the creation of an external
snapshot (which we DO know we will want) to occur via the existing API
(by extending the snapshot XML to include the checkpoint XML as a
sub-element), or via a new API (by passing the checkpoint XML as a
second parameter)?  Once we've answered that question, it then
determines what signature we want for virDomainBackupBegin() (either two
separate XML parameters, one for the backup job and one for the
checkpoint creation, as presented in v4 of the series, OR as one single
XML call where the checkpoint XML is a sub-element of the backup XML).

> 
> Since virDomainBlockCommit/virDomainBlockPull are basically a backdoor to do
> snapshot merging which is not actually recorded in the snapshot XML the
> same damage can happen to the checkpoints.
> 
> I'm not sure how we want to deal with that at this point though.
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 00/20] Incremental Backup API additions

Posted by Peter Krempa 9 weeks ago
On Mon, Feb 18, 2019 at 15:50:52 -0600, Eric Blake wrote:
> On 2/18/19 10:34 AM, Peter Krempa wrote:

[...]

> > In the ideal world of snapshots when deletion/revertion was implemented
> > we'd never expose the virDomainBlockCommit and virDomainBlockPull APIs
> > including the multi-use backdoor virDomainBlockRebase() which should
> > have never existed and users would do equivalent operations using the
> > snapshot APIs.
> > 
> > virDomainBlockCopy is useful on it's own though but badly combines with
> > snapshots. This will need some fixing.
> 
> Indeed, and that's true regardless of whether the backup API goes in
> (although the backup API probably compounds the issue on the number of
> corner cases we have to think about; the conservative approach is that
> at least in the beginning, you won't be able to run a BlockCopy and a
> BackupBegin job at the same time).

We currently do this kind of interlocking between blockjobs and
snapshots as well, so that will not be a problem. We don't even allow
two blockjobs on non-conflicting parts of the backing chain while qemu
allows that now.

Allowing that will require some API design considerations/compromises as
our job tracking and manipulation APIs and events identify the job by
the disk.

Given that a guest can detach a disk frontend without qemu interaction
we will somehow need to be able to display jobs which don't have a disk
assigned any more though which will have the same implications
basically.

> > With the new checkpoint APIs the situation is even more "fun" as
> > modification of the backing chain involves in some cases changes to the
> > bitmaps. Ideally these would do "the right thing (TM)" during snapshot
> > deletion/reversion. Given that at this time we don't support snapshot
> > deletion/reversion for external snapshots we can use the excuse that
> > snapshot management is not implemented so that checkpoints don't need
> > to be modified.
> 
> Somewhat correct - but we DO have to think about how we plan for the API
> to grow in the future when we eventually DO fix snapshot
> deletion/reversion.  Hence my question - would we rather have the
> creation of a checkpoint at the same time as the creation of an external
> snapshot (which we DO know we will want) to occur via the existing API
> (by extending the snapshot XML to include the checkpoint XML as a
> sub-element), or via a new API (by passing the checkpoint XML as a
> second parameter)?  Once we've answered that question, it then
> determines what signature we want for virDomainBackupBegin() (either two
> separate XML parameters, one for the backup job and one for the
> checkpoint creation, as presented in v4 of the series, OR as one single
> XML call where the checkpoint XML is a sub-element of the backup XML).

This is an interesting point. Given that the snapshot creates new files
(speaking of external snapshots obviously) you get an implicit
"checkpoint" at that moment as basically all blocks changed since that
moment are recorded in the overlay file. For any other snapshot you get
the same if you apply the overlay file on top of it.

Now the question is whether we need to be able to track that checkpoint
explicitly or whether it needs an explicit bitmap, but I'm not that
familiar with qemu implementation.

If we don't require any explicit bitmap or marking in qemu to be able to
track a checkpoint from a snapshot point we can e.g. allow the backup
API to take an external checkpoint as a reference  for the backup
itself.

Creating both a snapshot and a checkpoint is certainly possible but I
fear that making the snapshot more complex than it is will be an
unexpected can of worms.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 00/20] Incremental Backup API additions

Posted by Eric Blake 9 weeks ago
On 2/19/19 1:30 AM, Peter Krempa wrote:

>> Somewhat correct - but we DO have to think about how we plan for the API
>> to grow in the future when we eventually DO fix snapshot
>> deletion/reversion.  Hence my question - would we rather have the
>> creation of a checkpoint at the same time as the creation of an external
>> snapshot (which we DO know we will want) to occur via the existing API
>> (by extending the snapshot XML to include the checkpoint XML as a
>> sub-element), or via a new API (by passing the checkpoint XML as a
>> second parameter)?  Once we've answered that question, it then
>> determines what signature we want for virDomainBackupBegin() (either two
>> separate XML parameters, one for the backup job and one for the
>> checkpoint creation, as presented in v4 of the series, OR as one single
>> XML call where the checkpoint XML is a sub-element of the backup XML).
> 
> This is an interesting point. Given that the snapshot creates new files
> (speaking of external snapshots obviously) you get an implicit
> "checkpoint" at that moment as basically all blocks changed since that
> moment are recorded in the overlay file. For any other snapshot you get
> the same if you apply the overlay file on top of it.
> 
> Now the question is whether we need to be able to track that checkpoint
> explicitly or whether it needs an explicit bitmap, but I'm not that
> familiar with qemu implementation.

The easiest way to perform a differential backup is to bitwise-or all
the bitmaps from the requested checkpoint to the present (where I've
implemented things that only one bitmap is active at a time, to avoid
qemu having to write to an ever-increasing number of bitmaps as more
checkpoints are created).  But unless I create a bitmap at the same time
an external snapshot is created, there is no easy way to make current
qemu expose that all clusters in the current active layer are dirty, to
OR that with the clusters that were marked dirty prior to creating the
snapshot.  So yes, it would be best if the creation of a snapshot can
ALSO be used as the creation of a bitmap, which in turn implies that
creating an explicit checkpoint associated with the snapshot is the way
to go.  Fortunately, with blockdev-add, it is possible to create the
qcow2 file, then create the bitmap, and only then perform the snapshot
operation, so we don't need any new qemu commands to create an explicit
bitmap, but we DO need to figure out where in the libvirt API to wire in
this extra step when external snapshots and checkpoints are both used in
the same image.

> 
> If we don't require any explicit bitmap or marking in qemu to be able to
> track a checkpoint from a snapshot point we can e.g. allow the backup
> API to take an external checkpoint as a reference  for the backup
> itself.
> 
> Creating both a snapshot and a checkpoint is certainly possible but I
> fear that making the snapshot more complex than it is will be an
> unexpected can of worms.

I'm afraid that when I do integrate bitmaps with external snapshots,
that I'll have to modify the creation of snapshots to create a bitmap,
whether or not I also expose the ability in API to create an explicit
Checkpoint object at the same time.  So as long as we have to manage
bitmaps, we might as well also manage the explicit creation of a
checkpoint for both a snapshot as well as for a virDomainBackupBegin call.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list