Push backup is a backup when hypervisor itself copy backup data to destination
in contrast to pull backup when hypervisor exports backup data thru some
interface and mgmt itself make a copy.

This patch series basically adds creating backup to API/remote/qemu/virsh and
initial backup XML definition.

Just like other blockjobs backup creation is asynchronous. That is creation is
merely a backup start and client should track backup error/completion thru
blockjob events. As backup is done transactionally all individual disk backup
jobs will be aborted by qemu itself in case of error, client need not to do it
manually. Client can cancel the backup by aborting blockjob on any disk being
backed up.

Backup xml desription is similar to snapshot one with some exceptions and is
described in more details in definition patch [1].

I guess good client will track progress for every disk in backup to report
progress and detect hangs so it don't need extra backup complete/error event
that aggregate the overall backup result. However it looks like aborting backup
can be implemented in libvirt as code will be common for all clients. We need
to abort some of not yet completed per disk backups and retry if job to be
aborted is completed meanwhile.

Of coures this series is far from being complete. Incremental backups and
backup persistent metadata is to be implemented. Let's just start work in this
direction.

Nikolay Shirokovskiy (10):
  api: backup: add api to create backup
  api: backup: add driver based implementation
  remote: backup: add create backup implementation
  backup: qemu: monitor: add drive-backup command
  backup: misc: add backup block job type
  conf: backup: add backup xml definition                          [1]
  qemu: backup: add qemuDomainBackupCreateXML implementation
  qemu: backup: check backup destination before start
  qemu: backup: prepare backup destination
  virsh: backup: add backup-create command

Links:

In [1] we came to agreement to create distinct API to support backup operations
and there is a discussion of pull backup series in [2]. The latter is blocked
as some necessary operations are still experimental in qemu.

[1] https://www.redhat.com/archives/libvir-list/2016-March/msg00937.html
[2] https://www.redhat.com/archives/libvir-list/2016-September/msg00192.html

 daemon/remote.c                         |   8 +
 examples/object-events/event-test.c     |   3 +
 include/libvirt/libvirt-domain-backup.h |  59 +++++++
 include/libvirt/libvirt-domain.h        |   3 +
 include/libvirt/libvirt.h               |   1 +
 include/libvirt/virterror.h             |   2 +
 po/POTFILES.in                          |   2 +
 src/Makefile.am                         |   3 +
 src/access/viraccessperm.c              |   3 +-
 src/access/viraccessperm.h              |   6 +
 src/conf/backup_conf.c                  | 299 ++++++++++++++++++++++++++++++++
 src/conf/backup_conf.h                  |  70 ++++++++
 src/conf/domain_conf.c                  |   2 +-
 src/datatypes.c                         |  60 +++++++
 src/datatypes.h                         |  29 ++++
 src/driver-hypervisor.h                 |   5 +
 src/libvirt-domain-backup.c             | 202 +++++++++++++++++++++
 src/libvirt_private.syms                |   9 +
 src/libvirt_public.syms                 |  10 ++
 src/qemu/qemu_conf.h                    |   1 +
 src/qemu/qemu_driver.c                  | 239 ++++++++++++++++++++++++-
 src/qemu/qemu_monitor.c                 |  14 ++
 src/qemu/qemu_monitor.h                 |   5 +
 src/qemu/qemu_monitor_json.c            |  36 ++++
 src/qemu/qemu_monitor_json.h            |   5 +
 src/remote/remote_driver.c              |   7 +
 src/remote/remote_protocol.x            |  23 ++-
 src/rpc/gendispatch.pl                  |  29 +++-
 src/util/virerror.c                     |   6 +
 tools/Makefile.am                       |   1 +
 tools/virsh-backup.c                    | 100 +++++++++++
 tools/virsh-backup.h                    |  29 ++++
 tools/virsh-domain.c                    |   3 +-
 tools/virsh-util.c                      |  11 ++
 tools/virsh-util.h                      |   3 +
 tools/virsh.c                           |   2 +
 tools/virsh.h                           |   1 +
 37 files changed, 1279 insertions(+), 12 deletions(-)
 create mode 100644 include/libvirt/libvirt-domain-backup.h
 create mode 100644 src/conf/backup_conf.c
 create mode 100644 src/conf/backup_conf.h
 create mode 100644 src/libvirt-domain-backup.c
 create mode 100644 tools/virsh-backup.c
 create mode 100644 tools/virsh-backup.h

-- 
1.8.3.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Fri, May 05, 2017 at 09:22:21AM +0300, Nikolay Shirokovskiy wrote:
> Push backup is a backup when hypervisor itself copy backup data to destination
> in contrast to pull backup when hypervisor exports backup data thru some
> interface and mgmt itself make a copy.
> 
> This patch series basically adds creating backup to API/remote/qemu/virsh and
> initial backup XML definition.
> 
> Just like other blockjobs backup creation is asynchronous. That is creation is
> merely a backup start and client should track backup error/completion thru
> blockjob events. As backup is done transactionally all individual disk backup
> jobs will be aborted by qemu itself in case of error, client need not to do it
> manually. Client can cancel the backup by aborting blockjob on any disk being
> backed up.
> 
> Backup xml desription is similar to snapshot one with some exceptions and is
> described in more details in definition patch [1].
> 
> I guess good client will track progress for every disk in backup to report
> progress and detect hangs so it don't need extra backup complete/error event
> that aggregate the overall backup result. However it looks like aborting backup
> can be implemented in libvirt as code will be common for all clients. We need
> to abort some of not yet completed per disk backups and retry if job to be
> aborted is completed meanwhile.
> 
> Of coures this series is far from being complete. Incremental backups and
> backup persistent metadata is to be implemented. Let's just start work in this
> direction.

I've not done a detailed code review, but from a high level design POV, I
think things like API, RPC, etc, look good.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On 10.05.2017 11:11, Daniel P. Berrange wrote:
> On Fri, May 05, 2017 at 09:22:21AM +0300, Nikolay Shirokovskiy wrote:
>> Push backup is a backup when hypervisor itself copy backup data to destination
>> in contrast to pull backup when hypervisor exports backup data thru some
>> interface and mgmt itself make a copy.
>>
>> This patch series basically adds creating backup to API/remote/qemu/virsh and
>> initial backup XML definition.
>>
>> Just like other blockjobs backup creation is asynchronous. That is creation is
>> merely a backup start and client should track backup error/completion thru
>> blockjob events. As backup is done transactionally all individual disk backup
>> jobs will be aborted by qemu itself in case of error, client need not to do it
>> manually. Client can cancel the backup by aborting blockjob on any disk being
>> backed up.
>>
>> Backup xml desription is similar to snapshot one with some exceptions and is
>> described in more details in definition patch [1].
>>
>> I guess good client will track progress for every disk in backup to report
>> progress and detect hangs so it don't need extra backup complete/error event
>> that aggregate the overall backup result. However it looks like aborting backup
>> can be implemented in libvirt as code will be common for all clients. We need
>> to abort some of not yet completed per disk backups and retry if job to be
>> aborted is completed meanwhile.
>>
>> Of coures this series is far from being complete. Incremental backups and
>> backup persistent metadata is to be implemented. Let's just start work in this
>> direction.
> 
> I've not done a detailed code review, but from a high level design POV, I
> think things like API, RPC, etc, look good.
> 
> Regards,
> Daniel
> 

Thanx for review! Now when things look ok I'll add docs.

Nikolay

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list