[PATCH 0/3] Add support for sequential backups

Stefan Reiter posted 3 patches 3 years, 8 months ago
Test docker-quick@centos7 failed
Test docker-mingw@fedora failed
Test checkpatch failed
Test FreeBSD failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200826121359.15450-1-s.reiter@proxmox.com
Maintainers: Eric Blake <eblake@redhat.com>, Max Reitz <mreitz@redhat.com>, John Snow <jsnow@redhat.com>, Markus Armbruster <armbru@redhat.com>, Kevin Wolf <kwolf@redhat.com>
block/backup.c        |  4 ++--
blockdev.c            | 25 ++++++++++++++++++++++---
include/qemu/job.h    | 12 ++++++++++++
job.c                 | 24 ++++++++++++++++++++++++
qapi/transaction.json |  6 +++++-
5 files changed, 65 insertions(+), 6 deletions(-)
[PATCH 0/3] Add support for sequential backups
Posted by Stefan Reiter 3 years, 8 months ago
Backups can already be done for multiple drives in a transaction. However, these
jobs will start all at once, potentially hogging a lot of disk IO all at once.
This problem is made worse, since IO throttling is only available on a per-job
basis.

Add a flag to QMP to support sequential transactions for backups. This way,
every job will be executed one after the other, while still providing the
benefit of transactions (i.e. once one fails, all remaining ones will be
cancelled).

We've internally (in Proxmox VE) been doing sequential backups for a long time
with great success, albeit in a different fashion. This series is the result of
aligning our internal changes closer to upstream, and might be useful for other
people as well.


Stefan Reiter (3):
  job: add sequential transaction support
  blockdev: add sequential mode to *-backup transactions
  backup: initialize bcs bitmap on job create, not start

 block/backup.c        |  4 ++--
 blockdev.c            | 25 ++++++++++++++++++++++---
 include/qemu/job.h    | 12 ++++++++++++
 job.c                 | 24 ++++++++++++++++++++++++
 qapi/transaction.json |  6 +++++-
 5 files changed, 65 insertions(+), 6 deletions(-)

-- 
2.20.1



Re: [PATCH 0/3] Add support for sequential backups
Posted by Stefan Reiter 3 years, 7 months ago
Friendly ping :)

On 8/26/20 2:13 PM, Stefan Reiter wrote:
> Backups can already be done for multiple drives in a transaction. However, these
> jobs will start all at once, potentially hogging a lot of disk IO all at once.
> This problem is made worse, since IO throttling is only available on a per-job
> basis.
> 
> Add a flag to QMP to support sequential transactions for backups. This way,
> every job will be executed one after the other, while still providing the
> benefit of transactions (i.e. once one fails, all remaining ones will be
> cancelled).
> 
> We've internally (in Proxmox VE) been doing sequential backups for a long time
> with great success, albeit in a different fashion. This series is the result of
> aligning our internal changes closer to upstream, and might be useful for other
> people as well.
> 
> 
> Stefan Reiter (3):
>    job: add sequential transaction support
>    blockdev: add sequential mode to *-backup transactions
>    backup: initialize bcs bitmap on job create, not start
> 
>   block/backup.c        |  4 ++--
>   blockdev.c            | 25 ++++++++++++++++++++++---
>   include/qemu/job.h    | 12 ++++++++++++
>   job.c                 | 24 ++++++++++++++++++++++++
>   qapi/transaction.json |  6 +++++-
>   5 files changed, 65 insertions(+), 6 deletions(-)
>