[PATCH 00/13] incremental backup: Handle bitmaps during block-commit

Peter Krempa posted 13 patches 4 years, 1 month ago
Failed in applying to current master (apply log)
There is a newer version of this series
src/qemu/qemu_block.c                         | 217 ++++++++++++++++++
src/qemu/qemu_block.h                         |  14 ++
src/qemu/qemu_blockjob.c                      |  94 +++++++-
src/qemu/qemu_blockjob.h                      |   3 +
src/qemu/qemu_domain.c                        | 110 ++++++---
src/qemu/qemu_driver.c                        |  59 ++++-
tests/qemublocktest.c                         | 126 +++++++++-
.../bitmapblockcommit/basic-1-2               | 119 ++++++++++
.../bitmapblockcommit/basic-1-3               | 119 ++++++++++
.../bitmapblockcommit/basic-2-3               |   2 +
.../bitmapblockcommit/snapshots-1-2           |  49 ++++
.../bitmapblockcommit/snapshots-1-3           |  76 ++++++
.../bitmapblockcommit/snapshots-1-4           | 126 ++++++++++
.../bitmapblockcommit/snapshots-1-5           | 130 +++++++++++
.../bitmapblockcommit/snapshots-2-3           |  49 ++++
.../bitmapblockcommit/snapshots-2-4           |  99 ++++++++
.../bitmapblockcommit/snapshots-2-5           | 103 +++++++++
.../bitmapblockcommit/snapshots-3-4           |  72 ++++++
.../bitmapblockcommit/snapshots-3-5           |  76 ++++++
.../bitmapblockcommit/snapshots-4-4           |  11 +
.../bitmapblockcommit/snapshots-4-5           |  33 +++
.../snapshots-synthetic-broken-1-2            |  30 +++
.../snapshots-synthetic-broken-1-3            |  66 ++++++
.../snapshots-synthetic-broken-1-4            |  76 ++++++
.../snapshots-synthetic-broken-1-5            |  76 ++++++
.../snapshots-synthetic-broken-2-3            |  43 ++++
.../snapshots-synthetic-broken-2-4            |  53 +++++
.../snapshots-synthetic-broken-2-5            |  53 +++++
.../snapshots-synthetic-broken-3-4            |  30 +++
.../snapshots-synthetic-broken-3-5            |  30 +++
.../snapshots-synthetic-broken-4-5            |  20 ++
.../blockjob-blockdev-in.xml                  |   4 +
32 files changed, 2135 insertions(+), 33 deletions(-)
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-2
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-2-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-4-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5
[PATCH 00/13] incremental backup: Handle bitmaps during block-commit
Posted by Peter Krempa 4 years, 1 month ago
This implements handling of bitmaps during block-commit so that
incremental backups are not broken after commit.

Note that this applies on top of

https://www.redhat.com/archives/libvir-list/2020-February/msg01125.html

or can be fetched at

 git fetch https://gitlab.com/pipo.sk/libvirt.git reopen-impl

 https://gitlab.com/pipo.sk/libvirt/-/commits/reopen-impl

It uses blockdev-reopen so it's required to use that series. As noted
the branch mentioned above the series contains patches to enable
incremental backup and blockdev-reopen and depends on the following qemu
patches:

https://lists.gnu.org/archive/html/qemu-block/2020-02/msg01423.html
https://lists.gnu.org/archive/html/qemu-block/2020-02/msg01467.html

 git fetch https://gitlab.com/pipo.sk/qemu.git bitmap-reopen

 https://gitlab.com/pipo.sk/qemu/-/commits/bitmap-reopen

Peter Krempa (13):
  qemu: domain: Extract formatting of 'commit' blockjob data into a
    function
  qemu: domain: Extract parsing of 'commit' blockjob data into a
    function
  qemu: blockjob: Store list of bitmaps disabled prior to commit
  qemublocktest: Fix and optimize fake image chain
  qemu: block: Implement helpers for dealing with bitmaps during block
    commit
  qemublocktest: Add tests for handling of bitmaps during block-commit
  qemublocktest: Add more tests for block-commit bitmap handling with
    snapshots
  qemublocktest: Add tests of broken bitmap chain handling during
    block-commit
  qemuBlockJobDiskNewCommit: Propagate 'disabledBitmapsBase'
  qemuDomainBlockCommit: Handle bitmaps on start of commit
  qemuDomainBlockPivot: Handle merging of bitmaps when pivoting an
    active block-commit
  qemu: blockjob: Handle bitmaps after finish of normal block-commit
  qemu: blockjob: Re-enable bitmaps after failed block-copy

 src/qemu/qemu_block.c                         | 217 ++++++++++++++++++
 src/qemu/qemu_block.h                         |  14 ++
 src/qemu/qemu_blockjob.c                      |  94 +++++++-
 src/qemu/qemu_blockjob.h                      |   3 +
 src/qemu/qemu_domain.c                        | 110 ++++++---
 src/qemu/qemu_driver.c                        |  59 ++++-
 tests/qemublocktest.c                         | 126 +++++++++-
 .../bitmapblockcommit/basic-1-2               | 119 ++++++++++
 .../bitmapblockcommit/basic-1-3               | 119 ++++++++++
 .../bitmapblockcommit/basic-2-3               |   2 +
 .../bitmapblockcommit/snapshots-1-2           |  49 ++++
 .../bitmapblockcommit/snapshots-1-3           |  76 ++++++
 .../bitmapblockcommit/snapshots-1-4           | 126 ++++++++++
 .../bitmapblockcommit/snapshots-1-5           | 130 +++++++++++
 .../bitmapblockcommit/snapshots-2-3           |  49 ++++
 .../bitmapblockcommit/snapshots-2-4           |  99 ++++++++
 .../bitmapblockcommit/snapshots-2-5           | 103 +++++++++
 .../bitmapblockcommit/snapshots-3-4           |  72 ++++++
 .../bitmapblockcommit/snapshots-3-5           |  76 ++++++
 .../bitmapblockcommit/snapshots-4-4           |  11 +
 .../bitmapblockcommit/snapshots-4-5           |  33 +++
 .../snapshots-synthetic-broken-1-2            |  30 +++
 .../snapshots-synthetic-broken-1-3            |  66 ++++++
 .../snapshots-synthetic-broken-1-4            |  76 ++++++
 .../snapshots-synthetic-broken-1-5            |  76 ++++++
 .../snapshots-synthetic-broken-2-3            |  43 ++++
 .../snapshots-synthetic-broken-2-4            |  53 +++++
 .../snapshots-synthetic-broken-2-5            |  53 +++++
 .../snapshots-synthetic-broken-3-4            |  30 +++
 .../snapshots-synthetic-broken-3-5            |  30 +++
 .../snapshots-synthetic-broken-4-5            |  20 ++
 .../blockjob-blockdev-in.xml                  |   4 +
 32 files changed, 2135 insertions(+), 33 deletions(-)
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-2
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-3
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-2-3
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-4-4
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5
 create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5

-- 
2.24.1