From nobody Tue May 14 16:05:28 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1620664818; cv=none; d=zohomail.com; s=zohoarc; b=IWV8fd7psTpalyYBrOXeqq6ESap1jn31t67QUfzkyk6iyH7bVSmMPM+WPfpPjrcI88r1szAbz0vsnoX/3ykuiL5aLe/MHZR9Y/QaHIOMNFpR8QUCdGl9yZmvWzpSmHB9QyIzmhte6hZ1MaP/NhIe2R7wfZv/Z97vRBMXBQWNH00= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620664818; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1P+XM0r1M4snSxLlQjEAcvhA6LGh8gyZ6A+ttNhJScE=; b=HwsVYn0Pw9jhXBlVx1+4o2gBNotJyAAqIFb5HwEaWzqR8iYo0WY36YIyxfH7yFW8D874NSnFCr2YZBcnD4qKrsp8HgOrZa5gqB6/4QLUjVm2GwkyhTood9BzCqPLHCXkXXO4zi1V3Hmy8o5S0Q8uAAnwoVCBSJv3tK0h0186cbY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 162066481822695.34880748968192; Mon, 10 May 2021 09:40:18 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-392-eMP_Uoy2ORGA4e_11_6SEA-1; Mon, 10 May 2021 12:40:13 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 40BD1100806C; Mon, 10 May 2021 16:40:08 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E8F1561094; Mon, 10 May 2021 16:40:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B929844A6E; Mon, 10 May 2021 16:40:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 14AGe5EQ025973 for ; Mon, 10 May 2021 12:40:05 -0400 Received: by smtp.corp.redhat.com (Postfix) id 289EA6BC2E; Mon, 10 May 2021 16:40:05 +0000 (UTC) Received: from localhost.localdomain (ovpn-115-55.ams2.redhat.com [10.36.115.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 260FE19EF1; Mon, 10 May 2021 16:40:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620664816; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=1P+XM0r1M4snSxLlQjEAcvhA6LGh8gyZ6A+ttNhJScE=; b=IgGmxlNrNClowM9WO88/JgMiy58VJWYyOKRN4W5GuD3fMabiZWs/3KEZ1PkvAYrgag9FS9 BGWM+vEaS7EP76WctV6WXvCqmW6JcoAIogHX8EQJ1RrIhBmOqktduKo7w9PlKPADlrjrOa KNhIC3OTkQP68J2vk/61u3ucOrMTLv0= X-MC-Unique: eMP_Uoy2ORGA4e_11_6SEA-1 From: Kashyap Chamarthy To: libvir-list@redhat.com Subject: [libvirt PATCH v2 1/2] docs: kbase: Add a doc on live full disk backup Date: Mon, 10 May 2021 18:39:55 +0200 Message-Id: <20210510163956.549925-2-kchamart@redhat.com> In-Reply-To: <20210510163956.549925-1-kchamart@redhat.com> References: <20210510163956.549925-1-kchamart@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) This is a rewrite of: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit Once this commit merges, the above wiki should point to this kbase document. NB: I've intentionally left out the example for pull-based full backups. I'll tackle it once QMP `x-blockdev-reopen` comes out of experimental mode in upstream QEMU. Then pull-based can be described for both full and and differntial backups. Overall, future documents should cover: - full backups using both push- and pull-mode - differential backups using both push- and pull-mode Signed-off-by: Kashyap Chamarthy Reviewed-by: Michal Privoznik --- docs/kbase/live_full_disk_backup.rst | 186 +++++++++++++++++++++++++++ docs/kbase/meson.build | 1 + 2 files changed, 187 insertions(+) create mode 100644 docs/kbase/live_full_disk_backup.rst diff --git a/docs/kbase/live_full_disk_backup.rst b/docs/kbase/live_full_di= sk_backup.rst new file mode 100644 index 0000000000..19f027daac --- /dev/null +++ b/docs/kbase/live_full_disk_backup.rst @@ -0,0 +1,186 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D +Efficient live full disk backup +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +Live full disk backups are preferred in many scenarios, *despite* their +space requirements. The following outlines an efficient method to do +that using libvirt's APIs. This method involves concepts: the notion of +`backing chains `_, +`QCOW2 overlays +`_, +and a special operation called "active block-commit", which allows +live-merging an overlay disk image into its backing file. + +Two kinds of backup: "push" and "pull" +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +QEMU and libvirt combine the concept of `bitmaps +`_ and network +block device (NBD) to allow copying out modified data blocks. There are +two approaches to it: In the first, "push mode", when a user requests +for it, libvirt creates a full backup in an external location (i.e. +libvirt "pushes" the data to the target). + +In the other, "pull mode", libvirt (in coordination with QEMU) exposes +the data that needs to be written out and allows a third-party tool to +copy them out reliably (i.e. the data is being "pulled" from libvirt). +The pull-based backup provides more flexibility by letting an external +tool fetch the modified bits as it sees fit, rather than waiting on +libvirt to push out a full backup to a target location. + +The push- and pull-mode techniques also apply for differential backups +(it also includes incremental backups), which track what has changed +since *any* given backup. + +This document covers only the full backups using the the "push" mode. + + +Full disk backup using "push" mode +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The below approach uses the modern backup API, virDomainBackupBegin(). +This requires libvirt-7.2.0 and QEMU-4.2, or higher versions. + +#. Start the guest:: + + $> virsh start vm1 + Domain 'vm1' started + +#. Enumerate the disk(s) in use:: + + $> virsh domblklist vm1 + Target Source + -------------------------------------- + vda /var/lib/libvirt/images/vm1.qcow2 + +#. Begin the backup:: + + $> virsh backup-begin vm1 + Backup started + +#. Check the job status ("None" means the job has likely completed):: + + $> virsh domjobinfo vm1 + Job type: None + +#. Check the completed job status:: + + $> virsh domjobinfo vm1 --completed + Job type: Completed + Operation: Backup + Time elapsed: 183 ms + File processed: 39.250 MiB + File remaining: 0.000 B + File total: 39.250 MiB + +#. Now we see the copy of the backup:: + + $> ls -lash /var/lib/libvirt/images/vm1.qcow2* + 15M -rw-r--r--. 1 qemu qemu 15M May 10 12:22 vm1.qcow2 + 21M -rw-------. 1 root root 21M May 10 12:23 vm1.qcow2.1620642185 + + +Full backup with older libvirt versions +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This is the alternative in case you cannot use libvirt-7.2.0 and +QEMU-4.2 for some reason. But this assumes you're using *at least* QEMU +2.1 and libvirt-1.2.9. + +This backup approach is slightly more involved, and predates the +virDomainBackupBegin() API: Assuming a guest with a single disk image, +create a temporary live QCOW2 overlay (commonly called as "external +snapshot") to track the live guest writes. Then backup the original +disk image while the guest (live QEMU) keeps writing to the temporary +overlay. Finally, perform the "active block-commit" opertion to +live-merge the temporary overlay disk contents into the original image =E2= =80=94 +i.e. the backing file =E2=80=94 and "pivot" the live QEMU process to point= to +it. + + +#. Start with a guest with a single disk image, ``base.raw``, which is + where the live QEMU is pointing at, and recording the guest writes:: + + base.raw (live QEMU) + +#. List the current block device(s) in use:: + + $ virsh domblklist vm1 + Target Source + ------------------------------------------------ + vda /var/lib/libvirt/images/base.raw + +#. Create the live "external disk snapshot" (or more correctly, "an + overlay"):: + + $ virsh snapshot-create-as --domain vm1 overlay1 \ + --diskspec vda,file=3D/var/lib/libvirt/images/overlay1.qcow2 \ + --disk-only + + The disk image chain looks as follows:: + + base.raw <-- overlay1.qcow2 (live QEMU) + + .. note:: + Above, if you have QEMU guest agent installed in your virtual + machine, use the ``--quiesce`` option with ``virsh + snapshot-create-as [...]`` to ensure you have a consistent disk + state. + + Optionally, you can also supply the ``--no-metadata`` option to + ``virsh snapshot-create-as`` to tell libvirt not track the snapshot + metadata. Otherwise, when you decide to merge snapshot overlays, + you have to explicitly clean the libvirt metadata using ``virsh + snapshot-delete vm1 --metadata [name|--current]``. + +#. Now, take a backup the orignal image, ``base.raw``, to a different + location using ``cp`` or ``rsync``:: + + $ cp /var/lib/libvirt/images/base.raw + /export/backups/copy1_base.raw + + # Or: + + $ rsync -avhW --progress /var/lib/libvirt/images/base.raw \ + /export/backups/copy1_base.raw + +#. Enumerate the current block device(s) in use, again. Notice that the + current disk image in use is the above-created overlay, + ``overlay1.qcow2``:: + + $ virsh domblklist vm1 + Target Source + ------------------------------------------------ + vda vda,file=3D/var/lib/libvirt/images/overlay1.qcow2 + +#. Once the backup of the original image completes, now perform the + "active block-commit" to live-merge the contents of + ``overlay1.qcow2`` into ``base.raw`` *and* pivot the live QEMU back + to the original:: + + $ virsh blockcommit vm1 vda --active --verbose --pivot + +#. After the above operation completes, again list the current block + device(s) in use. And notice that the live QEMU is now writing to + the original base image:: + + $ virsh domblklist vm1 + Target Source + ------------------------------------------------ + vda /var/lib/libvirt/images/base.raw + + +The final updated disk image "chain" will be a single consolidated +disk:: + + [base.raw] (live QEMU) + + +Now you can safely **discard the overlay image**, ``overlay1.qcow2`` =E2= =80=94 +it is no longer valid; and its contents are now fully merged into the +base image. diff --git a/docs/kbase/meson.build b/docs/kbase/meson.build index 7b4e7abbd3..51d4bc7b90 100644 --- a/docs/kbase/meson.build +++ b/docs/kbase/meson.build @@ -6,6 +6,7 @@ docs_kbase_files =3D [ 'index', 'kvm-realtime', 'launch_security_sev', + 'live_full_disk_backup', 'locking-lockd', 'locking', 'locking-sanlock', --=20 2.30.2 From nobody Tue May 14 16:05:28 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1620664867; cv=none; d=zohomail.com; s=zohoarc; b=EYZm0GcqJPRXux5CkaX2OTJtqRXaweX1O4F0RxmCaioka2Dq6COVbadKWCTWHZz5SvBV7Wl5za4OqV/2VsZopTRjBtj7lcC4bDiuEZF4CjqoWjoUHAUQDB62zpth0nN37iVqnkUDh2xHRHMN//t1/5ONF24Fqa2iMhxusHO7EHQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620664867; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nLIZtOhKnSXDRTMacNUDxEcR7GuI4XZ1aPnFa+FPFsA=; b=Z8r7dAVk3we7at/NugcKMFNE1JpfWxJfxWTDKu2aNQ5JHdhncQBvYomcNStYxSiNnv2L6MfLiXvI9xOMGxoyE8pPmkTX6/jSrZ7iilJQyLxv0I0mpCqxTpZ5+2+mBWVwaZzbsSNf/EWkFaMJI0aIU8qRdmv+0eLMm6B60p4CFGk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1620664867064130.6591203086724; Mon, 10 May 2021 09:41:07 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-286-g_cXSycoO1iBeVEWnTI_fw-1; Mon, 10 May 2021 12:40:22 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 435CBA40C4; Mon, 10 May 2021 16:40:17 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 153E51E6; Mon, 10 May 2021 16:40:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id CE2CB1800B8A; Mon, 10 May 2021 16:40:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 14AGeAXO026001 for ; Mon, 10 May 2021 12:40:10 -0400 Received: by smtp.corp.redhat.com (Postfix) id 066CD60BD8; Mon, 10 May 2021 16:40:10 +0000 (UTC) Received: from localhost.localdomain (ovpn-115-55.ams2.redhat.com [10.36.115.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D46519EF1; Mon, 10 May 2021 16:40:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620664865; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=nLIZtOhKnSXDRTMacNUDxEcR7GuI4XZ1aPnFa+FPFsA=; b=OaxnFGqOkT9HiZ7UkYvMZey/fZTjNoXYDIVlThcEUJQ0lP7aLixjqWM3K8Lrzabmc0lsEj aHlIiacURIIuKawwptxjZNxhQ7i7LJ5lU/ypOt7gi4fdthWUCshEaFkgxOZaWyQKKG88qu 0XovmHZe6es0hhHle9BtGBy/1RllMwY= X-MC-Unique: g_cXSycoO1iBeVEWnTI_fw-1 From: Kashyap Chamarthy To: libvir-list@redhat.com Subject: [libvirt PATCH v2 2/2] docs: kbase: Add a doc on merging disk image chains Date: Mon, 10 May 2021 18:39:56 +0200 Message-Id: <20210510163956.549925-3-kchamart@redhat.com> In-Reply-To: <20210510163956.549925-1-kchamart@redhat.com> References: <20210510163956.549925-1-kchamart@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) This is a rewrite of: https://wiki.libvirt.org/page/Live-merge-an-entire-disk-image-chain-inc= luding-current-active-disk Once this commit merges, the above wiki should point to this kbase document. Signed-off-by: Kashyap Chamarthy Reviewed-by: Michal Privoznik --- docs/kbase/merging_disk_image_chains.rst | 200 +++++++++++++++++++++++ docs/kbase/meson.build | 1 + 2 files changed, 201 insertions(+) create mode 100644 docs/kbase/merging_disk_image_chains.rst diff --git a/docs/kbase/merging_disk_image_chains.rst b/docs/kbase/merging_= disk_image_chains.rst new file mode 100644 index 0000000000..7635bd3eec --- /dev/null +++ b/docs/kbase/merging_disk_image_chains.rst @@ -0,0 +1,200 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D +Merging disk image image chains +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Context +=3D=3D=3D=3D=3D=3D=3D + +Sometimes a `disk image chain +`_ can get long and +cumbersome. For the purpose of illustration, consider this smaller disk +image chain:: + + base.raw <-- a.qcow2 <-- b.qcow2 <-- c.qcow2 (live QEMU) + +You may want to reduce the backing chain length, or consolidate *all* +the disk images in the chain into a single image. But you want to +accomplish this *without* incurring guest down time. Here's how to go +about it. + +The same principles used in the `live full disk backup +` document are +used here too. + +Reducing the disk image chain length +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Starting the earlier image chain:: + + base.raw <-- a.qcow2 <-- b.qcow2 <-- c.qcow2 (live QEMU) + +Reduce the length of the chain by two images, with the resulting chain +being:: + + base.raw <-- c.qcow2 (live QEMU) + +Where the ``base.raw`` contains the contents of ``a.qcow2`` and +``b.qcow2``. + +#. Start by listing the current active disk image in use:: + + $ virsh domblklist vm1 + Target Source + ------------------------------------------------ + vda /var/lib/libvirt/images/base.raw + +#. Create the image chain by creating three QCOW2 overlays (or "external + snapshots") on top of each other, while adding some differentiating + content in each image:: + + $ virsh snapshot-create-as --domain vm1 snap1 \ + --diskspec vda,file=3D/var/lib/libvirt/images/a.qcow2 \ + --disk-only --no-metadata + + # + + $ virsh snapshot-create-as --domain vm1 snap2 \ + --diskspec vda,file=3D/var/lib/libvirt/images/b.qcow2 \ + --disk-only --no-metadata + + # + + $ virsh snapshot-create-as --domain vm1 snap3 \ + --diskspec vda,file=3D/var/lib/libvirt/images/c.qcow2 \ + --disk-only --no-metadata + +#. Enumerate the backing file chain (here the ``force-share`` option + simply allows ``qemu-img`` to safely query the disk image status + while it is active) :: + + $ qemu-img info --force-share --backing-chain /var/lib/libvirt/images/= cur.qcow2 + [...] + +#. Again, list the current active disk image in use:: + + $ virsh domblklist vm1 + Target Source + ------------------------------------------------ + vda /var/lib/libvirt/images/c.qcow2 + +#. Perform the "block-commit" by specify the "base" and "top" images, + i.e. merge the contents of ``b.qcow2`` *and* ``a.qcow2`` into + ``base.raw``, *and* :: + + $ virsh blockcommit vm1 vda \ + --base=3D/var/lib/libvirt/images/base.raw + --top=3D/var/lib/libvirt/images/b.qcow2 + +A note on accessing 'base' and 'top' images +------------------------------------------- + +Specifying file paths, as above, make sense when your disks are in the +local filesystem. However, when using more complicated setups such as +network block device (NBD) disks, there are no file paths. Hhere is +where accessing the disk by its index number comes into picture. + +As an example, the below is the ```` element of the guest XML for +with the original disk image chain of four images:: + + ... + + + + + + + + + + + + + + + + + + +
+ + ... + +And we can merge the images ``a.qcow2`` and ``b.qcow2`` into +``base.qcow2`` using the index numbers 1 (for ``base.qcow2``) and :: + + $> virsh blockcommit vm1 vda --base=3D'vda[1]' --top=3D'vda[3]' + +Note that the shell quoting is necessary here, since unquoted ``[1]`` +would do an unexpected shell "globbing" (i.e. file name expansion) if +you have a file '1' in the current directory + +Accessing the disk images via their index numbers is more useful when +you're using blockCommit() API programmatically. + + +Consolidating an entire disk image chain into a single image +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Again, starting the original image chain:: + + base.raw <-- a.qcow2 <-- b.qcow2 <-- c.qcow2 (live QEMU) + +Reduce the length of the chain by two images, with the resulting chain +being:: + + base.raw (live QEMU) + +Where the ``base.raw`` contains the contents of ``a.qcow2``, ``b.qcow2`` +and ``c.qcow2``; *and* the live QEMU is piovoted to point to the +``base.raw``. + + +#. Use the same procedure discussed earlier to create the disk image + chain. + + +#. Now perform the "active block-commit" operation:: + + $ virsh blockcommit vm1 vda --verbose --pivot --active + Block Commit: [100 %] + Successfully pivoted + + Notes: + + - ``--active``: It performs a two-stage operation: first, the contents + from top images (``a.qcow2``, ``b.qcow2``, and ``c.qcow2``) are + committed into the base image; and in the second stage, the the + "block-commit" operation remains awake to synchronize any further + changes from top images into base. Here the user can take two + actions: cancel the job, or pivot the job, i.e. adjust the base + image as the current active image. + + - ``--pivot``: Once data is committed from sn1, sn2 and current into + base, it pivots the live QEMU to use base as the active image. + + - ``--verbose``: It shows the progress of block operation. + + +#. Again, check the current active block device in use:: + + $ virsh domblklist vm1 + Target Source + ------------------------------------------------ + vda /var/lib/libvirt/images/base.raw + + +#. Enumerate the backing file chain:: + + $ qemu-img info --backing-chain /var/lib/libvirt/images/base.raw + [...] + + And the final resulting disk image "chain" will be a single, + consolidated disk image:: + + [base] (live QEMU) + +It is worth bearing in mind that once the above pivot completes, *all* +three overlay files =E2=80=94 ``a.qcow2``, ``b.qcow2``, and ``c.qcow2`` = =E2=80=94 are no +longer valid, and can be safely discarded. diff --git a/docs/kbase/meson.build b/docs/kbase/meson.build index 51d4bc7b90..7631b47018 100644 --- a/docs/kbase/meson.build +++ b/docs/kbase/meson.build @@ -10,6 +10,7 @@ docs_kbase_files =3D [ 'locking-lockd', 'locking', 'locking-sanlock', + 'merging_disk_image_chains', 'migrationinternals', 'qemu-passthrough-security', 'rpm-deployment', --=20 2.30.2