From nobody Fri Mar 29 07:31:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593007691; cv=none; d=zohomail.com; s=zohoarc; b=B6ep5Qe5e9ldWu9OJeH8rL/boYOZxqPTSDLmr362Sut3bh6Fn+Jv5NXj1nBS0WvzVZd6p1FLcj9M5fz18hk+CoBjqhld2bcUYhOZHbZ8gyqYGOiUpRFwikP67sz2OT9lwcKiWfHUXHZb7345Net7MeQd5eA2EJzSnSZdkiayjVk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007691; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bSkiwWioGR32A+/S7V1P7DcYOFEvWBGdUo7v0MXa6Cc=; b=Vc/tRXWAxh0eKHPDLB7/cLmvgEd0ZqadOm4BGZm1ZzLeICa8/9errpzi50ogrMPwU0ytiN4JOsb0Wxm7DuZUM+puewPdNpW5ADYIKo+INY6u0mBMsirJFo0TJEbKPgKV0Z9VL5CFaOk1sOf0oEHiiqKIaBawMFGFp0hFFJv0PMc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1593007691139112.23690805442084; Wed, 24 Jun 2020 07:08:11 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-111-7hfaiARlNBGF0TGQeNv4Zw-1; Wed, 24 Jun 2020 10:08:07 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D5B9519200E7; Wed, 24 Jun 2020 14:08:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DAC061CD; Wed, 24 Jun 2020 14:08:00 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1ADBE1048FD; Wed, 24 Jun 2020 14:08:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05OE7wGP017404 for ; Wed, 24 Jun 2020 10:07:58 -0400 Received: by smtp.corp.redhat.com (Postfix) id BD4B47FD07; Wed, 24 Jun 2020 14:07:58 +0000 (UTC) Received: from speedmetal.redhat.com (unknown [10.40.208.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C77E5BAEF for ; Wed, 24 Jun 2020 14:07:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593007689; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=bSkiwWioGR32A+/S7V1P7DcYOFEvWBGdUo7v0MXa6Cc=; b=fMhgedM9+ZEnaOFyZPI62wZkoqqqegOYtWB6zAqpNpgNJeoYnm1DJ8eo5ih95eqKWtcfDi q/+OYHiDtRsulDmeiObYzCWC8J9pWovikK8UCxgFJ9Ji/OuODTRqWFxGiNHQbHqxQYwvZX +Do2bZO0Ko0hUhN+tZy2UtReLb7Nukc= X-MC-Unique: 7hfaiARlNBGF0TGQeNv4Zw-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 1/6] kbase: incrementalbackupinternals: Add snapshot terminology Date: Wed, 24 Jun 2020 16:07:48 +0200 Message-Id: <3fe1a30faa04f8989ed865ec9d9cc3d14d5c28ed.1593007594.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Make it obvious what's meant by 'overlay' and 'backing image' for sake of extension of the document. Signed-off-by: Peter Krempa Reviewed-by: Eric Blake --- docs/kbase/incrementalbackupinternals.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/kbase/incrementalbackupinternals.rst b/docs/kbase/increme= ntalbackupinternals.rst index 0c4b4f7486..eada0d2935 100644 --- a/docs/kbase/incrementalbackupinternals.rst +++ b/docs/kbase/incrementalbackupinternals.rst @@ -94,6 +94,21 @@ so, even if we obviously can't guarantee that. Integration with external snapshots =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +External snapshot terminology +----------------------------- + +External snapshots on a disk level consist of layered chains of disk image= s. An +image in the chain can have a ``backing image`` placed below. Any chunk in= the +current image which was not written explicitly is transparent and if read = the +data from the backing image is passed through. An image placed on top of t= he +current image is called ``overlay``. + +The bottommost backing image at the end of the chain is also usually descr= ibed +as ``base image``. + +The topmost overlay is the image which is being written to by the VM and i= s also +described as the ``active`` layer or image. + Handling of bitmaps ------------------- --=20 2.26.2 From nobody Fri Mar 29 07:31:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593007691; cv=none; d=zohomail.com; s=zohoarc; b=dsb8HnKjz/VvNduQ2L8So46cQl3TkbQgQgp4mODO7TelaK+HJ+HKR3b3DWTu8Ac8xgHKKcY26rz0BlO0Sogy/IgL4ViJG9SwIkOZiRn2M7irehpeBJ3Qw6GIgWBckdFB0a/C3vE3mOnEYOvr26pKK0Xj0i5wjTWcsZ7ma8kaVCA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007691; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=f1nfJn/RGYHHR86by1pXshTXn0W+lSy3OmXSVg6o/48=; b=KrRoJ3D8BstfTCNrsX7oyIjp0yArBfGzjKl0LJZis1yY0I40YlfBZH4I/u7L96IkK6CMvv9gOUcLMzU7d6s0+uTMsXOVWKAI0ygRDwGcifRIJnosf3PDvXYsCdyH9GL3uJDJzFTkpkflj+WsE7qiUDFQfNOC82V9ikEmQEABQXA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1593007691286166.56111540826907; Wed, 24 Jun 2020 07:08:11 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-379-0Hx4xR10OLqTte0kIsYzHA-1; Wed, 24 Jun 2020 10:08:07 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1A9E3805EE5; Wed, 24 Jun 2020 14:08:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DB63A1002393; Wed, 24 Jun 2020 14:08:00 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id A04FA180954D; Wed, 24 Jun 2020 14:08:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05OE7xVu017412 for ; Wed, 24 Jun 2020 10:07:59 -0400 Received: by smtp.corp.redhat.com (Postfix) id A87FA5BAEF; Wed, 24 Jun 2020 14:07:59 +0000 (UTC) Received: from speedmetal.redhat.com (unknown [10.40.208.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1D4687F49D for ; Wed, 24 Jun 2020 14:07:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593007690; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=f1nfJn/RGYHHR86by1pXshTXn0W+lSy3OmXSVg6o/48=; b=AcW9vjhv9MlU2+D0Xf/DG28trGp5nQA/khyXppFlhzfhRpJHPsvDdeu7DNy6h5YO2uftPe /5KhUGvcfavaCIfQz3+Fp8CLjO3HOJd++U/QJQ+qSVNOudGMGJXE5mzv6FU0tOCMWwQBOh 4ZOkv7fNWm+wNAajhQZZx/qug9tfLdU= X-MC-Unique: 0Hx4xR10OLqTte0kIsYzHA-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 2/6] kbase: incrementalbackupinternals: Clarify language in snapshots section Date: Wed, 24 Jun 2020 16:07:49 +0200 Message-Id: <06897d2190bcf3d272f8293ed435b5a1790f33d9.1593007594.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Emphaisze what needs to happen and also that creating a snapshot doesn't create the appropriate bitmaps. Also mention that granularity is kept. Signed-off-by: Peter Krempa Reviewed-by: Eric Blake --- docs/kbase/incrementalbackupinternals.rst | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/docs/kbase/incrementalbackupinternals.rst b/docs/kbase/increme= ntalbackupinternals.rst index eada0d2935..dbef1e96f3 100644 --- a/docs/kbase/incrementalbackupinternals.rst +++ b/docs/kbase/incrementalbackupinternals.rst @@ -109,17 +109,18 @@ as ``base image``. The topmost overlay is the image which is being written to by the VM and i= s also described as the ``active`` layer or image. -Handling of bitmaps -------------------- - -Creating an external snapshot involves adding a new layer to the backing c= hain -on top of the previous chain. In this step there are no new bitmaps create= d by -default, which would mean that backups become impossible after this step. - -To prevent this from happening we need to re-create the active bitmaps in = the -new top/active layer of the backing chain which allows us to continue trac= king -the changes with same granularity as before and also allows libvirt to sti= tch -together all the corresponding bitmaps to do a backup across snapshots. +Handling of bitmaps during snapshots +------------------------------------ + +Creating an external snapshot involves adding a overlay on top of the prev= iously +active image. Libvirt requires that all ``block-dirty-bitmaps`` which corr= espond +to the checkpoint must be created in the new overlay before any write from= the +guest reaches the overlay to continue tracking which blocks are dirtied. + +Since there are no new bitmaps created by ``qemu`` or ``qemu-img`` by defa= ult +when creating an overlay, we need to re-create the appropriate (see below.) +bitmaps in the new overlay based on the previously active bitmaps in the a= ctive +image. The new bitmaps are created with the same granularity. After taking a snapshot of the ``vda`` disk from the example above placed = into ``vda-2.qcow2`` the following topology will be created: --=20 2.26.2 From nobody Fri Mar 29 07:31:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593007698; cv=none; d=zohomail.com; s=zohoarc; b=e75yk6B1rkNgu47YuyT/IeEz1fLbVRD87yZLfDvu09NrCMgyy/HTSeN/bx8hDvjtrjDi0V5ZTGCTeHgKnInVOS1mankJGih3BXgbbDZpfUUzg2ap4LgXQz+B43e4habmp6wXeQovlzvNq0m0vZL41iGIqvg64e20Wt02oWpBLDI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007698; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VRQeeDEb2bwYeWIxKSunMzph04tYHJAOOOzyylF6Pmc=; b=iwLP9gBO0ewvKKBjdnJotuqnX6pdzSdWbJm6hhEqISmoi3xdEieZ6GZbDQd0L8R0xfuBti3XaLjbkA2rvtOQ3BgniuZstE7lpho6Q+Xuay1Hx5tEMDvw3DztpN7BGXIrN7SFORCU8IIao+g39KgCHj5jS7tXq9WjTErdBaxGqbY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 159300769837020.241558784829067; Wed, 24 Jun 2020 07:08:18 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-354-HfMWiwIEM_-PtvI2pPSvNA-1; Wed, 24 Jun 2020 10:08:14 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B6F9819200CB; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8C83110013D9; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5AF711048FD; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05OE80Ij017423 for ; Wed, 24 Jun 2020 10:08:00 -0400 Received: by smtp.corp.redhat.com (Postfix) id 930807F49D; Wed, 24 Jun 2020 14:08:00 +0000 (UTC) Received: from speedmetal.redhat.com (unknown [10.40.208.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 122AF5BAEF for ; Wed, 24 Jun 2020 14:07:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593007697; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=VRQeeDEb2bwYeWIxKSunMzph04tYHJAOOOzyylF6Pmc=; b=UkiuUV4ALF3n9BDrjGBI6t8Qf2ESZPQzlV0qzBEwuPiiJhMLoOgFvIeAtPUBgtNk1On75p 3ipksOIlIgiYSTrhqctoK1SgeiHjfewXSF5vJAujembqL43YKFEioOJljCi8PUBaGzf7M7 qcRNE9myX589KpMLV0oPrShVHA2gU9I= X-MC-Unique: HfMWiwIEM_-PtvI2pPSvNA-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 3/6] kbase: incrementalbackupinternals: Add secion on bitmap handling in shell Date: Wed, 24 Jun 2020 16:07:50 +0200 Message-Id: <8ce1389bd0de6f5b86e9acb49e808cb8b2d22efa.1593007594.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Add a section that outlines usage of tools to handle bitmaps and introduce terms corresponding to the output of qemu-img to be used in further sections. With this we can simplify the section about checking bitmap health as we don't have to explain the qemu-img output but can refer to the newly defined terms. Signed-off-by: Peter Krempa Reviewed-by: Eric Blake --- docs/kbase/incrementalbackupinternals.rst | 58 ++++++++++++++++------- 1 file changed, 40 insertions(+), 18 deletions(-) diff --git a/docs/kbase/incrementalbackupinternals.rst b/docs/kbase/increme= ntalbackupinternals.rst index dbef1e96f3..adc415e282 100644 --- a/docs/kbase/incrementalbackupinternals.rst +++ b/docs/kbase/incrementalbackupinternals.rst @@ -139,29 +139,17 @@ After taking a snapshot of the ``vda`` disk from the = example above placed into | | | | +----------------+ +----------------+ -Checking bitmap health ----------------------- +Manipulating bitmaps in shell +----------------------------- -QEMU optimizes disk writes by only updating the bitmaps in certain cases. = This -also can cause problems in cases when e.g. QEMU crashes. +**NOTE:** Any of the examples expect that the full image chain isn't used = by any +running VM at the time. -For a chain of corresponding bitmaps in a backing chain to be considered v= alid -and eligible for use with ``virDomainBackupBegin`` it must conform to the -following rules: - -1) Top image must contain the bitmap -2) If any of the backing images in the chain contain the bitmap too, all - contiguous images must have the bitmap (no gaps) -3) all of the above bitmaps must be marked as active - (``auto`` flag in ``qemu-img`` output, ``recording`` in qemu) -4) none of the above bitmaps can be inconsistent - (``in-use`` flag in ``qemu-img`` provided that it's not used on image w= hich - is currently in use by a qemu instance, or ``inconsistent`` in qemu) +``qemu-img info`` command reports information about dirty bitmaps in an im= age: :: - # check that image has bitmaps - $ qemu-img info vda-1.qcow2 + $ qemu-img info -f qcow2 vda-1.qcow2 image: vda-1.qcow2 file format: qcow2 virtual size: 100 MiB (104857600 bytes) @@ -186,6 +174,40 @@ following rules: refcount bits: 16 corrupt: false +The ``flags`` have following meanings: + +``auto`` - **recording** + + The bitmap is automatically activated when the image is opened for wri= nting + and thus it's actively recording writes. + +``in-use`` - **inconsistent** + + The bitmap was not properly saved when the qemu process was shut down = last + time thus didn't conistently record all the changed sectors. + +It's reccomended to use ``--output=3Djson`` parameter to work with a machi= ne +readable output rather than trying to process the human readable output by +scripts. For processing JSON in shell the ``jq`` tool can be used. + +Checking bitmap health +---------------------- + +QEMU optimizes disk writes by only updating the bitmaps in certain cases. = This +also can cause problems in cases when e.g. QEMU crashes. + +For a chain of corresponding bitmaps in a backing chain images to be consi= dered +valid and eligible for use for an incremental backup with +``virDomainBackupBegin`` the bitmaps intended to be used must conform to t= he +following rules: + +1) active/topmost image must contain the bitmap +2) if bitmap with the same name is contained in one of the backing images = it + must be a contiguougs subchain starting from the topmost image which co= ntains + the bitmaps (no gaps) +3) all of the above bitmaps must be marked as **recording** +4) all of the above bitmaps must not be **inconsistent** + (See also the ``qemuBlockBitmapChainIsValid`` helper method in ``src/qemu/qemu_block.c``) --=20 2.26.2 From nobody Fri Mar 29 07:31:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593007753; cv=none; d=zohomail.com; s=zohoarc; b=Ut7+gm/+owGH1b8uXqj7613QH1r6GMuFO9sSGixzwkjFpq9x8CHUPvXC99hk4yXRdMdi9z0XNA3a1e1xRaGNL9e27S4sp1YYx2Z8y0agnTkk7Ej4beYVvQUrRUw/e4wZAteah8TiBlnMJ8qHWkmjjzPUl5Pc2t4E9KkqBiXzvK0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007753; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cpFhDswMMI0AhHIcoB/3WA1uzmyzWbLW7lebSlx2/yw=; b=EZyFtUxGrL29UE9GsY++HrEGdxLTwGz7dbOQ2tZyU6A4LunwXLQrnPrQCcux8cZubTp6BFYgbpe8dTw8IzSOgz15176+e5xY+ue6tnK6wz6pp256vfkvSmxjkeF08PLG+A5r4rM9X7RaWIH3UzyH2uI7BZYMEaiO4vp2349sPMY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1593007753490877.1720196461675; Wed, 24 Jun 2020 07:09:13 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-433--2ACjI_sOXGDvEQGKnRWBg-1; Wed, 24 Jun 2020 10:08:21 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 995268015F8; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 77FE57931F; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 46348104918; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05OE810A017431 for ; Wed, 24 Jun 2020 10:08:01 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7DCFA5BAEF; Wed, 24 Jun 2020 14:08:01 +0000 (UTC) Received: from speedmetal.redhat.com (unknown [10.40.208.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id E770C7FE92 for ; Wed, 24 Jun 2020 14:08:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593007752; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=cpFhDswMMI0AhHIcoB/3WA1uzmyzWbLW7lebSlx2/yw=; b=HWJRTCGZSbU/miVDfTDMHRDJPNP4MMTxsr7Gq9w5nwjNT1YAsCM+TDjPB11tgabaOPEPyv /4deES7rpaHa4w3iGDk3cT2pA7vN/2DgrDB4gFQLS2hBYn2Aep1+ZONDiKsWfMm4mDhXDJ 2AC3rBSOIv7b0j1RSPVbtjE8AxN9ht8= X-MC-Unique: -2ACjI_sOXGDvEQGKnRWBg-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 4/6] kbase: incrementalbackupinternals: Add section on 'qemu-img bitmap' use Date: Wed, 24 Jun 2020 16:07:51 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Define what users should look for when wanting to manipulate bitmaps themselves. Later on a patch will turn the bash algorithms into pseudocode for simplicity. Signed-off-by: Peter Krempa Reviewed-by: Eric Blake --- docs/kbase/incrementalbackupinternals.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/kbase/incrementalbackupinternals.rst b/docs/kbase/increme= ntalbackupinternals.rst index adc415e282..c8a0a66baa 100644 --- a/docs/kbase/incrementalbackupinternals.rst +++ b/docs/kbase/incrementalbackupinternals.rst @@ -190,6 +190,20 @@ It's reccomended to use ``--output=3Djson`` parameter = to work with a machine readable output rather than trying to process the human readable output by scripts. For processing JSON in shell the ``jq`` tool can be used. +``qemu-img bitmap`` command allows modification of block-dirty-bitmaps of = an +image. It supports the following operations relevant to this document (see= man +page for full list of operations): + +``--add NAME`` + Creates a new bitmap named ``NAME``. Optionally ``-g`` can be used to + specify granularity. + +``--remove NAME`` + Deletes bitmap ``NAME``. + +``--merge SRCBITMAP -b SRCFILE -F SRCFILEFMT DSTBITMAP`` + Merges bitmap ``SRCBITMAP`` from ``SRCFILE`` into ``DSTBITMAP``. + Checking bitmap health ---------------------- --=20 2.26.2 From nobody Fri Mar 29 07:31:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593007699; cv=none; d=zohomail.com; s=zohoarc; b=a8BbW2tW7j5ssAfZ/4P4ZK4VzsChOb6oSpcbhxuN0eSAbESrGZEjkrrKO8cX6mNd9OnjryZrTt7oSR+gEtJVzM6XwC1n+8lMk+td8iJdLy5y75DgYbtzROQvLpn5XMq5tymZTWrwAtm19Iw07w3804eVULH/SLXfYVie9uIculQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007699; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=E0/I1q5jLn88XDbT598ut2F0yO32gisV+8x8fQdyKDo=; b=JXPl3FqxyklXCgNwLI0Vso+iEU6+xaNpG1VV659/DcnlOxGJIkWT8Y+k/woY75OMtqmrmz8DG6R91KH9YmOEEROrdlOYznNQLVsLjEPjduY04BTxMSIbBWxTGJjIRvZt4dYnflknQMJWfgKKBiR+f1GnpQBizfbiHOcEbhM5fC0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1593007699628959.1183955361166; Wed, 24 Jun 2020 07:08:19 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-318-TeFxZnbANYSQV8j70fjE9A-1; Wed, 24 Jun 2020 10:08:16 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BE3B4EC1AA; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 772B22B47E; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 43E001800433; Wed, 24 Jun 2020 14:08:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05OE82s7017444 for ; Wed, 24 Jun 2020 10:08:02 -0400 Received: by smtp.corp.redhat.com (Postfix) id 6913A7FE9D; Wed, 24 Jun 2020 14:08:02 +0000 (UTC) Received: from speedmetal.redhat.com (unknown [10.40.208.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC0505BAEF for ; Wed, 24 Jun 2020 14:08:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593007698; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=E0/I1q5jLn88XDbT598ut2F0yO32gisV+8x8fQdyKDo=; b=hQA2en8ED8QVnOUI2kemroZSOV43FnByYUM+6AAWs9Rqvrjgx8q+1jgZJ06pPGOFItYro9 SVL5FFIDjvR+tdBKb1Ag2uFjPFQuNlCr9uCz1RC5+MtlYF648k6ZYQ/yurrTZ7WDsMZFxe nJS0Nq/BhSSlx817/x47D3TllV8D6LI= X-MC-Unique: TeFxZnbANYSQV8j70fjE9A-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 5/6] kbase: incrementalbackupinternals: Replace bash with pseudocode Date: Wed, 24 Jun 2020 16:07:52 +0200 Message-Id: <792d0d5a4a56ff8deff61ee146361410f64bde21.1593007594.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Simplify the docs and reduce maintenance burden by just describing the algorithm by a pseudo-language. Users are encouraged to use libvirt anyways and projects such as oVirt which do some management of storage themselves are unlikely to use bash anyways. Signed-off-by: Peter Krempa Reviewed-by: Eric Blake --- docs/kbase/incrementalbackupinternals.rst | 44 ++++++----------------- 1 file changed, 11 insertions(+), 33 deletions(-) diff --git a/docs/kbase/incrementalbackupinternals.rst b/docs/kbase/increme= ntalbackupinternals.rst index c8a0a66baa..9a96ef6df3 100644 --- a/docs/kbase/incrementalbackupinternals.rst +++ b/docs/kbase/incrementalbackupinternals.rst @@ -228,42 +228,20 @@ following rules: Creating external snapshots manually -------------------------------------- -To create the same topology outside of libvirt (e.g when doing snapshots o= ffline) -a new ``qemu-img`` which supports the ``bitmap`` subcommand is recommended= . The -following algorithm then ensures that the new image after snapshot will wo= rk -with backups (note that ``jq`` is a JSON processor): +To create the same topology outside of libvirt (e.g when doing snapshots +offline) the following pseudo-algorithm ensures that the new image after +snapshot will work with backups. ``OVERLAY`` corresponds to the new overlay +image, ``ACTIVE`` corresponds to the topmost image of the active chain pri= or to +the snapshot. :: - #!/bin/bash + create image OVERLAY on top of ACTIVE - # arguments - SNAP_IMG=3D"vda-2.qcow2" - BACKING_IMG=3D"vda-1.qcow2" + for each BITMAP in ACTIVE: + let GRANULARITY =3D granularity of BITMAP in ACTIVE - # constants - snapshots and bitmaps work only with qcow2 - SNAP_FMT=3D"qcow2" - BACKING_IMG_FMT=3D"qcow2" + if BITMAP isn't RECORDING or is INCONSISTENT: + continue - # create snapshot overlay - qemu-img create -f "$SNAP_FMT" -F "$BACKING_IMG_FMT" -b "$BACKING_IMG" "= $SNAP_IMG" - - BACKING_IMG_INFO=3D$(qemu-img info --output=3Djson -f "$BACKING_IMG_FMT"= "$BACKING_IMG") - BACKING_BITMAPS=3D$(jq '."format-specific".data.bitmaps' <<< "$BACKING_I= MG_INFO") - - if [ "x$BACKING_BITMAPS" =3D "xnull" ]; then - exit 0 - fi - - for BACKING_BITMAP_ in $(jq -c '.[]' <<< "$BACKING_BITMAPS"); do - BITMAP_FLAGS=3D$(jq -c -r '.flags[]' <<< "$BACKING_BITMAP_") - BITMAP_NAME=3D$(jq -r '.name' <<< "$BACKING_BITMAP_") - - if grep 'in-use' <<< "$BITMAP_FLAGS" || - grep -v 'auto' <<< "$BITMAP_FLAGS"; then - continue - fi - - qemu-img bitmap -f "$SNAP_FMT" "$SNAP_IMG" --add "$BITMAP_NAME" - - done + create RECORDING bitmap named BITMAP in OVERLAY with GRANULARITY --=20 2.26.2 From nobody Fri Mar 29 07:31:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593007705; cv=none; d=zohomail.com; s=zohoarc; b=YVQWs/Yow9TiNM22XrLwc9Ht5FKMpSSGNPZTSCQS1K/IE6ejmiQUE51enthngVVxUNlQQCfdKhMyiphmUS1hLJxj3Pl/CaY9/RYwyI3lWFLPLB4rH3Tm6PgGdL38RqA3hJM+KT6XIBUiZhMFeshremADAcjW7BufRHWq4VjaJmQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007705; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mE22v0LrCmD2gGVFHW6uXWwj01sJUyXPs0bNZHznYV0=; b=cVFgf8amoKX7n/P4YvYdj38BZVEvvzyaG90mRNjvqk3bDUV2sWJyZ6W0fhAsbit3lQ9yqnAE3rSFv44zjHJUccD1UaA/3yZ16db7cR1Y6kTFU06KF9Qt++HuJB/l+MSU3Q6ppQyf0NZsTZ/k5gbvUlxv/hld7fClKU8Se+jFJwM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1593007705132290.68181101831885; Wed, 24 Jun 2020 07:08:25 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-11-PEpTJSz7OEiZRQEgwoL3ww-1; Wed, 24 Jun 2020 10:08:20 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E5E67805EF6; Wed, 24 Jun 2020 14:08:09 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C81547931F; Wed, 24 Jun 2020 14:08:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9973587581; Wed, 24 Jun 2020 14:08:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05OE836Y017449 for ; Wed, 24 Jun 2020 10:08:03 -0400 Received: by smtp.corp.redhat.com (Postfix) id 55D415BAEF; Wed, 24 Jun 2020 14:08:03 +0000 (UTC) Received: from speedmetal.redhat.com (unknown [10.40.208.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id BD8C682473 for ; Wed, 24 Jun 2020 14:08:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593007703; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=mE22v0LrCmD2gGVFHW6uXWwj01sJUyXPs0bNZHznYV0=; b=XpV9tOX7aXsW8OuPPDOF1M6CJyLqFxs7iK0ptJ9Us8rK2Xc0i89v2K6/tE//E6ovMq3FTF rhtCcEEu66EEQw70xoYhcuYNncw2hGyYJ7uAe0JyYoI1+0IDzmL/5QsUzfJvoYJPua7EDo uQ9LMSJth4KCfmxqXD1nDQeg0U3u4Ik= X-MC-Unique: PEpTJSz7OEiZRQEgwoL3ww-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 6/6] kbase: incrementalbackupinternals: Describe 'block commit' Date: Wed, 24 Jun 2020 16:07:53 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" oVirt does merge images out of libvirt in some cases. Add docs outlining how it's done from a high level. Signed-off-by: Peter Krempa Reviewed-by: Eric Blake --- docs/kbase/incrementalbackupinternals.rst | 37 +++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/docs/kbase/incrementalbackupinternals.rst b/docs/kbase/increme= ntalbackupinternals.rst index 9a96ef6df3..e50bf52e89 100644 --- a/docs/kbase/incrementalbackupinternals.rst +++ b/docs/kbase/incrementalbackupinternals.rst @@ -245,3 +245,40 @@ the snapshot. continue create RECORDING bitmap named BITMAP in OVERLAY with GRANULARITY + +Commiting external snapshots manually +------------------------------------- + +``block commit`` refers to an operation where data from a subchain of the +backing chain is merged down into the backing image of the subchain removi= ng all +images in the subchain . + +``COMMIT_TOP`` refers to the top of the subchain to merge into ``COMMIT_BA= SE`` +(which stays in the new chain). + +It's strongly advised to use ``virDomainBlockCommit`` API in libvirt direc= tly if +possible. Inactive VMs can be started with ``VIR_DOMAIN_START_PAUSED`` flag +(``virsh start --paused``) to prevent OS from running. + +Otherwise the following pseudo-algorithm can be used: + +Note: A ``valid`` bitmap chain is a set of images containing bitmaps which +conform to the rules about valid bitmaps mentioned above. + +:: + + commit data from COMMIT_TOP to COMMIT_BASE + + let BITMAPS =3D valid bitmap chains in COMMIT_TOP + + for each BITMAP in BITMAPS + let GRANULARITY =3D granularity of BITMAP in ACTIVE + + if BITMAP is not present in COMMIT_BASE: + create RECORDING bitmap named BITMAP in COMMIT_BASE with GRANU= LARITY + + for each IMAGE between COMMIT_TOP(inclusive) and COMMIT_BASE(exclu= sive): + if BITMAP is not present in IMAGE: + break + + merge BITMAP in IMAGE into BITMAP in COMMIT_BASE --=20 2.26.2