From: Eric Blake <eblake@redhat.com>
We need an accurate count of the number of bits set in a bitmap
after a merge. In particular, since the merge operation short-circuits
a merge from an empty source, if you have bitmaps A, B, and C where
B started empty, then merge C into B, and B into A, an inaccurate
count meant that A did not get the contents of C.
In the worst case, we may falsely regard the bitmap as empty when
it has had new writes merged into it.
Fixes: be58721db
CC: qemu-stable@nongnu.org
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
---
v2: based off of Eric's cover letter, now rebased properly
on top of the jsnow/bitmaps staging branch to use the
correct bitmap target (result).
util/hbitmap.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/util/hbitmap.c b/util/hbitmap.c
index d5aca5159f..8d402c59d9 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -759,6 +759,9 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result)
}
}
+ /* Recompute the dirty count */
+ result->count = hb_count_between(result, 0, result->size - 1);
+
return true;
}
--
2.14.4
03.10.2018 02:33, John Snow wrote: > From: Eric Blake <eblake@redhat.com> > > We need an accurate count of the number of bits set in a bitmap > after a merge. In particular, since the merge operation short-circuits > a merge from an empty source, if you have bitmaps A, B, and C where > B started empty, then merge C into B, and B into A, an inaccurate > count meant that A did not get the contents of C. > > In the worst case, we may falsely regard the bitmap as empty when > it has had new writes merged into it. > > Fixes: be58721db > CC: qemu-stable@nongnu.org > Signed-off-by: Eric Blake <eblake@redhat.com> > Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Hm, I rememberd: commit 3260cdfffbf00f33923f5f9f6bef45932d7ac28b Author: Liang Li <liliangleo@didichuxing.com> Date: Wed Feb 7 11:35:49 2018 -0500 hbitmap: fix missing restore count when finish deserialization The .count of HBitmap is forgot to set in function hbitmap_deserialize_finish, let's set it to the right value. - I always forget to update this field.. We definitely should add some generic check on it somewhere, at least in tests. > --- > > v2: based off of Eric's cover letter, now rebased properly > on top of the jsnow/bitmaps staging branch to use the > correct bitmap target (result). > > > util/hbitmap.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/util/hbitmap.c b/util/hbitmap.c > index d5aca5159f..8d402c59d9 100644 > --- a/util/hbitmap.c > +++ b/util/hbitmap.c > @@ -759,6 +759,9 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) > } > } > > + /* Recompute the dirty count */ > + result->count = hb_count_between(result, 0, result->size - 1); > + > return true; > } > -- Best regards, Vladimir
On 10/3/18 9:32 AM, Vladimir Sementsov-Ogievskiy wrote: > 03.10.2018 02:33, John Snow wrote: >> From: Eric Blake <eblake@redhat.com> >> >> We need an accurate count of the number of bits set in a bitmap >> after a merge. In particular, since the merge operation short-circuits >> a merge from an empty source, if you have bitmaps A, B, and C where >> B started empty, then merge C into B, and B into A, an inaccurate >> count meant that A did not get the contents of C. >> >> In the worst case, we may falsely regard the bitmap as empty when >> it has had new writes merged into it. >> >> Fixes: be58721db >> CC: qemu-stable@nongnu.org >> Signed-off-by: Eric Blake <eblake@redhat.com> >> Signed-off-by: John Snow <jsnow@redhat.com> > > Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > > Hm, I rememberd: > commit 3260cdfffbf00f33923f5f9f6bef45932d7ac28b > Author: Liang Li <liliangleo@didichuxing.com> > Date: Wed Feb 7 11:35:49 2018 -0500 > > hbitmap: fix missing restore count when finish deserialization > > The .count of HBitmap is forgot to set in function > hbitmap_deserialize_finish, let's set it to the right value. > > > - I always forget to update this field.. We definitely should add some > generic check on it somewhere, at least in tests. My suggestion (in another thread) was to enhance x-debug-block-dirty-bitmap-sha256 to include 'count' alongside the checksum, to make it easier to write such tests. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On 10/03/2018 10:49 AM, Eric Blake wrote: > On 10/3/18 9:32 AM, Vladimir Sementsov-Ogievskiy wrote: >> 03.10.2018 02:33, John Snow wrote: >>> From: Eric Blake <eblake@redhat.com> >>> >>> We need an accurate count of the number of bits set in a bitmap >>> after a merge. In particular, since the merge operation short-circuits >>> a merge from an empty source, if you have bitmaps A, B, and C where >>> B started empty, then merge C into B, and B into A, an inaccurate >>> count meant that A did not get the contents of C. >>> >>> In the worst case, we may falsely regard the bitmap as empty when >>> it has had new writes merged into it. >>> >>> Fixes: be58721db >>> CC: qemu-stable@nongnu.org >>> Signed-off-by: Eric Blake <eblake@redhat.com> >>> Signed-off-by: John Snow <jsnow@redhat.com> >> >> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> >> >> Hm, I rememberd: >> commit 3260cdfffbf00f33923f5f9f6bef45932d7ac28b >> Author: Liang Li <liliangleo@didichuxing.com> >> Date: Wed Feb 7 11:35:49 2018 -0500 >> >> hbitmap: fix missing restore count when finish deserialization >> >> The .count of HBitmap is forgot to set in function >> hbitmap_deserialize_finish, let's set it to the right value. >> >> >> - I always forget to update this field.. We definitely should add some >> generic check on it somewhere, at least in tests. > > My suggestion (in another thread) was to enhance > x-debug-block-dirty-bitmap-sha256 to include 'count' alongside the > checksum, to make it easier to write such tests. > I suppose this is in preference to query-block? It would make it easier. --js
On 10/3/18 2:57 PM, John Snow wrote: >>> - I always forget to update this field.. We definitely should add some >>> generic check on it somewhere, at least in tests. >> >> My suggestion (in another thread) was to enhance >> x-debug-block-dirty-bitmap-sha256 to include 'count' alongside the >> checksum, to make it easier to write such tests. >> > > I suppose this is in preference to query-block? It would make it easier. Well, query-block remains the canonical way for users to access the count, while x-debug-block-dirty-bitmap-sha256 is for debugging only (basically, for our unit tests, as no one outside of qemu will get much use out of it). But yes, writing a test that uses bitmaps on an unconnected BDS is easier than a test that has to use a named device, since query-block won't list nodes that aren't connected to a named device, and requires a lot more parsing to get at the actual bitmap information in comparison to the debug command directly telling you about a single bitmap. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
03.10.2018 23:01, Eric Blake wrote: > On 10/3/18 2:57 PM, John Snow wrote: > >>>> - I always forget to update this field.. We definitely should >>>> add some >>>> generic check on it somewhere, at least in tests. >>> >>> My suggestion (in another thread) was to enhance >>> x-debug-block-dirty-bitmap-sha256 to include 'count' alongside the >>> checksum, to make it easier to write such tests. >>> >> >> I suppose this is in preference to query-block? It would make it easier. > > Well, query-block remains the canonical way for users to access the > count, while x-debug-block-dirty-bitmap-sha256 is for debugging only > (basically, for our unit tests, as no one outside of qemu will get > much use out of it). But yes, writing a test that uses bitmaps on an > unconnected BDS is easier than a test that has to use a named device, > since query-block won't list nodes that aren't connected to a named > device, and requires a lot more parsing to get at the actual bitmap > information in comparison to the debug command directly telling you > about a single bitmap. > I think the best place is bitmap unit test, which is tests/test-bitmaps, and we don't need additional api -- Best regards, Vladimir
On 10/02/2018 07:33 PM, John Snow wrote: > From: Eric Blake <eblake@redhat.com> > > We need an accurate count of the number of bits set in a bitmap > after a merge. In particular, since the merge operation short-circuits > a merge from an empty source, if you have bitmaps A, B, and C where > B started empty, then merge C into B, and B into A, an inaccurate > count meant that A did not get the contents of C. > > In the worst case, we may falsely regard the bitmap as empty when > it has had new writes merged into it. > > Fixes: be58721db > CC: qemu-stable@nongnu.org > Signed-off-by: Eric Blake <eblake@redhat.com> > Signed-off-by: John Snow <jsnow@redhat.com> > --- > > v2: based off of Eric's cover letter, now rebased properly > on top of the jsnow/bitmaps staging branch to use the > correct bitmap target (result). > > > util/hbitmap.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/util/hbitmap.c b/util/hbitmap.c > index d5aca5159f..8d402c59d9 100644 > --- a/util/hbitmap.c > +++ b/util/hbitmap.c > @@ -759,6 +759,9 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) > } > } > > + /* Recompute the dirty count */ > + result->count = hb_count_between(result, 0, result->size - 1); > + > return true; > } > > Tests on the way, but for now... Thanks, applied to my bitmaps tree: https://github.com/jnsnow/qemu/commits/bitmaps https://github.com/jnsnow/qemu.git --js
© 2016 - 2024 Red Hat, Inc.