Series comparison

-[Qemu-devel] [PULL 00/10] Block patches
+[Qemu-devel] [PULL 0/8] Block patches
-The following changes since commit 411ad78115ebeb3411cf4b7622784b93dfabe259:
+The following changes since commit 812b835fb4d23dd108b2f9802158472d50b73579:
-  Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2017-12-15-1' into staging (2017-12-17 15:27:41 +0000)
+  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-05-07' into staging (2019-05-09 16:31:12 +0100)
-are available in the git repository at:
+are available in the Git repository at:
-  git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request
+  https://github.com/stefanha/qemu.git tags/block-pull-request
-for you to fetch changes up to 996922de45299878cdc4c15b72b19edf2bc618a4:
+for you to fetch changes up to e84125761f78919fe63616d9888ea45e72dc956f:
-  block/curl: fix minor memory leaks (2017-12-18 15:44:39 -0500)
+  docs: add Security chapter to the documentation (2019-05-10 10:53:52 +0100)
 ----------------------------------------------------------------
-Blockjob and protocol patches
+Pull request
 ----------------------------------------------------------------
-Jeff Cody (4):
+Andrey Shinkevich (1):
-  block/sheepdog: remove spurious NULL check
+  block/io.c: fix for the allocation failure
   block/sheepdog: code beautification
   block/curl: check error return of curl_global_init()
   block/curl: fix minor memory leaks
-John Snow (1):
+Jules Irenge (3):
-  blockjob: kick jobs on set-speed
+  util/readline: add a space to fix errors by checkpatch tool
   util: readline: replace tab indent by four spaces to fix checkpatch
     errors
   util/readline: Add braces to fix checkpatch errors
-Vladimir Sementsov-Ogievskiy (5):
+Nikita Alekseev (1):
-  hbitmap: add next_zero function
+  block: Add coroutine_fn to bdrv_check_co_entry
   backup: move from done_bitmap to copy_bitmap
   backup: init copy_bitmap from sync_bitmap for incremental
   backup: simplify non-dirty bits progress processing
   backup: use copy_bitmap in incremental backup
- block/backup.c               | 116 +++++++++++++++++-------------
+Paolo Bonzini (1):
- block/curl.c                 |  24 +++++--
+  aio-posix: ensure poll mode is left when aio_notify is called
- block/dirty-bitmap.c         |   5 ++
- block/sheepdog.c             | 166 +++++++++++++++++++++----------------------
+Stefan Hajnoczi (2):
- blockjob.c                   |  30 +++++++-
+  docs: add Secure Coding Practices to developer docs
- include/block/dirty-bitmap.h |   1 +
+  docs: add Security chapter to the documentation
- include/qemu/hbitmap.h       |   8 +++
- tests/test-hbitmap.c         |  61 ++++++++++++++++
+ Makefile                               |   2 +-
- util/hbitmap.c               |  39 ++++++++++
+ block.c                                |   2 +-
-files changed, 309 insertions(+), 141 deletions(-)
+ block/io.c                             |   2 +-
  util/aio-posix.c                       |  12 +-
  util/readline.c                        | 174 ++++++++++++++-----------
  docs/devel/index.rst                   |   1 +
  docs/devel/secure-coding-practices.rst | 106 +++++++++++++++
  docs/security.texi                     | 131 +++++++++++++++++++
  qemu-doc.texi                          |   3 +
 files changed, 347 insertions(+), 86 deletions(-)
  create mode 100644 docs/devel/secure-coding-practices.rst
  create mode 100644 docs/security.texi
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 01/10] hbitmap: add next_zero function
+Deleted patch
-From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-The function searches for next zero bit.
-Also add interface for BdrvDirtyBitmap and unit test.
-Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
-Message-id: 20171012135313.227864-2-vsementsov@virtuozzo.com
-Signed-off-by: Jeff Cody <jcody@redhat.com>
----
- block/dirty-bitmap.c         |  5 ++++
- include/block/dirty-bitmap.h |  1 +
- include/qemu/hbitmap.h       |  8 ++++++
- tests/test-hbitmap.c         | 61 ++++++++++++++++++++++++++++++++++++++++++++
- util/hbitmap.c               | 39 ++++++++++++++++++++++++++++
-files changed, 114 insertions(+)
-diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/dirty-bitmap.c
-+++ b/block/dirty-bitmap.c
-@@ -XXX,XX +XXX,XX @@ char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp)
- {
-     return hbitmap_sha256(bitmap->bitmap, errp);
- }
-+
-+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset)
-+{
-+    return hbitmap_next_zero(bitmap->bitmap, offset);
-+}
-diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/block/dirty-bitmap.h
-+++ b/include/block/dirty-bitmap.h
-@@ -XXX,XX +XXX,XX @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
- BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
-                                         BdrvDirtyBitmap *bitmap);
- char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
-+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start);
- #endif
-diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/qemu/hbitmap.h
-+++ b/include/qemu/hbitmap.h
-@@ -XXX,XX +XXX,XX @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
-  */
- unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
-+/* hbitmap_next_zero:
-+ * @hb: The HBitmap to operate on
-+ * @start: The bit to start from.
-+ *
-+ * Find next not dirty bit.
-+ */
-+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start);
-+
- /* hbitmap_create_meta:
-  * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
-  * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
-diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
-index XXXXXXX..XXXXXXX 100644
---- a/tests/test-hbitmap.c
-+++ b/tests/test-hbitmap.c
-@@ -XXX,XX +XXX,XX @@ static void test_hbitmap_iter_and_reset(TestHBitmapData *data,
-     hbitmap_iter_next(&hbi);
- }
-+static void test_hbitmap_next_zero_check(TestHBitmapData *data, int64_t start)
-+{
-+    int64_t ret1 = hbitmap_next_zero(data->hb, start);
-+    int64_t ret2 = start;
-+    for ( ; ret2 < data->size && hbitmap_get(data->hb, ret2); ret2++) {
-+        ;
-+    }
-+    if (ret2 == data->size) {
-+        ret2 = -1;
-+    }
-+
-+    g_assert_cmpint(ret1, ==, ret2);
-+}
-+
-+static void test_hbitmap_next_zero_do(TestHBitmapData *data, int granularity)
-+{
-+    hbitmap_test_init(data, L3, granularity);
-+    test_hbitmap_next_zero_check(data, 0);
-+    test_hbitmap_next_zero_check(data, L3 - 1);
-+
-+    hbitmap_set(data->hb, L2, 1);
-+    test_hbitmap_next_zero_check(data, 0);
-+    test_hbitmap_next_zero_check(data, L2 - 1);
-+    test_hbitmap_next_zero_check(data, L2);
-+    test_hbitmap_next_zero_check(data, L2 + 1);
-+
-+    hbitmap_set(data->hb, L2 + 5, L1);
-+    test_hbitmap_next_zero_check(data, 0);
-+    test_hbitmap_next_zero_check(data, L2 + 1);
-+    test_hbitmap_next_zero_check(data, L2 + 2);
-+    test_hbitmap_next_zero_check(data, L2 + 5);
-+    test_hbitmap_next_zero_check(data, L2 + L1 - 1);
-+    test_hbitmap_next_zero_check(data, L2 + L1);
-+
-+    hbitmap_set(data->hb, L2 * 2, L3 - L2 * 2);
-+    test_hbitmap_next_zero_check(data, L2 * 2 - L1);
-+    test_hbitmap_next_zero_check(data, L2 * 2 - 2);
-+    test_hbitmap_next_zero_check(data, L2 * 2 - 1);
-+    test_hbitmap_next_zero_check(data, L2 * 2);
-+    test_hbitmap_next_zero_check(data, L3 - 1);
-+
-+    hbitmap_set(data->hb, 0, L3);
-+    test_hbitmap_next_zero_check(data, 0);
-+}
-+
-+static void test_hbitmap_next_zero_0(TestHBitmapData *data, const void *unused)
-+{
-+    test_hbitmap_next_zero_do(data, 0);
-+}
-+
-+static void test_hbitmap_next_zero_4(TestHBitmapData *data, const void *unused)
-+{
-+    test_hbitmap_next_zero_do(data, 4);
-+}
-+
- int main(int argc, char **argv)
- {
-     g_test_init(&argc, &argv, NULL);
-@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
-     hbitmap_test_add("/hbitmap/iter/iter_and_reset",
-                      test_hbitmap_iter_and_reset);
-+
-+    hbitmap_test_add("/hbitmap/next_zero/next_zero_0",
-+                     test_hbitmap_next_zero_0);
-+    hbitmap_test_add("/hbitmap/next_zero/next_zero_4",
-+                     test_hbitmap_next_zero_4);
-+
-     g_test_run();
-     return 0;
-diff --git a/util/hbitmap.c b/util/hbitmap.c
-index XXXXXXX..XXXXXXX 100644
---- a/util/hbitmap.c
-+++ b/util/hbitmap.c
-@@ -XXX,XX +XXX,XX @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first)
-     }
- }
-+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start)
-+{
-+    size_t pos = (start >> hb->granularity) >> BITS_PER_LEVEL;
-+    unsigned long *last_lev = hb->levels[HBITMAP_LEVELS - 1];
-+    uint64_t sz = hb->sizes[HBITMAP_LEVELS - 1];
-+    unsigned long cur = last_lev[pos];
-+    unsigned start_bit_offset =
-+            (start >> hb->granularity) & (BITS_PER_LONG - 1);
-+    int64_t res;
-+
-+    cur |= (1UL << start_bit_offset) - 1;
-+    assert((start >> hb->granularity) < hb->size);
-+
-+    if (cur == (unsigned long)-1) {
-+        do {
-+            pos++;
-+        } while (pos < sz && last_lev[pos] == (unsigned long)-1);
-+
-+        if (pos >= sz) {
-+            return -1;
-+        }
-+
-+        cur = last_lev[pos];
-+    }
-+
-+    res = (pos << BITS_PER_LEVEL) + ctol(cur);
-+    if (res >= hb->size) {
-+        return -1;
-+    }
-+
-+    res = res << hb->granularity;
-+    if (res < start) {
-+        assert(((start - res) >> hb->granularity) == 0);
-+        return start;
-+    }
-+
-+    return res;
-+}
-+
- bool hbitmap_empty(const HBitmap *hb)
- {
-     return hb->count == 0;
---
-.9.5

-[Qemu-devel] [PULL 02/10] backup: move from done_bitmap to copy_bitmap
+Deleted patch
-From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Use HBitmap copy_bitmap instead of done_bitmap. This is needed to
-improve incremental backup in following patches and to unify backup
-loop for full/incremental modes in future patches.
-Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: Jeff Cody <jcody@redhat.com>
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
-Message-id: 20171012135313.227864-3-vsementsov@virtuozzo.com
-Signed-off-by: Jeff Cody <jcody@redhat.com>
----
- block/backup.c | 23 ++++++++++++++---------
-file changed, 14 insertions(+), 9 deletions(-)
-diff --git a/block/backup.c b/block/backup.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/backup.c
-+++ b/block/backup.c
-@@ -XXX,XX +XXX,XX @@ typedef struct BackupBlockJob {
-     BlockdevOnError on_target_error;
-     CoRwlock flush_rwlock;
-     uint64_t bytes_read;
--    unsigned long *done_bitmap;
-     int64_t cluster_size;
-     bool compress;
-     NotifierWithReturn before_write;
-     QLIST_HEAD(, CowRequest) inflight_reqs;
-+
-+    HBitmap *copy_bitmap;
- } BackupBlockJob;
- /* See if in-flight requests overlap and wait for them to complete */
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
-     cow_request_begin(&cow_request, job, start, end);
-     for (; start < end; start += job->cluster_size) {
--        if (test_bit(start / job->cluster_size, job->done_bitmap)) {
-+        if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) {
-             trace_backup_do_cow_skip(job, start);
-             continue; /* already copied */
-         }
-+        hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1);
-         trace_backup_do_cow_process(job, start);
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
-             if (error_is_read) {
-                 *error_is_read = true;
-             }
-+            hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1);
-             goto out;
-         }
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
-             if (error_is_read) {
-                 *error_is_read = false;
-             }
-+            hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1);
-             goto out;
-         }
--        set_bit(start / job->cluster_size, job->done_bitmap);
--
-         /* Publish progress, guest I/O counts as progress too.  Note that the
-          * offset field is an opaque progress value, it is not a disk offset.
-          */
-@@ -XXX,XX +XXX,XX @@ void backup_do_checkpoint(BlockJob *job, Error **errp)
-     }
-     len = DIV_ROUND_UP(backup_job->common.len, backup_job->cluster_size);
--    bitmap_zero(backup_job->done_bitmap, len);
-+    hbitmap_set(backup_job->copy_bitmap, 0, len);
- }
- void backup_wait_for_overlapping_requests(BlockJob *job, int64_t offset,
-@@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque)
-     BackupBlockJob *job = opaque;
-     BackupCompleteData *data;
-     BlockDriverState *bs = blk_bs(job->common.blk);
--    int64_t offset;
-+    int64_t offset, nb_clusters;
-     int ret = 0;
-     QLIST_INIT(&job->inflight_reqs);
-     qemu_co_rwlock_init(&job->flush_rwlock);
--    job->done_bitmap = bitmap_new(DIV_ROUND_UP(job->common.len,
--                                               job->cluster_size));
-+    nb_clusters = DIV_ROUND_UP(job->common.len, job->cluster_size);
-+    job->copy_bitmap = hbitmap_alloc(nb_clusters, 0);
-+    hbitmap_set(job->copy_bitmap, 0, nb_clusters);
-     job->before_write.notify = backup_before_write_notify;
-     bdrv_add_before_write_notifier(bs, &job->before_write);
-     if (job->sync_mode == MIRROR_SYNC_MODE_NONE) {
-+        /* All bits are set in copy_bitmap to allow any cluster to be copied.
-+         * This does not actually require them to be copied. */
-         while (!block_job_is_cancelled(&job->common)) {
-             /* Yield until the job is cancelled.  We just let our before_write
-              * notify callback service CoW requests. */
-@@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque)
-     /* wait until pending backup_do_cow() calls have completed */
-     qemu_co_rwlock_wrlock(&job->flush_rwlock);
-     qemu_co_rwlock_unlock(&job->flush_rwlock);
--    g_free(job->done_bitmap);
-+    hbitmap_free(job->copy_bitmap);
-     data = g_malloc(sizeof(*data));
-     data->ret = ret;
---
-.9.5

-[Qemu-devel] [PULL 10/10] block/curl: fix minor memory leaks
+[Qemu-devel] [PULL 1/8] util/readline: add a space to fix errors by checkpatch tool
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+From: Jules Irenge <jbi.octave@gmail.com>
-Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+util/readline: add a space to fix errors reported by checkpatch.pl tool
 "ERROR: space required before the open parenthesis"
 "ERROR: space required after that ..."
 within "util/redline.c" file
 Signed-off-by: Jules Irenge <jbi.octave@gmail.com>
 Reviewed-by: Thomas Huth <thuth@redhat.com>
 Message-id: 20190401024406.10819-2-jbi.octave@gmail.com
 Message-Id: <20190401024406.10819-2-jbi.octave@gmail.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/curl.c | 6 ++++++
+ util/readline.c | 34 +++++++++++++++++-----------------
-file changed, 6 insertions(+)
+file changed, 17 insertions(+), 17 deletions(-)
-diff --git a/block/curl.c b/block/curl.c
+diff --git a/util/readline.c b/util/readline.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
+--- a/util/readline.c
-+++ b/block/curl.c
++++ b/util/readline.c
-@@ -XXX,XX +XXX,XX @@ out_noclean:
+@@ -XXX,XX +XXX,XX @@ static void readline_update(ReadLineState *rs)
-     qemu_mutex_destroy(&s->mutex);
-     g_free(s->cookie);
+     if (rs->cmd_buf_size != rs->last_cmd_buf_size ||
-     g_free(s->url);
+         memcmp(rs->cmd_buf, rs->last_cmd_buf, rs->cmd_buf_size) != 0) {
-+    g_free(s->username);
+-        for(i = 0; i < rs->last_cmd_buf_index; i++) {
-+    g_free(s->proxyusername);
++        for (i = 0; i < rs->last_cmd_buf_index; i++) {
-+    g_free(s->proxypassword);
+             rs->printf_func(rs->opaque, "\033[D");
-     qemu_opts_del(opts);
+         }
-     return -EINVAL;
+         rs->cmd_buf[rs->cmd_buf_size] = '\0';
- }
+         if (rs->read_password) {
-@@ -XXX,XX +XXX,XX @@ static void curl_close(BlockDriverState *bs)
+             len = strlen(rs->cmd_buf);
+-            for(i = 0; i < len; i++)
-     g_free(s->cookie);
++            for (i = 0; i < len; i++)
-     g_free(s->url);
+                 rs->printf_func(rs->opaque, "*");
-+    g_free(s->username);
+         } else {
-+    g_free(s->proxyusername);
+             rs->printf_func(rs->opaque, "%s", rs->cmd_buf);
-+    g_free(s->proxypassword);
+@@ -XXX,XX +XXX,XX @@ static void readline_update(ReadLineState *rs)
- }
+     if (rs->cmd_buf_index != rs->last_cmd_buf_index) {
+         delta = rs->cmd_buf_index - rs->last_cmd_buf_index;
- static int64_t curl_getlength(BlockDriverState *bs)
+         if (delta > 0) {
 -            for(i = 0;i < delta; i++) {
 +            for (i = 0; i < delta; i++) {
                  rs->printf_func(rs->opaque, "\033[C");
              }
          } else {
              delta = -delta;
 -            for(i = 0;i < delta; i++) {
 +            for (i = 0; i < delta; i++) {
                  rs->printf_func(rs->opaque, "\033[D");
              }
          }
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
          return;
      if (rs->nb_completions == 1) {
          len = strlen(rs->completions[0]);
 -        for(i = rs->completion_index; i < len; i++) {
 +        for (i = rs->completion_index; i < len; i++) {
              readline_insert_char(rs, rs->completions[0][i]);
          }
          /* extra space for next argument. XXX: make it more generic */
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
                completion_comp);
          rs->printf_func(rs->opaque, "\n");
          max_width = 0;
 -        max_prefix = 0;
 -        for(i = 0; i < rs->nb_completions; i++) {
 +        max_prefix = 0;
 +        for (i = 0; i < rs->nb_completions; i++) {
              len = strlen(rs->completions[i]);
 -            if (i==0) {
 +            if (i == 0) {
                  max_prefix = len;
              } else {
                  if (len < max_prefix)
                      max_prefix = len;
 -                for(j=0; j<max_prefix; j++) {
 +                for (j = 0; j < max_prefix; j++) {
                      if (rs->completions[i][j] != rs->completions[0][j])
                          max_prefix = j;
                  }
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
              if (len > max_width)
                  max_width = len;
          }
 -        if (max_prefix > 0)
 -            for(i = rs->completion_index; i < max_prefix; i++) {
 +        if (max_prefix > 0)
 +            for (i = rs->completion_index; i < max_prefix; i++) {
                  readline_insert_char(rs, rs->completions[0][i]);
              }
          max_width += 2;
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
              max_width = 80;
          nb_cols = 80 / max_width;
          j = 0;
 -        for(i = 0; i < rs->nb_completions; i++) {
 +        for (i = 0; i < rs->nb_completions; i++) {
              rs->printf_func(rs->opaque, "%-*s", max_width, rs->completions[i]);
              if (++j == nb_cols || i == (rs->nb_completions - 1)) {
                  rs->printf_func(rs->opaque, "\n");
@@ -XXX,XX +XXX,XX @@ static void readline_clear_screen(ReadLineState *rs)
  /* return true if command handled */
  void readline_handle_byte(ReadLineState *rs, int ch)
  {
 -    switch(rs->esc_state) {
 +    switch (rs->esc_state) {
      case IS_NORM:
 -        switch(ch) {
 +        switch (ch) {
          case 1:
              readline_bol(rs);
              break;
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
          }
          break;
      case IS_CSI:
 -        switch(ch) {
 +        switch (ch) {
      case 'A':
      case 'F':
          readline_up_char(rs);
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
              rs->esc_param = rs->esc_param * 10 + (ch - '0');
              goto the_end;
          case '~':
 -            switch(rs->esc_param) {
 +            switch (rs->esc_param) {
              case 1:
                  readline_bol(rs);
                  break;
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
      the_end:
          break;
      case IS_SS3:
 -        switch(ch) {
 +        switch (ch) {
          case 'F':
              readline_eol(rs);
              break;
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 06/10] blockjob: kick jobs on set-speed
+[Qemu-devel] [PULL 2/8] util: readline: replace tab indent by four spaces to fix checkpatch errors
-From: John Snow <jsnow@redhat.com>
+From: Jules Irenge <jbi.octave@gmail.com>
-If users set an unreasonably low speed (like one byte per second), the
+Replace tab indent by four spaces to fix errors issued by checkpatch.pl tool
-calculated delay may exceed many hours. While we like to punish users
+"ERROR: code indent should never use tabs" within "util/readline.c" file.
 for asking for stupid things, we do also like to allow users to correct
 their wicked ways.
-When a user provides a new speed, kick the job to allow it to recalculate
+Signed-off-by: Jules Irenge <jbi.octave@gmail.com>
-its delay.
+Reviewed-by: Thomas Huth <thuth@redhat.com>
 Message-id: 20190401024406.10819-3-jbi.octave@gmail.com
 Message-Id: <20190401024406.10819-3-jbi.octave@gmail.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
  util/readline.c | 98 ++++++++++++++++++++++++-------------------------
 file changed, 49 insertions(+), 49 deletions(-)
-Signed-off-by: John Snow <jsnow@redhat.com>
+diff --git a/util/readline.c b/util/readline.c
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Message-id: 20171213204611.26276-1-jsnow@redhat.com
 Signed-off-by: Jeff Cody <jcody@redhat.com>
 ---
  blockjob.c | 30 +++++++++++++++++++++++++++++-
 file changed, 29 insertions(+), 1 deletion(-)
 diff --git a/blockjob.c b/blockjob.c
 index XXXXXXX..XXXXXXX 100644
---- a/blockjob.c
+--- a/util/readline.c
-+++ b/blockjob.c
++++ b/util/readline.c
-@@ -XXX,XX +XXX,XX @@ static void __attribute__((__constructor__)) block_job_init(void)
+@@ -XXX,XX +XXX,XX @@ static void readline_up_char(ReadLineState *rs)
+     int idx;
- static void block_job_event_cancelled(BlockJob *job);
- static void block_job_event_completed(BlockJob *job, const char *msg);
+     if (rs->hist_entry == 0)
-+static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job));
+-    return;
++        return;
- /* Transactional group of block jobs */
+     if (rs->hist_entry == -1) {
- struct BlockJobTxn {
+-    /* Find latest entry */
-@@ -XXX,XX +XXX,XX @@ static void block_job_completed_txn_success(BlockJob *job)
+-    for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
 -        if (rs->history[idx] == NULL)
 -        break;
 -    }
 -    rs->hist_entry = idx;
 +        /* Find latest entry */
 +        for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
 +            if (rs->history[idx] == NULL)
 +                break;
 +        }
 +        rs->hist_entry = idx;
      }
      rs->hist_entry--;
      if (rs->hist_entry >= 0) {
 -    pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
 +        pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
                  rs->history[rs->hist_entry]);
 -    rs->cmd_buf_index = rs->cmd_buf_size = strlen(rs->cmd_buf);
 +        rs->cmd_buf_index = rs->cmd_buf_size = strlen(rs->cmd_buf);
      }
  }
-+/* Assumes the block_job_mutex is held */
+@@ -XXX,XX +XXX,XX @@ static void readline_down_char(ReadLineState *rs)
-+static bool block_job_timer_pending(BlockJob *job)
+         return;
-+{
+     if (rs->hist_entry < READLINE_MAX_CMDS - 1 &&
-+    return timer_pending(&job->sleep_timer);
+         rs->history[++rs->hist_entry] != NULL) {
-+}
+-    pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
-+
++        pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
- void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
+                 rs->history[rs->hist_entry]);
- {
+     } else {
-     Error *local_err = NULL;
+         rs->cmd_buf[0] = 0;
-+    int64_t old_speed = job->speed;
+-    rs->hist_entry = -1;
++        rs->hist_entry = -1;
      if (!job->driver->set_speed) {
          error_setg(errp, QERR_UNSUPPORTED);
@@ -XXX,XX +XXX,XX @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
      }
+     rs->cmd_buf_index = rs->cmd_buf_size = strlen(rs->cmd_buf);
-     job->speed = speed;
+ }
-+    if (speed <= old_speed) {
+@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
      int idx;
      if (cmdline[0] == '\0')
 -    return;
 +        return;
-+    }
+     new_entry = NULL;
-+
+     if (rs->hist_entry != -1) {
-+    /* kick only if a timer is pending */
+-    /* We were editing an existing history entry: replace it */
-+    block_job_enter_cond(job, block_job_timer_pending);
+-    hist_entry = rs->history[rs->hist_entry];
- }
+-    idx = rs->hist_entry;
+-    if (strcmp(hist_entry, cmdline) == 0) {
- void block_job_complete(BlockJob *job, Error **errp)
+-        goto same_entry;
-@@ -XXX,XX +XXX,XX @@ void block_job_resume_all(void)
+-    }
 +        /* We were editing an existing history entry: replace it */
 +        hist_entry = rs->history[rs->hist_entry];
 +        idx = rs->hist_entry;
 +        if (strcmp(hist_entry, cmdline) == 0) {
 +            goto same_entry;
 +        }
      }
- }
+     /* Search cmdline in history buffers */
+     for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
--void block_job_enter(BlockJob *job)
+-    hist_entry = rs->history[idx];
-+/*
+-    if (hist_entry == NULL)
-+ * Conditionally enter a block_job pending a call to fn() while
+-        break;
-+ * under the block_job_lock critical section.
+-    if (strcmp(hist_entry, cmdline) == 0) {
-+ */
+-    same_entry:
-+static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job))
+-        new_entry = hist_entry;
- {
+-        /* Put this entry at the end of history */
-     if (!block_job_started(job)) {
+-        memmove(&rs->history[idx], &rs->history[idx + 1],
-         return;
+-            (READLINE_MAX_CMDS - (idx + 1)) * sizeof(char *));
-@@ -XXX,XX +XXX,XX @@ void block_job_enter(BlockJob *job)
+-        rs->history[READLINE_MAX_CMDS - 1] = NULL;
-         return;
+-        for (; idx < READLINE_MAX_CMDS; idx++) {
 -        if (rs->history[idx] == NULL)
 -            break;
 -        }
 -        break;
 -    }
 +        hist_entry = rs->history[idx];
 +        if (hist_entry == NULL)
 +            break;
 +        if (strcmp(hist_entry, cmdline) == 0) {
 +        same_entry:
 +            new_entry = hist_entry;
 +            /* Put this entry at the end of history */
 +            memmove(&rs->history[idx], &rs->history[idx + 1],
 +                    (READLINE_MAX_CMDS - (idx + 1)) * sizeof(char *));
 +            rs->history[READLINE_MAX_CMDS - 1] = NULL;
 +            for (; idx < READLINE_MAX_CMDS; idx++) {
 +                if (rs->history[idx] == NULL)
 +                    break;
 +            }
 +            break;
 +        }
      }
+     if (idx == READLINE_MAX_CMDS) {
-+    if (fn && !fn(job)) {
+-    /* Need to get one free slot */
-+        block_job_unlock();
++        /* Need to get one free slot */
-+        return;
+         g_free(rs->history[0]);
-+    }
+-    memmove(rs->history, &rs->history[1],
-+
+-            (READLINE_MAX_CMDS - 1) * sizeof(char *));
-     assert(!job->deferred_to_main_loop);
+-    rs->history[READLINE_MAX_CMDS - 1] = NULL;
-     timer_del(&job->sleep_timer);
+-    idx = READLINE_MAX_CMDS - 1;
-     job->busy = true;
++        memmove(rs->history, &rs->history[1],
-@@ -XXX,XX +XXX,XX @@ void block_job_enter(BlockJob *job)
++                (READLINE_MAX_CMDS - 1) * sizeof(char *));
-     aio_co_wake(job->co);
++        rs->history[READLINE_MAX_CMDS - 1] = NULL;
- }
++        idx = READLINE_MAX_CMDS - 1;
+     }
-+void block_job_enter(BlockJob *job)
+     if (new_entry == NULL)
-+{
+         new_entry = g_strdup(cmdline);
-+    block_job_enter_cond(job, NULL);
+@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
-+}
+         case 8:
-+
+             readline_backspace(rs);
- bool block_job_is_cancelled(BlockJob *job)
+             break;
- {
+-    case 155:
-     return job->cancelled;
++        case 155:
              rs->esc_state = IS_CSI;
 -        break;
 +            break;
          default:
              if (ch >= 32) {
                  readline_insert_char(rs, ch);
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
          break;
      case IS_CSI:
          switch (ch) {
 -    case 'A':
 -    case 'F':
 -        readline_up_char(rs);
 -        break;
 -    case 'B':
 -    case 'E':
 -        readline_down_char(rs);
 -        break;
 +        case 'A':
 +        case 'F':
 +            readline_up_char(rs);
 +            break;
 +        case 'B':
 +        case 'E':
 +            readline_down_char(rs);
 +            break;
          case 'D':
              readline_backward_char(rs);
              break;
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 05/10] backup: use copy_bitmap in incremental backup
+[Qemu-devel] [PULL 3/8] util/readline: Add braces to fix checkpatch errors
-From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+From: Jules Irenge <jbi.octave@gmail.com>
-We can use copy_bitmap instead of sync_bitmap. copy_bitmap is
+Add braces to fix errors issued by checkpatch.pl tool
-initialized from sync_bitmap and it is more informative: we will not try
+"ERROR: braces {} are necessary for all arms of this statement"
-to process data, that is already in progress (by write notifier).
+Within "util/readline.c" file
 Message-Id: <20190330112142.14082-1-jbi.octave@gmail.com>
-Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: John Snow <jsnow@redhat.com>
 Reviewed-by: Jeff Cody <jcody@redhat.com>
 Message-id: 20171012135313.227864-6-vsementsov@virtuozzo.com
 Signed-off-by: Jeff Cody <jcody@redhat.com>
 ---
- block/backup.c | 55 +++++++++++++++++--------------------------------------
+ util/readline.c | 50 ++++++++++++++++++++++++++++++++-----------------
-file changed, 17 insertions(+), 38 deletions(-)
+file changed, 33 insertions(+), 17 deletions(-)
-diff --git a/block/backup.c b/block/backup.c
+diff --git a/util/readline.c b/util/readline.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/backup.c
+--- a/util/readline.c
-+++ b/block/backup.c
++++ b/util/readline.c
-@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job)
+@@ -XXX,XX +XXX,XX @@ static void readline_update(ReadLineState *rs)
+         rs->cmd_buf[rs->cmd_buf_size] = '\0';
- static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
+         if (rs->read_password) {
              len = strlen(rs->cmd_buf);
 -            for (i = 0; i < len; i++)
 +            for (i = 0; i < len; i++) {
                  rs->printf_func(rs->opaque, "*");
 +            }
          } else {
              rs->printf_func(rs->opaque, "%s", rs->cmd_buf);
          }
@@ -XXX,XX +XXX,XX @@ static void readline_up_char(ReadLineState *rs)
  {
-+    int ret;
+     int idx;
-     bool error_is_read;
--    int ret = 0;
+-    if (rs->hist_entry == 0)
--    int clusters_per_iter;
++    if (rs->hist_entry == 0) {
--    uint32_t granularity;
+         return;
--    int64_t offset;
++    }
-     int64_t cluster;
+     if (rs->hist_entry == -1) {
--    int64_t end;
+         /* Find latest entry */
--    BdrvDirtyBitmapIter *dbi;
+         for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
-+    HBitmapIter hbi;
+-            if (rs->history[idx] == NULL)
++            if (rs->history[idx] == NULL) {
--    granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap);
+                 break;
 -    clusters_per_iter = MAX((granularity / job->cluster_size), 1);
 -    dbi = bdrv_dirty_iter_new(job->sync_bitmap);
 -
 -    /* Find the next dirty sector(s) */
 -    while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
 -        cluster = offset / job->cluster_size;
 -
 -        for (end = cluster + clusters_per_iter; cluster < end; cluster++) {
 -            do {
 -                if (yield_and_check(job)) {
 -                    goto out;
 -                }
 -                ret = backup_do_cow(job, cluster * job->cluster_size,
 -                                    job->cluster_size, &error_is_read,
 -                                    false);
 -                if ((ret < 0) &&
 -                    backup_error_action(job, error_is_read, -ret) ==
 -                    BLOCK_ERROR_ACTION_REPORT) {
 -                    goto out;
 -                }
 -            } while (ret < 0);
 -        }
 -
 -        /* If the bitmap granularity is smaller than the backup granularity,
 -         * we need to advance the iterator pointer to the next cluster. */
 -        if (granularity < job->cluster_size) {
 -            bdrv_set_dirty_iter(dbi, cluster * job->cluster_size);
 -        }
 +    hbitmap_iter_init(&hbi, job->copy_bitmap, 0);
 +    while ((cluster = hbitmap_iter_next(&hbi)) != -1) {
 +        do {
 +            if (yield_and_check(job)) {
 +                return 0;
 +            }
-+            ret = backup_do_cow(job, cluster * job->cluster_size,
+         }
-+                                job->cluster_size, &error_is_read, false);
+         rs->hist_entry = idx;
-+            if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
+     }
-+                           BLOCK_ERROR_ACTION_REPORT)
+@@ -XXX,XX +XXX,XX @@ static void readline_up_char(ReadLineState *rs)
-+            {
-+                return ret;
+ static void readline_down_char(ReadLineState *rs)
  {
 -    if (rs->hist_entry == -1)
 +    if (rs->hist_entry == -1) {
          return;
 +    }
      if (rs->hist_entry < READLINE_MAX_CMDS - 1 &&
          rs->history[++rs->hist_entry] != NULL) {
          pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
      char *hist_entry, *new_entry;
      int idx;
 -    if (cmdline[0] == '\0')
 +    if (cmdline[0] == '\0') {
          return;
 +    }
      new_entry = NULL;
      if (rs->hist_entry != -1) {
          /* We were editing an existing history entry: replace it */
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
      /* Search cmdline in history buffers */
      for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
          hist_entry = rs->history[idx];
 -        if (hist_entry == NULL)
 +        if (hist_entry == NULL) {
              break;
 +        }
          if (strcmp(hist_entry, cmdline) == 0) {
          same_entry:
              new_entry = hist_entry;
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
                      (READLINE_MAX_CMDS - (idx + 1)) * sizeof(char *));
              rs->history[READLINE_MAX_CMDS - 1] = NULL;
              for (; idx < READLINE_MAX_CMDS; idx++) {
 -                if (rs->history[idx] == NULL)
 +                if (rs->history[idx] == NULL) {
                      break;
 +                }
              }
              break;
          }
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
          rs->history[READLINE_MAX_CMDS - 1] = NULL;
          idx = READLINE_MAX_CMDS - 1;
      }
 -    if (new_entry == NULL)
 +    if (new_entry == NULL) {
          new_entry = g_strdup(cmdline);
 +    }
      rs->history[idx] = new_entry;
      rs->hist_entry = -1;
  }
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
      g_free(cmdline);
      /* no completion found */
 -    if (rs->nb_completions <= 0)
 +    if (rs->nb_completions <= 0) {
          return;
 +    }
      if (rs->nb_completions == 1) {
          len = strlen(rs->completions[0]);
          for (i = rs->completion_index; i < len; i++) {
              readline_insert_char(rs, rs->completions[0][i]);
          }
          /* extra space for next argument. XXX: make it more generic */
 -        if (len > 0 && rs->completions[0][len - 1] != '/')
 +        if (len > 0 && rs->completions[0][len - 1] != '/') {
              readline_insert_char(rs, ' ');
 +        }
      } else {
          qsort(rs->completions, rs->nb_completions, sizeof(char *),
                completion_comp);
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
              if (i == 0) {
                  max_prefix = len;
              } else {
 -                if (len < max_prefix)
 +                if (len < max_prefix) {
                      max_prefix = len;
 +                }
                  for (j = 0; j < max_prefix; j++) {
 -                    if (rs->completions[i][j] != rs->completions[0][j])
 +                    if (rs->completions[i][j] != rs->completions[0][j]) {
                          max_prefix = j;
 +                    }
                  }
              }
 -            if (len > max_width)
 +            if (len > max_width) {
                  max_width = len;
 +            }
-+        } while (ret < 0);
+         }
-     }
+         if (max_prefix > 0)
+             for (i = rs->completion_index; i < max_prefix; i++) {
--out:
+                 readline_insert_char(rs, rs->completions[0][i]);
--    bdrv_dirty_iter_free(dbi);
+             }
--    return ret;
+         max_width += 2;
-+    return 0;
+-        if (max_width < 10)
 +        if (max_width < 10) {
              max_width = 10;
 -        else if (max_width > 80)
 +        } else if (max_width > 80) {
              max_width = 80;
 +        }
          nb_cols = 80 / max_width;
          j = 0;
          for (i = 0; i < rs->nb_completions; i++) {
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
          case 10:
          case 13:
              rs->cmd_buf[rs->cmd_buf_size] = '\0';
 -            if (!rs->read_password)
 +            if (!rs->read_password) {
                  readline_hist_add(rs, rs->cmd_buf);
 +            }
              rs->printf_func(rs->opaque, "\n");
              rs->cmd_buf_index = 0;
              rs->cmd_buf_size = 0;
@@ -XXX,XX +XXX,XX @@ void readline_restart(ReadLineState *rs)
  const char *readline_get_history(ReadLineState *rs, unsigned int index)
  {
 -    if (index >= READLINE_MAX_CMDS)
 +    if (index >= READLINE_MAX_CMDS) {
          return NULL;
 +    }
      return rs->history[index];
  }
- /* init copy_bitmap from sync_bitmap */
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 07/10] block/sheepdog: remove spurious NULL check
+[Qemu-devel] [PULL 4/8] block: Add coroutine_fn to bdrv_check_co_entry
-'tag' is already checked in the lines immediately preceding this check,
+From: Nikita Alekseev <n.alekseev2104@gmail.com>
 and set to non-NULL if NULL.  No need to check again, it hasn't changed.
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+bdrv_check_co_entry calls bdrv_co_check, which is a coroutine function.
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Thus, it also needs to be marked as a coroutine.
-Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+Signed-off-by: Nikita Alekseev <n.alekseev2104@gmail.com>
 Message-id: 20190401093051.16488-1-n.alekseev2104@gmail.com
 Message-Id: <20190401093051.16488-1-n.alekseev2104@gmail.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/sheepdog.c | 2 +-
+ block.c | 2 +-
 file changed, 1 insertion(+), 1 deletion(-)
-diff --git a/block/sheepdog.c b/block/sheepdog.c
+diff --git a/block.c b/block.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/sheepdog.c
+--- a/block.c
-+++ b/block/sheepdog.c
++++ b/block.c
-@@ -XXX,XX +XXX,XX @@ static int sd_open(BlockDriverState *bs, QDict *options, int flags,
+@@ -XXX,XX +XXX,XX @@ typedef struct CheckCo {
-     if (!tag) {
+     int ret;
-         tag = "";
+ } CheckCo;
-     }
--    if (tag && strlen(tag) >= SD_MAX_VDI_TAG_LEN) {
+-static void bdrv_check_co_entry(void *opaque)
-+    if (strlen(tag) >= SD_MAX_VDI_TAG_LEN) {
++static void coroutine_fn bdrv_check_co_entry(void *opaque)
-         error_setg(errp, "value of parameter 'tag' is too long");
+ {
-         ret = -EINVAL;
+     CheckCo *cco = opaque;
-         goto err_no_fd;
+     cco->ret = bdrv_co_check(cco->bs, cco->res, cco->fix);
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 04/10] backup: simplify non-dirty bits progress processing
+[Qemu-devel] [PULL 5/8] block/io.c: fix for the allocation failure
-From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
-Set fake progress for non-dirty clusters in copy_bitmap initialization,
+On a file system used by the customer, fallocate() returns an error
-to. It simplifies code and allows further refactoring.
+if the block is not properly aligned. So, bdrv_co_pwrite_zeroes()
 fails. We can handle that case the same way as it is done for the
 unsupported cases, namely, call to bdrv_driver_pwritev() that writes
 zeroes to an image for the unaligned chunk of the block.
-This patch changes user's view of backup progress, but formally it
+Suggested-by: Denis V. Lunev <den@openvz.org>
-doesn't changed: progress hops are just moved to the beginning.
+Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
 Reviewed-by: John Snow <jsnow@redhat.com>
 Message-id: 1554474244-553661-1-git-send-email-andrey.shinkevich@virtuozzo.com
 Message-Id: <1554474244-553661-1-git-send-email-andrey.shinkevich@virtuozzo.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
  block/io.c | 2 +-
 file changed, 1 insertion(+), 1 deletion(-)
-Actually it's just a point of view: when do we actually skip clusters?
+diff --git a/block/io.c b/block/io.c
 We can say in the very beginning, that we skip these clusters and do
 not think about them later.
 Of course, if go through disk sequentially, it's logical to say, that
 we skip clusters between copied portions to the left and to the right
 of them. But even now copying progress is not sequential because of
 write notifiers. Future patches will introduce new backup architecture
 which will do copying in several coroutines in parallel, so it will
 make no sense to publish fake progress by parts in parallel with
 other copying requests.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: John Snow <jsnow@redhat.com>
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: Jeff Cody <jcody@redhat.com>
 Message-id: 20171012135313.227864-5-vsementsov@virtuozzo.com
 Signed-off-by: Jeff Cody <jcody@redhat.com>
 ---
  block/backup.c | 18 +++---------------
 file changed, 3 insertions(+), 15 deletions(-)
 diff --git a/block/backup.c b/block/backup.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/backup.c
+--- a/block/io.c
-+++ b/block/backup.c
++++ b/block/io.c
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
-     int64_t offset;
+             assert(!bs->supported_zero_flags);
      int64_t cluster;
      int64_t end;
 -    int64_t last_cluster = -1;
      BdrvDirtyBitmapIter *dbi;
      granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap);
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
      while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
          cluster = offset / job->cluster_size;
 -        /* Fake progress updates for any clusters we skipped */
 -        if (cluster != last_cluster + 1) {
 -            job->common.offset += ((cluster - last_cluster - 1) *
 -                                   job->cluster_size);
 -        }
 -
          for (end = cluster + clusters_per_iter; cluster < end; cluster++) {
              do {
                  if (yield_and_check(job)) {
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
          if (granularity < job->cluster_size) {
              bdrv_set_dirty_iter(dbi, cluster * job->cluster_size);
          }
--
--        last_cluster = cluster - 1;
+-        if (ret == -ENOTSUP && !(flags & BDRV_REQ_NO_FALLBACK)) {
--    }
++        if (ret < 0 && !(flags & BDRV_REQ_NO_FALLBACK)) {
--
+             /* Fall back to bounce buffer if write zeroes is unsupported */
--    /* Play some final catchup with the progress meter */
+             BdrvRequestFlags write_flags = flags & ~BDRV_REQ_ZERO_WRITE;
 -    end = DIV_ROUND_UP(job->common.len, job->cluster_size);
 -    if (last_cluster + 1 < end) {
 -        job->common.offset += ((end - last_cluster - 1) * job->cluster_size);
      }
  out:
@@ -XXX,XX +XXX,XX @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
          bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size);
      }
 +    job->common.offset = job->common.len -
 +                         hbitmap_count(job->copy_bitmap) * job->cluster_size;
 +
      bdrv_dirty_iter_free(dbi);
  }
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 09/10] block/curl: check error return of curl_global_init()
+[Qemu-devel] [PULL 6/8] aio-posix: ensure poll mode is left when aio_notify is called
-If curl_global_init() fails, per the documentation no other curl
+From: Paolo Bonzini <pbonzini@redhat.com>
 functions may be called, so make sure to check the return value.
-Also, some minor changes to the initialization latch variable 'inited':
+With aio=thread, adaptive polling makes latency worse rather than
 better, because it delays the execution of the ThreadPool's
 completion bottom half.
-- Make it static in the file, for clarity
+event_notifier_poll() does run while polling, detecting that
-- Change the name for clarity
+a bottom half was scheduled by a worker thread, but because
-- Make it a bool
+ctx->notifier is explicitly ignored in run_poll_handlers_once(),
 scheduling the BH does not count as making progress and
 run_poll_handlers() keeps running.  Fix this by recomputing
 the deadline after *timeout could have changed.
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+With this change, ThreadPool still cannot participate in polling
-Reviewed-by: Eric Blake <eblake@redhat.com>
+but at least it does not suffer from extra latency.
-Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
-Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
+Reported-by: Sergio Lopez <slp@redhat.com>
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
 Message-id: 20190409122823.12416-1-pbonzini@redhat.com
 Cc: Stefan Hajnoczi <stefanha@gmail.com>
 Cc: Kevin Wolf <kwolf@redhat.com>
 Cc: qemu-block@nongnu.org
 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
 Message-Id: <1553692145-86728-1-git-send-email-pbonzini@redhat.com>
 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
 Message-Id: <20190409122823.12416-1-pbonzini@redhat.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/curl.c | 18 ++++++++++++------
+ util/aio-posix.c | 12 ++++++++----
-file changed, 12 insertions(+), 6 deletions(-)
+file changed, 8 insertions(+), 4 deletions(-)
-diff --git a/block/curl.c b/block/curl.c
+diff --git a/util/aio-posix.c b/util/aio-posix.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
+--- a/util/aio-posix.c
-+++ b/block/curl.c
++++ b/util/aio-posix.c
-@@ -XXX,XX +XXX,XX @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
+@@ -XXX,XX +XXX,XX @@ static bool run_poll_handlers_once(AioContext *ctx, int64_t *timeout)
+         if (!node->deleted && node->io_poll &&
- struct BDRVCURLState;
+             aio_node_check(ctx, node->is_external) &&
+             node->io_poll(node->opaque)) {
-+static bool libcurl_initialized;
++            /*
-+
++             * Polling was successful, exit try_poll_mode immediately
- typedef struct CURLAIOCB {
++             * to adjust the next polling time.
-     Coroutine *co;
++             */
-     QEMUIOVector *qiov;
+             *timeout = 0;
-@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
+             if (node->opaque != &ctx->notifier) {
-     double d;
+                 progress = true;
-     const char *secretid;
+@@ -XXX,XX +XXX,XX @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns, int64_t *timeout)
-     const char *protocol_delimiter;
+     do {
-+    int ret;
+         progress = run_poll_handlers_once(ctx, timeout);
+         elapsed_time = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start_time;
--    static int inited = 0;
+-    } while (!progress && elapsed_time < max_ns
+-             && !atomic_read(&ctx->poll_disable_cnt));
-     if (flags & BDRV_O_RDWR) {
++        max_ns = qemu_soonest_timeout(*timeout, max_ns);
-         error_setg(errp, "curl block device does not support writes");
++        assert(!(max_ns && progress));
-         return -EROFS;
++    } while (elapsed_time < max_ns && !atomic_read(&ctx->poll_disable_cnt));
-     }
+     /* If time has passed with no successful polling, adjust *timeout to
-+    if (!libcurl_initialized) {
+      * keep the same ending time.
-+        ret = curl_global_init(CURL_GLOBAL_ALL);
+@@ -XXX,XX +XXX,XX @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns, int64_t *timeout)
-+        if (ret) {
+  */
-+            error_setg(errp, "libcurl initialization failed with %d", ret);
+ static bool try_poll_mode(AioContext *ctx, int64_t *timeout)
-+            return -EIO;
+ {
-+        }
+-    /* See qemu_soonest_timeout() uint64_t hack */
-+        libcurl_initialized = true;
+-    int64_t max_ns = MIN((uint64_t)*timeout, (uint64_t)ctx->poll_ns);
-+    }
++    int64_t max_ns = qemu_soonest_timeout(*timeout, ctx->poll_ns);
-+
-     qemu_mutex_init(&s->mutex);
+     if (max_ns && !atomic_read(&ctx->poll_disable_cnt)) {
-     opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
+         poll_set_started(ctx, true);
      qemu_opts_absorb_qdict(opts, options, &local_err);
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
          }
      }
 -    if (!inited) {
 -        curl_global_init(CURL_GLOBAL_ALL);
 -        inited = 1;
 -    }
 -
      DPRINTF("CURL: Opening %s\n", file);
      QSIMPLEQ_INIT(&s->free_state_waitq);
      s->aio_context = bdrv_get_aio_context(bs);
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 08/10] block/sheepdog: code beautification
+[Qemu-devel] [PULL 7/8] docs: add Secure Coding Practices to developer docs
-No functional changes, just whitespace manipulation.
+At KVM Forum 2018 I gave a presentation on security in QEMU:
 https://www.youtube.com/watch?v=YAdRf_hwxU8 (video)
 https://vmsplice.net/~stefan/stefanha-kvm-forum-2018.pdf (slides)
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+This patch adds a guide to secure coding practices.  This document
-Reviewed-by: Eric Blake <eblake@redhat.com>
+covers things that developers should know about security in QEMU.  It is
-Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
+just a starting point that we can expand on later.  I hope it will be
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+useful as a resource for new contributors and will save code reviewers
 from explaining the same concepts many times.
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 Acked-by: Stefano Garzarella <sgarzare@redhat.com>
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
 Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
 Reviewed-by: Li Qiang <liq3ea@gmail.com>
 Message-id: 20190509121820.16294-2-stefanha@redhat.com
 Message-Id: <20190509121820.16294-2-stefanha@redhat.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/sheepdog.c | 164 +++++++++++++++++++++++++++----------------------------
+ docs/devel/index.rst                   |   1 +
-file changed, 82 insertions(+), 82 deletions(-)
+ docs/devel/secure-coding-practices.rst | 106 +++++++++++++++++++++++++
 files changed, 107 insertions(+)
  create mode 100644 docs/devel/secure-coding-practices.rst
-diff --git a/block/sheepdog.c b/block/sheepdog.c
+diff --git a/docs/devel/index.rst b/docs/devel/index.rst
 index XXXXXXX..XXXXXXX 100644
---- a/block/sheepdog.c
+--- a/docs/devel/index.rst
-+++ b/block/sheepdog.c
++++ b/docs/devel/index.rst
-@@ -XXX,XX +XXX,XX @@ typedef struct BDRVSheepdogReopenState {
+@@ -XXX,XX +XXX,XX @@ Contents:
-     int cache_flags;
+    stable-process
- } BDRVSheepdogReopenState;
+    testing
+    decodetree
--static const char * sd_strerror(int err)
++   secure-coding-practices
-+static const char *sd_strerror(int err)
+diff --git a/docs/devel/secure-coding-practices.rst b/docs/devel/secure-coding-practices.rst
- {
+new file mode 100644
-     int i;
+index XXXXXXX..XXXXXXX
+--- /dev/null
-@@ -XXX,XX +XXX,XX @@ static QemuOptsList sd_create_opts = {
++++ b/docs/devel/secure-coding-practices.rst
- };
+@@ -XXX,XX +XXX,XX @@
++=======================
- static BlockDriver bdrv_sheepdog = {
++Secure Coding Practices
--    .format_name    = "sheepdog",
++=======================
--    .protocol_name  = "sheepdog",
++This document covers topics that both developers and security researchers must
--    .instance_size  = sizeof(BDRVSheepdogState),
++be aware of so that they can develop safe code and audit existing code
--    .bdrv_parse_filename    = sd_parse_filename,
++properly.
--    .bdrv_file_open = sd_open,
++
--    .bdrv_reopen_prepare    = sd_reopen_prepare,
++Reporting Security Bugs
--    .bdrv_reopen_commit     = sd_reopen_commit,
++-----------------------
--    .bdrv_reopen_abort      = sd_reopen_abort,
++For details on how to report security bugs or ask questions about potential
--    .bdrv_close     = sd_close,
++security bugs, see the `Security Process wiki page
--    .bdrv_create    = sd_create,
++<https://wiki.qemu.org/SecurityProcess>`_.
--    .bdrv_has_zero_init = bdrv_has_zero_init_1,
++
--    .bdrv_getlength = sd_getlength,
++General Secure C Coding Practices
-+    .format_name                  = "sheepdog",
++---------------------------------
-+    .protocol_name                = "sheepdog",
++Most CVEs (security bugs) reported against QEMU are not specific to
-+    .instance_size                = sizeof(BDRVSheepdogState),
++virtualization or emulation.  They are simply C programming bugs.  Therefore
-+    .bdrv_parse_filename          = sd_parse_filename,
++it's critical to be aware of common classes of security bugs.
-+    .bdrv_file_open               = sd_open,
++
-+    .bdrv_reopen_prepare          = sd_reopen_prepare,
++There is a wide selection of resources available covering secure C coding.  For
-+    .bdrv_reopen_commit           = sd_reopen_commit,
++example, the `CERT C Coding Standard
-+    .bdrv_reopen_abort            = sd_reopen_abort,
++<https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Coding+Standard>`_
-+    .bdrv_close                   = sd_close,
++covers the most important classes of security bugs.
-+    .bdrv_create                  = sd_create,
++
-+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
++Instead of describing them in detail here, only the names of the most important
-+    .bdrv_getlength               = sd_getlength,
++classes of security bugs are mentioned:
-     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
++
--    .bdrv_truncate  = sd_truncate,
++* Buffer overflows
-+    .bdrv_truncate                = sd_truncate,
++* Use-after-free and double-free
++* Integer overflows
--    .bdrv_co_readv  = sd_co_readv,
++* Format string vulnerabilities
--    .bdrv_co_writev = sd_co_writev,
++
--    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
++Some of these classes of bugs can be detected by analyzers.  Static analysis is
--    .bdrv_co_pdiscard = sd_co_pdiscard,
++performed regularly by Coverity and the most obvious of these bugs are even
--    .bdrv_co_get_block_status = sd_co_get_block_status,
++reported by compilers.  Dynamic analysis is possible with valgrind, tsan, and
-+    .bdrv_co_readv                = sd_co_readv,
++asan.
-+    .bdrv_co_writev               = sd_co_writev,
++
-+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
++Input Validation
-+    .bdrv_co_pdiscard             = sd_co_pdiscard,
++----------------
-+    .bdrv_co_get_block_status     = sd_co_get_block_status,
++Inputs from the guest or external sources (e.g. network, files) cannot be
++trusted and may be invalid.  Inputs must be checked before using them in a way
--    .bdrv_snapshot_create   = sd_snapshot_create,
++that could crash the program, expose host memory to the guest, or otherwise be
--    .bdrv_snapshot_goto     = sd_snapshot_goto,
++exploitable by an attacker.
--    .bdrv_snapshot_delete   = sd_snapshot_delete,
++
--    .bdrv_snapshot_list     = sd_snapshot_list,
++The most sensitive attack surface is device emulation.  All hardware register
-+    .bdrv_snapshot_create         = sd_snapshot_create,
++accesses and data read from guest memory must be validated.  A typical example
-+    .bdrv_snapshot_goto           = sd_snapshot_goto,
++is a device that contains multiple units that are selectable by the guest via
-+    .bdrv_snapshot_delete         = sd_snapshot_delete,
++an index register::
-+    .bdrv_snapshot_list           = sd_snapshot_list,
++
++  typedef struct {
--    .bdrv_save_vmstate  = sd_save_vmstate,
++      ProcessingUnit unit[2];
--    .bdrv_load_vmstate  = sd_load_vmstate,
++      ...
-+    .bdrv_save_vmstate            = sd_save_vmstate,
++  } MyDeviceState;
-+    .bdrv_load_vmstate            = sd_load_vmstate,
++
++  static void mydev_writel(void *opaque, uint32_t addr, uint32_t val)
--    .bdrv_detach_aio_context = sd_detach_aio_context,
++  {
--    .bdrv_attach_aio_context = sd_attach_aio_context,
++      MyDeviceState *mydev = opaque;
-+    .bdrv_detach_aio_context      = sd_detach_aio_context,
++      ProcessingUnit *unit;
-+    .bdrv_attach_aio_context      = sd_attach_aio_context,
++
++      switch (addr) {
--    .create_opts    = &sd_create_opts,
++      case MYDEV_SELECT_UNIT:
-+    .create_opts                  = &sd_create_opts,
++          unit = &mydev->unit[val];   <-- this input wasn't validated!
- };
++          ...
++      }
- static BlockDriver bdrv_sheepdog_tcp = {
++  }
--    .format_name    = "sheepdog",
++
--    .protocol_name  = "sheepdog+tcp",
++If ``val`` is not in range [0, 1] then an out-of-bounds memory access will take
--    .instance_size  = sizeof(BDRVSheepdogState),
++place when ``unit`` is dereferenced.  The code must check that ``val`` is 0 or
--    .bdrv_parse_filename    = sd_parse_filename,
++1 and handle the case where it is invalid.
--    .bdrv_file_open = sd_open,
++
--    .bdrv_reopen_prepare    = sd_reopen_prepare,
++Unexpected Device Accesses
--    .bdrv_reopen_commit     = sd_reopen_commit,
++--------------------------
--    .bdrv_reopen_abort      = sd_reopen_abort,
++The guest may access device registers in unusual orders or at unexpected
--    .bdrv_close     = sd_close,
++moments.  Device emulation code must not assume that the guest follows the
--    .bdrv_create    = sd_create,
++typical "theory of operation" presented in driver writer manuals.  The guest
--    .bdrv_has_zero_init = bdrv_has_zero_init_1,
++may make nonsense accesses to device registers such as starting operations
--    .bdrv_getlength = sd_getlength,
++before the device has been fully initialized.
-+    .format_name                  = "sheepdog",
++
-+    .protocol_name                = "sheepdog+tcp",
++A related issue is that device emulation code must be prepared for unexpected
-+    .instance_size                = sizeof(BDRVSheepdogState),
++device register accesses while asynchronous operations are in progress.  A
-+    .bdrv_parse_filename          = sd_parse_filename,
++well-behaved guest might wait for a completion interrupt before accessing
-+    .bdrv_file_open               = sd_open,
++certain device registers.  Device emulation code must handle the case where the
-+    .bdrv_reopen_prepare          = sd_reopen_prepare,
++guest overwrites registers or submits further requests before an ongoing
-+    .bdrv_reopen_commit           = sd_reopen_commit,
++request completes.  Unexpected accesses must not cause memory corruption or
-+    .bdrv_reopen_abort            = sd_reopen_abort,
++leaks in QEMU.
-+    .bdrv_close                   = sd_close,
++
-+    .bdrv_create                  = sd_create,
++Invalid device register accesses can be reported with
-+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
++``qemu_log_mask(LOG_GUEST_ERROR, ...)``.  The ``-d guest_errors`` command-line
-+    .bdrv_getlength               = sd_getlength,
++option enables these log messages.
-     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
++
--    .bdrv_truncate  = sd_truncate,
++Live Migration
-+    .bdrv_truncate                = sd_truncate,
++--------------
++Device state can be saved to disk image files and shared with other users.
--    .bdrv_co_readv  = sd_co_readv,
++Live migration code must validate inputs when loading device state so an
--    .bdrv_co_writev = sd_co_writev,
++attacker cannot gain control by crafting invalid device states.  Device state
--    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
++is therefore considered untrusted even though it is typically generated by QEMU
--    .bdrv_co_pdiscard = sd_co_pdiscard,
++itself.
--    .bdrv_co_get_block_status = sd_co_get_block_status,
++
-+    .bdrv_co_readv                = sd_co_readv,
++Guest Memory Access Races
-+    .bdrv_co_writev               = sd_co_writev,
++-------------------------
-+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
++Guests with multiple vCPUs may modify guest RAM while device emulation code is
-+    .bdrv_co_pdiscard             = sd_co_pdiscard,
++running.  Device emulation code must copy in descriptors and other guest RAM
-+    .bdrv_co_get_block_status     = sd_co_get_block_status,
++structures and only process the local copy.  This prevents
++time-of-check-to-time-of-use (TOCTOU) race conditions that could cause QEMU to
--    .bdrv_snapshot_create   = sd_snapshot_create,
++crash when a vCPU thread modifies guest RAM while device emulation is
--    .bdrv_snapshot_goto     = sd_snapshot_goto,
++processing it.
 -    .bdrv_snapshot_delete   = sd_snapshot_delete,
 -    .bdrv_snapshot_list     = sd_snapshot_list,
 +    .bdrv_snapshot_create         = sd_snapshot_create,
 +    .bdrv_snapshot_goto           = sd_snapshot_goto,
 +    .bdrv_snapshot_delete         = sd_snapshot_delete,
 +    .bdrv_snapshot_list           = sd_snapshot_list,
 -    .bdrv_save_vmstate  = sd_save_vmstate,
 -    .bdrv_load_vmstate  = sd_load_vmstate,
 +    .bdrv_save_vmstate            = sd_save_vmstate,
 +    .bdrv_load_vmstate            = sd_load_vmstate,
 -    .bdrv_detach_aio_context = sd_detach_aio_context,
 -    .bdrv_attach_aio_context = sd_attach_aio_context,
 +    .bdrv_detach_aio_context      = sd_detach_aio_context,
 +    .bdrv_attach_aio_context      = sd_attach_aio_context,
 -    .create_opts    = &sd_create_opts,
 +    .create_opts                  = &sd_create_opts,
  };
  static BlockDriver bdrv_sheepdog_unix = {
 -    .format_name    = "sheepdog",
 -    .protocol_name  = "sheepdog+unix",
 -    .instance_size  = sizeof(BDRVSheepdogState),
 -    .bdrv_parse_filename    = sd_parse_filename,
 -    .bdrv_file_open = sd_open,
 -    .bdrv_reopen_prepare    = sd_reopen_prepare,
 -    .bdrv_reopen_commit     = sd_reopen_commit,
 -    .bdrv_reopen_abort      = sd_reopen_abort,
 -    .bdrv_close     = sd_close,
 -    .bdrv_create    = sd_create,
 -    .bdrv_has_zero_init = bdrv_has_zero_init_1,
 -    .bdrv_getlength = sd_getlength,
 +    .format_name                  = "sheepdog",
 +    .protocol_name                = "sheepdog+unix",
 +    .instance_size                = sizeof(BDRVSheepdogState),
 +    .bdrv_parse_filename          = sd_parse_filename,
 +    .bdrv_file_open               = sd_open,
 +    .bdrv_reopen_prepare          = sd_reopen_prepare,
 +    .bdrv_reopen_commit           = sd_reopen_commit,
 +    .bdrv_reopen_abort            = sd_reopen_abort,
 +    .bdrv_close                   = sd_close,
 +    .bdrv_create                  = sd_create,
 +    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
 +    .bdrv_getlength               = sd_getlength,
      .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
 -    .bdrv_truncate  = sd_truncate,
 +    .bdrv_truncate                = sd_truncate,
 -    .bdrv_co_readv  = sd_co_readv,
 -    .bdrv_co_writev = sd_co_writev,
 -    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
 -    .bdrv_co_pdiscard = sd_co_pdiscard,
 -    .bdrv_co_get_block_status = sd_co_get_block_status,
 +    .bdrv_co_readv                = sd_co_readv,
 +    .bdrv_co_writev               = sd_co_writev,
 +    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
 +    .bdrv_co_pdiscard             = sd_co_pdiscard,
 +    .bdrv_co_get_block_status     = sd_co_get_block_status,
 -    .bdrv_snapshot_create   = sd_snapshot_create,
 -    .bdrv_snapshot_goto     = sd_snapshot_goto,
 -    .bdrv_snapshot_delete   = sd_snapshot_delete,
 -    .bdrv_snapshot_list     = sd_snapshot_list,
 +    .bdrv_snapshot_create         = sd_snapshot_create,
 +    .bdrv_snapshot_goto           = sd_snapshot_goto,
 +    .bdrv_snapshot_delete         = sd_snapshot_delete,
 +    .bdrv_snapshot_list           = sd_snapshot_list,
 -    .bdrv_save_vmstate  = sd_save_vmstate,
 -    .bdrv_load_vmstate  = sd_load_vmstate,
 +    .bdrv_save_vmstate            = sd_save_vmstate,
 +    .bdrv_load_vmstate            = sd_load_vmstate,
 -    .bdrv_detach_aio_context = sd_detach_aio_context,
 -    .bdrv_attach_aio_context = sd_attach_aio_context,
 +    .bdrv_detach_aio_context      = sd_detach_aio_context,
 +    .bdrv_attach_aio_context      = sd_attach_aio_context,
 -    .create_opts    = &sd_create_opts,
 +    .create_opts                  = &sd_create_opts,
  };
  static void bdrv_sheepdog_init(void)
 --
-.9.5
+.21.0

-[Qemu-devel] [PULL 03/10] backup: init copy_bitmap from sync_bitmap for incremental
+[Qemu-devel] [PULL 8/8] docs: add Security chapter to the documentation
-From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+This new chapter in the QEMU documentation covers the security
 requirements that QEMU is designed to meet and principles for securely
 deploying QEMU.
-We should not copy non-dirty clusters in write notifiers. So,
+It is just a starting point that can be extended in the future with more
-initialize copy_bitmap from sync_bitmap.
+information.
-Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
+Acked-by: Stefano Garzarella <sgarzare@redhat.com>
-Reviewed-by: Jeff Cody <jcody@redhat.com>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
-Message-id: 20171012135313.227864-4-vsementsov@virtuozzo.com
+Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
-Signed-off-by: Jeff Cody <jcody@redhat.com>
+Reviewed-by: Li Qiang <liq3ea@gmail.com>
 Message-id: 20190509121820.16294-3-stefanha@redhat.com
 Message-Id: <20190509121820.16294-3-stefanha@redhat.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/backup.c | 44 +++++++++++++++++++++++++++++++++++++++++++-
+ Makefile           |   2 +-
-file changed, 43 insertions(+), 1 deletion(-)
+ docs/security.texi | 131 +++++++++++++++++++++++++++++++++++++++++++++
  qemu-doc.texi      |   3 ++
 files changed, 135 insertions(+), 1 deletion(-)
  create mode 100644 docs/security.texi
-diff --git a/block/backup.c b/block/backup.c
+diff --git a/Makefile b/Makefile
 index XXXXXXX..XXXXXXX 100644
---- a/block/backup.c
+--- a/Makefile
-+++ b/block/backup.c
++++ b/Makefile
-@@ -XXX,XX +XXX,XX @@ out:
+@@ -XXX,XX +XXX,XX @@ qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
-     return ret;
+     qemu-img.texi qemu-nbd.texi qemu-options.texi qemu-option-trace.texi \
- }
+     qemu-deprecated.texi qemu-monitor.texi qemu-img-cmds.texi qemu-ga.texi \
+     qemu-monitor-info.texi docs/qemu-block-drivers.texi \
-+/* init copy_bitmap from sync_bitmap */
+-    docs/qemu-cpu-models.texi
-+static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
++    docs/qemu-cpu-models.texi docs/security.texi
-+{
-+    BdrvDirtyBitmapIter *dbi;
+ docs/interop/qemu-ga-ref.dvi docs/interop/qemu-ga-ref.html \
-+    int64_t offset;
+     docs/interop/qemu-ga-ref.info docs/interop/qemu-ga-ref.pdf \
-+    int64_t end = DIV_ROUND_UP(bdrv_dirty_bitmap_size(job->sync_bitmap),
+diff --git a/docs/security.texi b/docs/security.texi
-+                               job->cluster_size);
+new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/docs/security.texi
@@ -XXX,XX +XXX,XX @@
 +@node Security
 +@chapter Security
 +
-+    dbi = bdrv_dirty_iter_new(job->sync_bitmap);
++@section Overview
 +    while ((offset = bdrv_dirty_iter_next(dbi)) != -1) {
 +        int64_t cluster = offset / job->cluster_size;
 +        int64_t next_cluster;
 +
-+        offset += bdrv_dirty_bitmap_granularity(job->sync_bitmap);
++This chapter explains the security requirements that QEMU is designed to meet
-+        if (offset >= bdrv_dirty_bitmap_size(job->sync_bitmap)) {
++and principles for securely deploying QEMU.
 +            hbitmap_set(job->copy_bitmap, cluster, end - cluster);
 +            break;
 +        }
 +
-+        offset = bdrv_dirty_bitmap_next_zero(job->sync_bitmap, offset);
++@section Security Requirements
 +        if (offset == -1) {
 +            hbitmap_set(job->copy_bitmap, cluster, end - cluster);
 +            break;
 +        }
 +
-+        next_cluster = DIV_ROUND_UP(offset, job->cluster_size);
++QEMU supports many different use cases, some of which have stricter security
-+        hbitmap_set(job->copy_bitmap, cluster, next_cluster - cluster);
++requirements than others.  The community has agreed on the overall security
-+        if (next_cluster >= end) {
++requirements that users may depend on.  These requirements define what is
-+            break;
++considered supported from a security perspective.
 +        }
 +
-+        bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size);
++@subsection Virtualization Use Case
 +    }
 +
-+    bdrv_dirty_iter_free(dbi);
++The virtualization use case covers cloud and virtual private server (VPS)
-+}
++hosting, as well as traditional data center and desktop virtualization.  These
 +use cases rely on hardware virtualization extensions to execute guest code
 +safely on the physical CPU at close-to-native speed.
 +
- static void coroutine_fn backup_run(void *opaque)
++The following entities are untrusted, meaning that they may be buggy or
- {
++malicious:
      BackupBlockJob *job = opaque;
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque)
      nb_clusters = DIV_ROUND_UP(job->common.len, job->cluster_size);
      job->copy_bitmap = hbitmap_alloc(nb_clusters, 0);
 -    hbitmap_set(job->copy_bitmap, 0, nb_clusters);
 +    if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
 +        backup_incremental_init_copy_bitmap(job);
 +    } else {
 +        hbitmap_set(job->copy_bitmap, 0, nb_clusters);
 +    }
 +
++@itemize
-     job->before_write.notify = backup_before_write_notify;
++@item Guest
-     bdrv_add_before_write_notifier(bs, &job->before_write);
++@item User-facing interfaces (e.g. VNC, SPICE, WebSocket)
 +@item Network protocols (e.g. NBD, live migration)
 +@item User-supplied files (e.g. disk images, kernels, device trees)
 +@item Passthrough devices (e.g. PCI, USB)
 +@end itemize
 +
 +Bugs affecting these entities are evaluated on whether they can cause damage in
 +real-world use cases and treated as security bugs if this is the case.
 +
 +@subsection Non-virtualization Use Case
 +
 +The non-virtualization use case covers emulation using the Tiny Code Generator
 +(TCG).  In principle the TCG and device emulation code used in conjunction with
 +the non-virtualization use case should meet the same security requirements as
 +the virtualization use case.  However, for historical reasons much of the
 +non-virtualization use case code was not written with these security
 +requirements in mind.
 +
 +Bugs affecting the non-virtualization use case are not considered security
 +bugs at this time.  Users with non-virtualization use cases must not rely on
 +QEMU to provide guest isolation or any security guarantees.
 +
 +@section Architecture
 +
 +This section describes the design principles that ensure the security
 +requirements are met.
 +
 +@subsection Guest Isolation
 +
 +Guest isolation is the confinement of guest code to the virtual machine.  When
 +guest code gains control of execution on the host this is called escaping the
 +virtual machine.  Isolation also includes resource limits such as throttling of
 +CPU, memory, disk, or network.  Guests must be unable to exceed their resource
 +limits.
 +
 +QEMU presents an attack surface to the guest in the form of emulated devices.
 +The guest must not be able to gain control of QEMU.  Bugs in emulated devices
 +could allow malicious guests to gain code execution in QEMU.  At this point the
 +guest has escaped the virtual machine and is able to act in the context of the
 +QEMU process on the host.
 +
 +Guests often interact with other guests and share resources with them.  A
 +malicious guest must not gain control of other guests or access their data.
 +Disk image files and network traffic must be protected from other guests unless
 +explicitly shared between them by the user.
 +
 +@subsection Principle of Least Privilege
 +
 +The principle of least privilege states that each component only has access to
 +the privileges necessary for its function.  In the case of QEMU this means that
 +each process only has access to resources belonging to the guest.
 +
 +The QEMU process should not have access to any resources that are inaccessible
 +to the guest.  This way the guest does not gain anything by escaping into the
 +QEMU process since it already has access to those same resources from within
 +the guest.
 +
 +Following the principle of least privilege immediately fulfills guest isolation
 +requirements.  For example, guest A only has access to its own disk image file
 +@code{a.img} and not guest B's disk image file @code{b.img}.
 +
 +In reality certain resources are inaccessible to the guest but must be
 +available to QEMU to perform its function.  For example, host system calls are
 +necessary for QEMU but are not exposed to guests.  A guest that escapes into
 +the QEMU process can then begin invoking host system calls.
 +
 +New features must be designed to follow the principle of least privilege.
 +Should this not be possible for technical reasons, the security risk must be
 +clearly documented so users are aware of the trade-off of enabling the feature.
 +
 +@subsection Isolation mechanisms
 +
 +Several isolation mechanisms are available to realize this architecture of
 +guest isolation and the principle of least privilege.  With the exception of
 +Linux seccomp, these mechanisms are all deployed by management tools that
 +launch QEMU, such as libvirt.  They are also platform-specific so they are only
 +described briefly for Linux here.
 +
 +The fundamental isolation mechanism is that QEMU processes must run as
 +unprivileged users.  Sometimes it seems more convenient to launch QEMU as
 +root to give it access to host devices (e.g. @code{/dev/net/tun}) but this poses a
 +huge security risk.  File descriptor passing can be used to give an otherwise
 +unprivileged QEMU process access to host devices without running QEMU as root.
 +It is also possible to launch QEMU as a non-root user and configure UNIX groups
 +for access to @code{/dev/kvm}, @code{/dev/net/tun}, and other device nodes.
 +Some Linux distros already ship with UNIX groups for these devices by default.
 +
 +@itemize
 +@item SELinux and AppArmor make it possible to confine processes beyond the
 +traditional UNIX process and file permissions model.  They restrict the QEMU
 +process from accessing processes and files on the host system that are not
 +needed by QEMU.
 +
 +@item Resource limits and cgroup controllers provide throughput and utilization
 +limits on key resources such as CPU time, memory, and I/O bandwidth.
 +
 +@item Linux namespaces can be used to make process, file system, and other system
 +resources unavailable to QEMU.  A namespaced QEMU process is restricted to only
 +those resources that were granted to it.
 +
 +@item Linux seccomp is available via the QEMU @option{--sandbox} option.  It disables
 +system calls that are not needed by QEMU, thereby reducing the host kernel
 +attack surface.
 +@end itemize
 diff --git a/qemu-doc.texi b/qemu-doc.texi
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-doc.texi
 +++ b/qemu-doc.texi
@@ -XXX,XX +XXX,XX @@
  * QEMU Guest Agent::
  * QEMU User space emulator::
  * System requirements::
 +* Security::
  * Implementation notes::
  * Deprecated features::
  * Supported build platforms::
@@ -XXX,XX +XXX,XX @@ added with Linux 4.5 which is supported by the major distros. And even
  if RHEL7 has kernel 3.10, KVM there has the required functionality there
  to make it close to a 4.5 or newer kernel.
 +@include docs/security.texi
 +
  @include qemu-tech.texi
  @include qemu-deprecated.texi
 --
-.9.5
+.21.0

The following changes since commit 411ad78115ebeb3411cf4b7622784b93dfabe259:

Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2017-12-15-1' into staging (2017-12-17 15:27:41 +0000)

are available in the git repository at:

git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request

for you to fetch changes up to 996922de45299878cdc4c15b72b19edf2bc618a4:

block/curl: fix minor memory leaks (2017-12-18 15:44:39 -0500)

----------------------------------------------------------------
Blockjob and protocol patches
----------------------------------------------------------------

Jeff Cody (4):
  block/sheepdog: remove spurious NULL check
  block/sheepdog: code beautification
  block/curl: check error return of curl_global_init()
  block/curl: fix minor memory leaks

John Snow (1):
  blockjob: kick jobs on set-speed

Vladimir Sementsov-Ogievskiy (5):
  hbitmap: add next_zero function
  backup: move from done_bitmap to copy_bitmap
  backup: init copy_bitmap from sync_bitmap for incremental
  backup: simplify non-dirty bits progress processing
  backup: use copy_bitmap in incremental backup

-- 
2.9.5

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The function searches for next zero bit.
Also add interface for BdrvDirtyBitmap and unit test.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20171012135313.227864-2-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/dirty-bitmap.c         |  5 ++++
 include/block/dirty-bitmap.h |  1 +
 include/qemu/hbitmap.h       |  8 ++++++
 tests/test-hbitmap.c         | 61 ++++++++++++++++++++++++++++++++++++++++++++
 util/hbitmap.c               | 39 ++++++++++++++++++++++++++++
 5 files changed, 114 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index XXXXXXX..XXXXXXX 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -XXX,XX +XXX,XX @@ char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp)
 {
     return hbitmap_sha256(bitmap->bitmap, errp);
 }
+
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset)
+{
+    return hbitmap_next_zero(bitmap->bitmap, offset);
+}
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -XXX,XX +XXX,XX @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
 BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
                                         BdrvDirtyBitmap *bitmap);
 char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start);
 
 #endif
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -XXX,XX +XXX,XX @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
  */
 unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
 
+/* hbitmap_next_zero:
+ * @hb: The HBitmap to operate on
+ * @start: The bit to start from.
+ *
+ * Find next not dirty bit.
+ */
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start);
+
 /* hbitmap_create_meta:
  * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
  * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index XXXXXXX..XXXXXXX 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -XXX,XX +XXX,XX @@ static void test_hbitmap_iter_and_reset(TestHBitmapData *data,
     hbitmap_iter_next(&hbi);
 }
 
+static void test_hbitmap_next_zero_check(TestHBitmapData *data, int64_t start)
+{
+    int64_t ret1 = hbitmap_next_zero(data->hb, start);
+    int64_t ret2 = start;
+    for ( ; ret2 < data->size && hbitmap_get(data->hb, ret2); ret2++) {
+        ;
+    }
+    if (ret2 == data->size) {
+        ret2 = -1;
+    }
+
+    g_assert_cmpint(ret1, ==, ret2);
+}
+
+static void test_hbitmap_next_zero_do(TestHBitmapData *data, int granularity)
+{
+    hbitmap_test_init(data, L3, granularity);
+    test_hbitmap_next_zero_check(data, 0);
+    test_hbitmap_next_zero_check(data, L3 - 1);
+
+    hbitmap_set(data->hb, L2, 1);
+    test_hbitmap_next_zero_check(data, 0);
+    test_hbitmap_next_zero_check(data, L2 - 1);
+    test_hbitmap_next_zero_check(data, L2);
+    test_hbitmap_next_zero_check(data, L2 + 1);
+
+    hbitmap_set(data->hb, L2 + 5, L1);
+    test_hbitmap_next_zero_check(data, 0);
+    test_hbitmap_next_zero_check(data, L2 + 1);
+    test_hbitmap_next_zero_check(data, L2 + 2);
+    test_hbitmap_next_zero_check(data, L2 + 5);
+    test_hbitmap_next_zero_check(data, L2 + L1 - 1);
+    test_hbitmap_next_zero_check(data, L2 + L1);
+
+    hbitmap_set(data->hb, L2 * 2, L3 - L2 * 2);
+    test_hbitmap_next_zero_check(data, L2 * 2 - L1);
+    test_hbitmap_next_zero_check(data, L2 * 2 - 2);
+    test_hbitmap_next_zero_check(data, L2 * 2 - 1);
+    test_hbitmap_next_zero_check(data, L2 * 2);
+    test_hbitmap_next_zero_check(data, L3 - 1);
+
+    hbitmap_set(data->hb, 0, L3);
+    test_hbitmap_next_zero_check(data, 0);
+}
+
+static void test_hbitmap_next_zero_0(TestHBitmapData *data, const void *unused)
+{
+    test_hbitmap_next_zero_do(data, 0);
+}
+
+static void test_hbitmap_next_zero_4(TestHBitmapData *data, const void *unused)
+{
+    test_hbitmap_next_zero_do(data, 4);
+}
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
 
     hbitmap_test_add("/hbitmap/iter/iter_and_reset",
                      test_hbitmap_iter_and_reset);
+
+    hbitmap_test_add("/hbitmap/next_zero/next_zero_0",
+                     test_hbitmap_next_zero_0);
+    hbitmap_test_add("/hbitmap/next_zero/next_zero_4",
+                     test_hbitmap_next_zero_4);
+
     g_test_run();
 
     return 0;
diff --git a/util/hbitmap.c b/util/hbitmap.c
index XXXXXXX..XXXXXXX 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -XXX,XX +XXX,XX @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first)
     }
 }
 
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start)
+{
+    size_t pos = (start >> hb->granularity) >> BITS_PER_LEVEL;
+    unsigned long *last_lev = hb->levels[HBITMAP_LEVELS - 1];
+    uint64_t sz = hb->sizes[HBITMAP_LEVELS - 1];
+    unsigned long cur = last_lev[pos];
+    unsigned start_bit_offset =
+            (start >> hb->granularity) & (BITS_PER_LONG - 1);
+    int64_t res;
+
+    cur |= (1UL << start_bit_offset) - 1;
+    assert((start >> hb->granularity) < hb->size);
+
+    if (cur == (unsigned long)-1) {
+        do {
+            pos++;
+        } while (pos < sz && last_lev[pos] == (unsigned long)-1);
+
+        if (pos >= sz) {
+            return -1;
+        }
+
+        cur = last_lev[pos];
+    }
+
+    res = (pos << BITS_PER_LEVEL) + ctol(cur);
+    if (res >= hb->size) {
+        return -1;
+    }
+
+    res = res << hb->granularity;
+    if (res < start) {
+        assert(((start - res) >> hb->granularity) == 0);
+        return start;
+    }
+
+    return res;
+}
+
 bool hbitmap_empty(const HBitmap *hb)
 {
     return hb->count == 0;
-- 
2.9.5

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Use HBitmap copy_bitmap instead of done_bitmap. This is needed to
improve incremental backup in following patches and to unify backup
loop for full/incremental modes in future patches.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20171012135313.227864-3-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index XXXXXXX..XXXXXXX 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -XXX,XX +XXX,XX @@ typedef struct BackupBlockJob {
     BlockdevOnError on_target_error;
     CoRwlock flush_rwlock;
     uint64_t bytes_read;
-    unsigned long *done_bitmap;
     int64_t cluster_size;
     bool compress;
     NotifierWithReturn before_write;
     QLIST_HEAD(, CowRequest) inflight_reqs;
+
+    HBitmap *copy_bitmap;
 } BackupBlockJob;
 
 /* See if in-flight requests overlap and wait for them to complete */
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
     cow_request_begin(&cow_request, job, start, end);
 
     for (; start < end; start += job->cluster_size) {
-        if (test_bit(start / job->cluster_size, job->done_bitmap)) {
+        if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) {
             trace_backup_do_cow_skip(job, start);
             continue; /* already copied */
         }
+        hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1);
 
         trace_backup_do_cow_process(job, start);
 
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
             if (error_is_read) {
                 *error_is_read = true;
             }
+            hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1);
             goto out;
         }
 
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
             if (error_is_read) {
                 *error_is_read = false;
             }
+            hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1);
             goto out;
         }
 
-        set_bit(start / job->cluster_size, job->done_bitmap);
-
         /* Publish progress, guest I/O counts as progress too.  Note that the
          * offset field is an opaque progress value, it is not a disk offset.
          */
@@ -XXX,XX +XXX,XX @@ void backup_do_checkpoint(BlockJob *job, Error **errp)
     }
 
     len = DIV_ROUND_UP(backup_job->common.len, backup_job->cluster_size);
-    bitmap_zero(backup_job->done_bitmap, len);
+    hbitmap_set(backup_job->copy_bitmap, 0, len);
 }
 
 void backup_wait_for_overlapping_requests(BlockJob *job, int64_t offset,
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque)
     BackupBlockJob *job = opaque;
     BackupCompleteData *data;
     BlockDriverState *bs = blk_bs(job->common.blk);
-    int64_t offset;
+    int64_t offset, nb_clusters;
     int ret = 0;
 
     QLIST_INIT(&job->inflight_reqs);
     qemu_co_rwlock_init(&job->flush_rwlock);
 
-    job->done_bitmap = bitmap_new(DIV_ROUND_UP(job->common.len,
-                                               job->cluster_size));
+    nb_clusters = DIV_ROUND_UP(job->common.len, job->cluster_size);
+    job->copy_bitmap = hbitmap_alloc(nb_clusters, 0);
+    hbitmap_set(job->copy_bitmap, 0, nb_clusters);
 
     job->before_write.notify = backup_before_write_notify;
     bdrv_add_before_write_notifier(bs, &job->before_write);
 
     if (job->sync_mode == MIRROR_SYNC_MODE_NONE) {
+        /* All bits are set in copy_bitmap to allow any cluster to be copied.
+         * This does not actually require them to be copied. */
         while (!block_job_is_cancelled(&job->common)) {
             /* Yield until the job is cancelled.  We just let our before_write
              * notify callback service CoW requests. */
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque)
     /* wait until pending backup_do_cow() calls have completed */
     qemu_co_rwlock_wrlock(&job->flush_rwlock);
     qemu_co_rwlock_unlock(&job->flush_rwlock);
-    g_free(job->done_bitmap);
+    hbitmap_free(job->copy_bitmap);
 
     data = g_malloc(sizeof(*data));
     data->ret = ret;
-- 
2.9.5

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We should not copy non-dirty clusters in write notifiers. So,
initialize copy_bitmap from sync_bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20171012135313.227864-4-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 44 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 43 insertions(+), 1 deletion(-)

diff --git a/block/backup.c b/block/backup.c
index XXXXXXX..XXXXXXX 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -XXX,XX +XXX,XX @@ out:
     return ret;
 }
 
+/* init copy_bitmap from sync_bitmap */
+static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
+{
+    BdrvDirtyBitmapIter *dbi;
+    int64_t offset;
+    int64_t end = DIV_ROUND_UP(bdrv_dirty_bitmap_size(job->sync_bitmap),
+                               job->cluster_size);
+
+    dbi = bdrv_dirty_iter_new(job->sync_bitmap);
+    while ((offset = bdrv_dirty_iter_next(dbi)) != -1) {
+        int64_t cluster = offset / job->cluster_size;
+        int64_t next_cluster;
+
+        offset += bdrv_dirty_bitmap_granularity(job->sync_bitmap);
+        if (offset >= bdrv_dirty_bitmap_size(job->sync_bitmap)) {
+            hbitmap_set(job->copy_bitmap, cluster, end - cluster);
+            break;
+        }
+
+        offset = bdrv_dirty_bitmap_next_zero(job->sync_bitmap, offset);
+        if (offset == -1) {
+            hbitmap_set(job->copy_bitmap, cluster, end - cluster);
+            break;
+        }
+
+        next_cluster = DIV_ROUND_UP(offset, job->cluster_size);
+        hbitmap_set(job->copy_bitmap, cluster, next_cluster - cluster);
+        if (next_cluster >= end) {
+            break;
+        }
+
+        bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size);
+    }
+
+    bdrv_dirty_iter_free(dbi);
+}
+
 static void coroutine_fn backup_run(void *opaque)
 {
     BackupBlockJob *job = opaque;
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn backup_run(void *opaque)
 
     nb_clusters = DIV_ROUND_UP(job->common.len, job->cluster_size);
     job->copy_bitmap = hbitmap_alloc(nb_clusters, 0);
-    hbitmap_set(job->copy_bitmap, 0, nb_clusters);
+    if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
+        backup_incremental_init_copy_bitmap(job);
+    } else {
+        hbitmap_set(job->copy_bitmap, 0, nb_clusters);
+    }
+
 
     job->before_write.notify = backup_before_write_notify;
     bdrv_add_before_write_notifier(bs, &job->before_write);
-- 
2.9.5

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Set fake progress for non-dirty clusters in copy_bitmap initialization,
to. It simplifies code and allows further refactoring.

This patch changes user's view of backup progress, but formally it
doesn't changed: progress hops are just moved to the beginning.

Actually it's just a point of view: when do we actually skip clusters?
We can say in the very beginning, that we skip these clusters and do
not think about them later.

Of course, if go through disk sequentially, it's logical to say, that
we skip clusters between copied portions to the left and to the right
of them. But even now copying progress is not sequential because of
write notifiers. Future patches will introduce new backup architecture
which will do copying in several coroutines in parallel, so it will
make no sense to publish fake progress by parts in parallel with
other copying requests.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 20171012135313.227864-5-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index XXXXXXX..XXXXXXX 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
     int64_t offset;
     int64_t cluster;
     int64_t end;
-    int64_t last_cluster = -1;
     BdrvDirtyBitmapIter *dbi;
 
     granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap);
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
     while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
         cluster = offset / job->cluster_size;
 
-        /* Fake progress updates for any clusters we skipped */
-        if (cluster != last_cluster + 1) {
-            job->common.offset += ((cluster - last_cluster - 1) *
-                                   job->cluster_size);
-        }
-
         for (end = cluster + clusters_per_iter; cluster < end; cluster++) {
             do {
                 if (yield_and_check(job)) {
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
         if (granularity < job->cluster_size) {
             bdrv_set_dirty_iter(dbi, cluster * job->cluster_size);
         }
-
-        last_cluster = cluster - 1;
-    }
-
-    /* Play some final catchup with the progress meter */
-    end = DIV_ROUND_UP(job->common.len, job->cluster_size);
-    if (last_cluster + 1 < end) {
-        job->common.offset += ((end - last_cluster - 1) * job->cluster_size);
     }
 
 out:
@@ -XXX,XX +XXX,XX @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
         bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size);
     }
 
+    job->common.offset = job->common.len -
+                         hbitmap_count(job->copy_bitmap) * job->cluster_size;
+
     bdrv_dirty_iter_free(dbi);
 }
 
-- 
2.9.5

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We can use copy_bitmap instead of sync_bitmap. copy_bitmap is
initialized from sync_bitmap and it is more informative: we will not try
to process data, that is already in progress (by write notifier).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 20171012135313.227864-6-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 55 +++++++++++++++++--------------------------------------
 1 file changed, 17 insertions(+), 38 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index XXXXXXX..XXXXXXX 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job)
 
 static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
 {
+    int ret;
     bool error_is_read;
-    int ret = 0;
-    int clusters_per_iter;
-    uint32_t granularity;
-    int64_t offset;
     int64_t cluster;
-    int64_t end;
-    BdrvDirtyBitmapIter *dbi;
+    HBitmapIter hbi;
 
-    granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap);
-    clusters_per_iter = MAX((granularity / job->cluster_size), 1);
-    dbi = bdrv_dirty_iter_new(job->sync_bitmap);
-
-    /* Find the next dirty sector(s) */
-    while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
-        cluster = offset / job->cluster_size;
-
-        for (end = cluster + clusters_per_iter; cluster < end; cluster++) {
-            do {
-                if (yield_and_check(job)) {
-                    goto out;
-                }
-                ret = backup_do_cow(job, cluster * job->cluster_size,
-                                    job->cluster_size, &error_is_read,
-                                    false);
-                if ((ret < 0) &&
-                    backup_error_action(job, error_is_read, -ret) ==
-                    BLOCK_ERROR_ACTION_REPORT) {
-                    goto out;
-                }
-            } while (ret < 0);
-        }
-
-        /* If the bitmap granularity is smaller than the backup granularity,
-         * we need to advance the iterator pointer to the next cluster. */
-        if (granularity < job->cluster_size) {
-            bdrv_set_dirty_iter(dbi, cluster * job->cluster_size);
-        }
+    hbitmap_iter_init(&hbi, job->copy_bitmap, 0);
+    while ((cluster = hbitmap_iter_next(&hbi)) != -1) {
+        do {
+            if (yield_and_check(job)) {
+                return 0;
+            }
+            ret = backup_do_cow(job, cluster * job->cluster_size,
+                                job->cluster_size, &error_is_read, false);
+            if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
+                           BLOCK_ERROR_ACTION_REPORT)
+            {
+                return ret;
+            }
+        } while (ret < 0);
     }
 
-out:
-    bdrv_dirty_iter_free(dbi);
-    return ret;
+    return 0;
 }
 
 /* init copy_bitmap from sync_bitmap */
-- 
2.9.5

From: John Snow <jsnow@redhat.com>

If users set an unreasonably low speed (like one byte per second), the
calculated delay may exceed many hours. While we like to punish users
for asking for stupid things, we do also like to allow users to correct
their wicked ways.

When a user provides a new speed, kick the job to allow it to recalculate
its delay.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20171213204611.26276-1-jsnow@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 blockjob.c | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/blockjob.c b/blockjob.c
index XXXXXXX..XXXXXXX 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -XXX,XX +XXX,XX @@ static void __attribute__((__constructor__)) block_job_init(void)
 
 static void block_job_event_cancelled(BlockJob *job);
 static void block_job_event_completed(BlockJob *job, const char *msg);
+static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job));
 
 /* Transactional group of block jobs */
 struct BlockJobTxn {
@@ -XXX,XX +XXX,XX @@ static void block_job_completed_txn_success(BlockJob *job)
     }
 }
 
+/* Assumes the block_job_mutex is held */
+static bool block_job_timer_pending(BlockJob *job)
+{
+    return timer_pending(&job->sleep_timer);
+}
+
 void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
 {
     Error *local_err = NULL;
+    int64_t old_speed = job->speed;
 
     if (!job->driver->set_speed) {
         error_setg(errp, QERR_UNSUPPORTED);
@@ -XXX,XX +XXX,XX @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
     }
 
     job->speed = speed;
+    if (speed <= old_speed) {
+        return;
+    }
+
+    /* kick only if a timer is pending */
+    block_job_enter_cond(job, block_job_timer_pending);
 }
 
 void block_job_complete(BlockJob *job, Error **errp)
@@ -XXX,XX +XXX,XX @@ void block_job_resume_all(void)
     }
 }
 
-void block_job_enter(BlockJob *job)
+/*
+ * Conditionally enter a block_job pending a call to fn() while
+ * under the block_job_lock critical section.
+ */
+static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job))
 {
     if (!block_job_started(job)) {
         return;
@@ -XXX,XX +XXX,XX @@ void block_job_enter(BlockJob *job)
         return;
     }
 
+    if (fn && !fn(job)) {
+        block_job_unlock();
+        return;
+    }
+
     assert(!job->deferred_to_main_loop);
     timer_del(&job->sleep_timer);
     job->busy = true;
@@ -XXX,XX +XXX,XX @@ void block_job_enter(BlockJob *job)
     aio_co_wake(job->co);
 }
 
+void block_job_enter(BlockJob *job)
+{
+    block_job_enter_cond(job, NULL);
+}
+
 bool block_job_is_cancelled(BlockJob *job)
 {
     return job->cancelled;
-- 
2.9.5

No functional changes, just whitespace manipulation.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/sheepdog.c | 164 +++++++++++++++++++++++++++----------------------------
 1 file changed, 82 insertions(+), 82 deletions(-)

diff --git a/block/sheepdog.c b/block/sheepdog.c
index XXXXXXX..XXXXXXX 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVSheepdogReopenState {
     int cache_flags;
 } BDRVSheepdogReopenState;
 
-static const char * sd_strerror(int err)
+static const char *sd_strerror(int err)
 {
     int i;
 
@@ -XXX,XX +XXX,XX @@ static QemuOptsList sd_create_opts = {
 };
 
 static BlockDriver bdrv_sheepdog = {
-    .format_name    = "sheepdog",
-    .protocol_name  = "sheepdog",
-    .instance_size  = sizeof(BDRVSheepdogState),
-    .bdrv_parse_filename    = sd_parse_filename,
-    .bdrv_file_open = sd_open,
-    .bdrv_reopen_prepare    = sd_reopen_prepare,
-    .bdrv_reopen_commit     = sd_reopen_commit,
-    .bdrv_reopen_abort      = sd_reopen_abort,
-    .bdrv_close     = sd_close,
-    .bdrv_create    = sd_create,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
-    .bdrv_getlength = sd_getlength,
+    .format_name                  = "sheepdog",
+    .protocol_name                = "sheepdog",
+    .instance_size                = sizeof(BDRVSheepdogState),
+    .bdrv_parse_filename          = sd_parse_filename,
+    .bdrv_file_open               = sd_open,
+    .bdrv_reopen_prepare          = sd_reopen_prepare,
+    .bdrv_reopen_commit           = sd_reopen_commit,
+    .bdrv_reopen_abort            = sd_reopen_abort,
+    .bdrv_close                   = sd_close,
+    .bdrv_create                  = sd_create,
+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+    .bdrv_getlength               = sd_getlength,
     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
-    .bdrv_truncate  = sd_truncate,
+    .bdrv_truncate                = sd_truncate,
 
-    .bdrv_co_readv  = sd_co_readv,
-    .bdrv_co_writev = sd_co_writev,
-    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
-    .bdrv_co_pdiscard = sd_co_pdiscard,
-    .bdrv_co_get_block_status = sd_co_get_block_status,
+    .bdrv_co_readv                = sd_co_readv,
+    .bdrv_co_writev               = sd_co_writev,
+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
+    .bdrv_co_pdiscard             = sd_co_pdiscard,
+    .bdrv_co_get_block_status     = sd_co_get_block_status,
 
-    .bdrv_snapshot_create   = sd_snapshot_create,
-    .bdrv_snapshot_goto     = sd_snapshot_goto,
-    .bdrv_snapshot_delete   = sd_snapshot_delete,
-    .bdrv_snapshot_list     = sd_snapshot_list,
+    .bdrv_snapshot_create         = sd_snapshot_create,
+    .bdrv_snapshot_goto           = sd_snapshot_goto,
+    .bdrv_snapshot_delete         = sd_snapshot_delete,
+    .bdrv_snapshot_list           = sd_snapshot_list,
 
-    .bdrv_save_vmstate  = sd_save_vmstate,
-    .bdrv_load_vmstate  = sd_load_vmstate,
+    .bdrv_save_vmstate            = sd_save_vmstate,
+    .bdrv_load_vmstate            = sd_load_vmstate,
 
-    .bdrv_detach_aio_context = sd_detach_aio_context,
-    .bdrv_attach_aio_context = sd_attach_aio_context,
+    .bdrv_detach_aio_context      = sd_detach_aio_context,
+    .bdrv_attach_aio_context      = sd_attach_aio_context,
 
-    .create_opts    = &sd_create_opts,
+    .create_opts                  = &sd_create_opts,
 };
 
 static BlockDriver bdrv_sheepdog_tcp = {
-    .format_name    = "sheepdog",
-    .protocol_name  = "sheepdog+tcp",
-    .instance_size  = sizeof(BDRVSheepdogState),
-    .bdrv_parse_filename    = sd_parse_filename,
-    .bdrv_file_open = sd_open,
-    .bdrv_reopen_prepare    = sd_reopen_prepare,
-    .bdrv_reopen_commit     = sd_reopen_commit,
-    .bdrv_reopen_abort      = sd_reopen_abort,
-    .bdrv_close     = sd_close,
-    .bdrv_create    = sd_create,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
-    .bdrv_getlength = sd_getlength,
+    .format_name                  = "sheepdog",
+    .protocol_name                = "sheepdog+tcp",
+    .instance_size                = sizeof(BDRVSheepdogState),
+    .bdrv_parse_filename          = sd_parse_filename,
+    .bdrv_file_open               = sd_open,
+    .bdrv_reopen_prepare          = sd_reopen_prepare,
+    .bdrv_reopen_commit           = sd_reopen_commit,
+    .bdrv_reopen_abort            = sd_reopen_abort,
+    .bdrv_close                   = sd_close,
+    .bdrv_create                  = sd_create,
+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+    .bdrv_getlength               = sd_getlength,
     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
-    .bdrv_truncate  = sd_truncate,
+    .bdrv_truncate                = sd_truncate,
 
-    .bdrv_co_readv  = sd_co_readv,
-    .bdrv_co_writev = sd_co_writev,
-    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
-    .bdrv_co_pdiscard = sd_co_pdiscard,
-    .bdrv_co_get_block_status = sd_co_get_block_status,
+    .bdrv_co_readv                = sd_co_readv,
+    .bdrv_co_writev               = sd_co_writev,
+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
+    .bdrv_co_pdiscard             = sd_co_pdiscard,
+    .bdrv_co_get_block_status     = sd_co_get_block_status,
 
-    .bdrv_snapshot_create   = sd_snapshot_create,
-    .bdrv_snapshot_goto     = sd_snapshot_goto,
-    .bdrv_snapshot_delete   = sd_snapshot_delete,
-    .bdrv_snapshot_list     = sd_snapshot_list,
+    .bdrv_snapshot_create         = sd_snapshot_create,
+    .bdrv_snapshot_goto           = sd_snapshot_goto,
+    .bdrv_snapshot_delete         = sd_snapshot_delete,
+    .bdrv_snapshot_list           = sd_snapshot_list,
 
-    .bdrv_save_vmstate  = sd_save_vmstate,
-    .bdrv_load_vmstate  = sd_load_vmstate,
+    .bdrv_save_vmstate            = sd_save_vmstate,
+    .bdrv_load_vmstate            = sd_load_vmstate,
 
-    .bdrv_detach_aio_context = sd_detach_aio_context,
-    .bdrv_attach_aio_context = sd_attach_aio_context,
+    .bdrv_detach_aio_context      = sd_detach_aio_context,
+    .bdrv_attach_aio_context      = sd_attach_aio_context,
 
-    .create_opts    = &sd_create_opts,
+    .create_opts                  = &sd_create_opts,
 };
 
 static BlockDriver bdrv_sheepdog_unix = {
-    .format_name    = "sheepdog",
-    .protocol_name  = "sheepdog+unix",
-    .instance_size  = sizeof(BDRVSheepdogState),
-    .bdrv_parse_filename    = sd_parse_filename,
-    .bdrv_file_open = sd_open,
-    .bdrv_reopen_prepare    = sd_reopen_prepare,
-    .bdrv_reopen_commit     = sd_reopen_commit,
-    .bdrv_reopen_abort      = sd_reopen_abort,
-    .bdrv_close     = sd_close,
-    .bdrv_create    = sd_create,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
-    .bdrv_getlength = sd_getlength,
+    .format_name                  = "sheepdog",
+    .protocol_name                = "sheepdog+unix",
+    .instance_size                = sizeof(BDRVSheepdogState),
+    .bdrv_parse_filename          = sd_parse_filename,
+    .bdrv_file_open               = sd_open,
+    .bdrv_reopen_prepare          = sd_reopen_prepare,
+    .bdrv_reopen_commit           = sd_reopen_commit,
+    .bdrv_reopen_abort            = sd_reopen_abort,
+    .bdrv_close                   = sd_close,
+    .bdrv_create                  = sd_create,
+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+    .bdrv_getlength               = sd_getlength,
     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
-    .bdrv_truncate  = sd_truncate,
+    .bdrv_truncate                = sd_truncate,
 
-    .bdrv_co_readv  = sd_co_readv,
-    .bdrv_co_writev = sd_co_writev,
-    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
-    .bdrv_co_pdiscard = sd_co_pdiscard,
-    .bdrv_co_get_block_status = sd_co_get_block_status,
+    .bdrv_co_readv                = sd_co_readv,
+    .bdrv_co_writev               = sd_co_writev,
+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
+    .bdrv_co_pdiscard             = sd_co_pdiscard,
+    .bdrv_co_get_block_status     = sd_co_get_block_status,
 
-    .bdrv_snapshot_create   = sd_snapshot_create,
-    .bdrv_snapshot_goto     = sd_snapshot_goto,
-    .bdrv_snapshot_delete   = sd_snapshot_delete,
-    .bdrv_snapshot_list     = sd_snapshot_list,
+    .bdrv_snapshot_create         = sd_snapshot_create,
+    .bdrv_snapshot_goto           = sd_snapshot_goto,
+    .bdrv_snapshot_delete         = sd_snapshot_delete,
+    .bdrv_snapshot_list           = sd_snapshot_list,
 
-    .bdrv_save_vmstate  = sd_save_vmstate,
-    .bdrv_load_vmstate  = sd_load_vmstate,
+    .bdrv_save_vmstate            = sd_save_vmstate,
+    .bdrv_load_vmstate            = sd_load_vmstate,
 
-    .bdrv_detach_aio_context = sd_detach_aio_context,
-    .bdrv_attach_aio_context = sd_attach_aio_context,
+    .bdrv_detach_aio_context      = sd_detach_aio_context,
+    .bdrv_attach_aio_context      = sd_attach_aio_context,
 
-    .create_opts    = &sd_create_opts,
+    .create_opts                  = &sd_create_opts,
 };
 
 static void bdrv_sheepdog_init(void)
-- 
2.9.5

If curl_global_init() fails, per the documentation no other curl
functions may be called, so make sure to check the return value.

Also, some minor changes to the initialization latch variable 'inited':

- Make it static in the file, for clarity
- Change the name for clarity
- Make it a bool

Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/curl.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
 
 struct BDRVCURLState;
 
+static bool libcurl_initialized;
+
 typedef struct CURLAIOCB {
     Coroutine *co;
     QEMUIOVector *qiov;
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
     double d;
     const char *secretid;
     const char *protocol_delimiter;
+    int ret;
 
-    static int inited = 0;
 
     if (flags & BDRV_O_RDWR) {
         error_setg(errp, "curl block device does not support writes");
         return -EROFS;
     }
 
+    if (!libcurl_initialized) {
+        ret = curl_global_init(CURL_GLOBAL_ALL);
+        if (ret) {
+            error_setg(errp, "libcurl initialization failed with %d", ret);
+            return -EIO;
+        }
+        libcurl_initialized = true;
+    }
+
     qemu_mutex_init(&s->mutex);
     opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
     qemu_opts_absorb_qdict(opts, options, &local_err);
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
         }
     }
 
-    if (!inited) {
-        curl_global_init(CURL_GLOBAL_ALL);
-        inited = 1;
-    }
-
     DPRINTF("CURL: Opening %s\n", file);
     QSIMPLEQ_INIT(&s->free_state_waitq);
     s->aio_context = bdrv_get_aio_context(bs);
-- 
2.9.5

The following changes since commit 812b835fb4d23dd108b2f9802158472d50b73579:

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-05-07' into staging (2019-05-09 16:31:12 +0100)

are available in the Git repository at:

https://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to e84125761f78919fe63616d9888ea45e72dc956f:

docs: add Security chapter to the documentation (2019-05-10 10:53:52 +0100)

----------------------------------------------------------------
Pull request

----------------------------------------------------------------

Andrey Shinkevich (1):
  block/io.c: fix for the allocation failure

Jules Irenge (3):
  util/readline: add a space to fix errors by checkpatch tool
  util: readline: replace tab indent by four spaces to fix checkpatch
    errors
  util/readline: Add braces to fix checkpatch errors

Nikita Alekseev (1):
  block: Add coroutine_fn to bdrv_check_co_entry

Paolo Bonzini (1):
  aio-posix: ensure poll mode is left when aio_notify is called

Stefan Hajnoczi (2):
  docs: add Secure Coding Practices to developer docs
  docs: add Security chapter to the documentation

-- 
2.21.0

From: Jules Irenge <jbi.octave@gmail.com>

util/readline: add a space to fix errors reported by checkpatch.pl tool
"ERROR: space required before the open parenthesis"
"ERROR: space required after that ..."
within "util/redline.c" file

Signed-off-by: Jules Irenge <jbi.octave@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190401024406.10819-2-jbi.octave@gmail.com
Message-Id: <20190401024406.10819-2-jbi.octave@gmail.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/readline.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/util/readline.c b/util/readline.c
index XXXXXXX..XXXXXXX 100644
--- a/util/readline.c
+++ b/util/readline.c
@@ -XXX,XX +XXX,XX @@ static void readline_update(ReadLineState *rs)
 
     if (rs->cmd_buf_size != rs->last_cmd_buf_size ||
         memcmp(rs->cmd_buf, rs->last_cmd_buf, rs->cmd_buf_size) != 0) {
-        for(i = 0; i < rs->last_cmd_buf_index; i++) {
+        for (i = 0; i < rs->last_cmd_buf_index; i++) {
             rs->printf_func(rs->opaque, "\033[D");
         }
         rs->cmd_buf[rs->cmd_buf_size] = '\0';
         if (rs->read_password) {
             len = strlen(rs->cmd_buf);
-            for(i = 0; i < len; i++)
+            for (i = 0; i < len; i++)
                 rs->printf_func(rs->opaque, "*");
         } else {
             rs->printf_func(rs->opaque, "%s", rs->cmd_buf);
@@ -XXX,XX +XXX,XX @@ static void readline_update(ReadLineState *rs)
     if (rs->cmd_buf_index != rs->last_cmd_buf_index) {
         delta = rs->cmd_buf_index - rs->last_cmd_buf_index;
         if (delta > 0) {
-            for(i = 0;i < delta; i++) {
+            for (i = 0; i < delta; i++) {
                 rs->printf_func(rs->opaque, "\033[C");
             }
         } else {
             delta = -delta;
-            for(i = 0;i < delta; i++) {
+            for (i = 0; i < delta; i++) {
                 rs->printf_func(rs->opaque, "\033[D");
             }
         }
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
         return;
     if (rs->nb_completions == 1) {
         len = strlen(rs->completions[0]);
-        for(i = rs->completion_index; i < len; i++) {
+        for (i = rs->completion_index; i < len; i++) {
             readline_insert_char(rs, rs->completions[0][i]);
         }
         /* extra space for next argument. XXX: make it more generic */
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
               completion_comp);
         rs->printf_func(rs->opaque, "\n");
         max_width = 0;
-        max_prefix = 0;	
-        for(i = 0; i < rs->nb_completions; i++) {
+        max_prefix = 0;
+        for (i = 0; i < rs->nb_completions; i++) {
             len = strlen(rs->completions[i]);
-            if (i==0) {
+            if (i == 0) {
                 max_prefix = len;
             } else {
                 if (len < max_prefix)
                     max_prefix = len;
-                for(j=0; j<max_prefix; j++) {
+                for (j = 0; j < max_prefix; j++) {
                     if (rs->completions[i][j] != rs->completions[0][j])
                         max_prefix = j;
                 }
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
             if (len > max_width)
                 max_width = len;
         }
-        if (max_prefix > 0) 
-            for(i = rs->completion_index; i < max_prefix; i++) {
+        if (max_prefix > 0)
+            for (i = rs->completion_index; i < max_prefix; i++) {
                 readline_insert_char(rs, rs->completions[0][i]);
             }
         max_width += 2;
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
             max_width = 80;
         nb_cols = 80 / max_width;
         j = 0;
-        for(i = 0; i < rs->nb_completions; i++) {
+        for (i = 0; i < rs->nb_completions; i++) {
             rs->printf_func(rs->opaque, "%-*s", max_width, rs->completions[i]);
             if (++j == nb_cols || i == (rs->nb_completions - 1)) {
                 rs->printf_func(rs->opaque, "\n");
@@ -XXX,XX +XXX,XX @@ static void readline_clear_screen(ReadLineState *rs)
 /* return true if command handled */
 void readline_handle_byte(ReadLineState *rs, int ch)
 {
-    switch(rs->esc_state) {
+    switch (rs->esc_state) {
     case IS_NORM:
-        switch(ch) {
+        switch (ch) {
         case 1:
             readline_bol(rs);
             break;
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
         }
         break;
     case IS_CSI:
-        switch(ch) {
+        switch (ch) {
 	case 'A':
 	case 'F':
 	    readline_up_char(rs);
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
             rs->esc_param = rs->esc_param * 10 + (ch - '0');
             goto the_end;
         case '~':
-            switch(rs->esc_param) {
+            switch (rs->esc_param) {
             case 1:
                 readline_bol(rs);
                 break;
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
     the_end:
         break;
     case IS_SS3:
-        switch(ch) {
+        switch (ch) {
         case 'F':
             readline_eol(rs);
             break;
-- 
2.21.0

From: Jules Irenge <jbi.octave@gmail.com>

Replace tab indent by four spaces to fix errors issued by checkpatch.pl tool
"ERROR: code indent should never use tabs" within "util/readline.c" file.

Signed-off-by: Jules Irenge <jbi.octave@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190401024406.10819-3-jbi.octave@gmail.com
Message-Id: <20190401024406.10819-3-jbi.octave@gmail.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/readline.c | 98 ++++++++++++++++++++++++-------------------------
 1 file changed, 49 insertions(+), 49 deletions(-)

diff --git a/util/readline.c b/util/readline.c
index XXXXXXX..XXXXXXX 100644
--- a/util/readline.c
+++ b/util/readline.c
@@ -XXX,XX +XXX,XX @@ static void readline_up_char(ReadLineState *rs)
     int idx;
 
     if (rs->hist_entry == 0)
-	return;
+        return;
     if (rs->hist_entry == -1) {
-	/* Find latest entry */
-	for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
-	    if (rs->history[idx] == NULL)
-		break;
-	}
-	rs->hist_entry = idx;
+        /* Find latest entry */
+        for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
+            if (rs->history[idx] == NULL)
+                break;
+        }
+        rs->hist_entry = idx;
     }
     rs->hist_entry--;
     if (rs->hist_entry >= 0) {
-	pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
+        pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
                 rs->history[rs->hist_entry]);
-	rs->cmd_buf_index = rs->cmd_buf_size = strlen(rs->cmd_buf);
+        rs->cmd_buf_index = rs->cmd_buf_size = strlen(rs->cmd_buf);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ static void readline_down_char(ReadLineState *rs)
         return;
     if (rs->hist_entry < READLINE_MAX_CMDS - 1 &&
         rs->history[++rs->hist_entry] != NULL) {
-	pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
+        pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
                 rs->history[rs->hist_entry]);
     } else {
         rs->cmd_buf[0] = 0;
-	rs->hist_entry = -1;
+        rs->hist_entry = -1;
     }
     rs->cmd_buf_index = rs->cmd_buf_size = strlen(rs->cmd_buf);
 }
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
     int idx;
 
     if (cmdline[0] == '\0')
-	return;
+        return;
     new_entry = NULL;
     if (rs->hist_entry != -1) {
-	/* We were editing an existing history entry: replace it */
-	hist_entry = rs->history[rs->hist_entry];
-	idx = rs->hist_entry;
-	if (strcmp(hist_entry, cmdline) == 0) {
-	    goto same_entry;
-	}
+        /* We were editing an existing history entry: replace it */
+        hist_entry = rs->history[rs->hist_entry];
+        idx = rs->hist_entry;
+        if (strcmp(hist_entry, cmdline) == 0) {
+            goto same_entry;
+        }
     }
     /* Search cmdline in history buffers */
     for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
-	hist_entry = rs->history[idx];
-	if (hist_entry == NULL)
-	    break;
-	if (strcmp(hist_entry, cmdline) == 0) {
-	same_entry:
-	    new_entry = hist_entry;
-	    /* Put this entry at the end of history */
-	    memmove(&rs->history[idx], &rs->history[idx + 1],
-		    (READLINE_MAX_CMDS - (idx + 1)) * sizeof(char *));
-	    rs->history[READLINE_MAX_CMDS - 1] = NULL;
-	    for (; idx < READLINE_MAX_CMDS; idx++) {
-		if (rs->history[idx] == NULL)
-		    break;
-	    }
-	    break;
-	}
+        hist_entry = rs->history[idx];
+        if (hist_entry == NULL)
+            break;
+        if (strcmp(hist_entry, cmdline) == 0) {
+        same_entry:
+            new_entry = hist_entry;
+            /* Put this entry at the end of history */
+            memmove(&rs->history[idx], &rs->history[idx + 1],
+                    (READLINE_MAX_CMDS - (idx + 1)) * sizeof(char *));
+            rs->history[READLINE_MAX_CMDS - 1] = NULL;
+            for (; idx < READLINE_MAX_CMDS; idx++) {
+                if (rs->history[idx] == NULL)
+                    break;
+            }
+            break;
+        }
     }
     if (idx == READLINE_MAX_CMDS) {
-	/* Need to get one free slot */
+        /* Need to get one free slot */
         g_free(rs->history[0]);
-	memmove(rs->history, &rs->history[1],
-	        (READLINE_MAX_CMDS - 1) * sizeof(char *));
-	rs->history[READLINE_MAX_CMDS - 1] = NULL;
-	idx = READLINE_MAX_CMDS - 1;
+        memmove(rs->history, &rs->history[1],
+                (READLINE_MAX_CMDS - 1) * sizeof(char *));
+        rs->history[READLINE_MAX_CMDS - 1] = NULL;
+        idx = READLINE_MAX_CMDS - 1;
     }
     if (new_entry == NULL)
         new_entry = g_strdup(cmdline);
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
         case 8:
             readline_backspace(rs);
             break;
-	case 155:
+        case 155:
             rs->esc_state = IS_CSI;
-	    break;
+            break;
         default:
             if (ch >= 32) {
                 readline_insert_char(rs, ch);
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
         break;
     case IS_CSI:
         switch (ch) {
-	case 'A':
-	case 'F':
-	    readline_up_char(rs);
-	    break;
-	case 'B':
-	case 'E':
-	    readline_down_char(rs);
-	    break;
+        case 'A':
+        case 'F':
+            readline_up_char(rs);
+            break;
+        case 'B':
+        case 'E':
+            readline_down_char(rs);
+            break;
         case 'D':
             readline_backward_char(rs);
             break;
-- 
2.21.0

From: Jules Irenge <jbi.octave@gmail.com>

Add braces to fix errors issued by checkpatch.pl tool
"ERROR: braces {} are necessary for all arms of this statement"
Within "util/readline.c" file
Message-Id: <20190330112142.14082-1-jbi.octave@gmail.com>

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/readline.c | 50 ++++++++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 17 deletions(-)

diff --git a/util/readline.c b/util/readline.c
index XXXXXXX..XXXXXXX 100644
--- a/util/readline.c
+++ b/util/readline.c
@@ -XXX,XX +XXX,XX @@ static void readline_update(ReadLineState *rs)
         rs->cmd_buf[rs->cmd_buf_size] = '\0';
         if (rs->read_password) {
             len = strlen(rs->cmd_buf);
-            for (i = 0; i < len; i++)
+            for (i = 0; i < len; i++) {
                 rs->printf_func(rs->opaque, "*");
+            }
         } else {
             rs->printf_func(rs->opaque, "%s", rs->cmd_buf);
         }
@@ -XXX,XX +XXX,XX @@ static void readline_up_char(ReadLineState *rs)
 {
     int idx;
 
-    if (rs->hist_entry == 0)
+    if (rs->hist_entry == 0) {
         return;
+    }
     if (rs->hist_entry == -1) {
         /* Find latest entry */
         for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
-            if (rs->history[idx] == NULL)
+            if (rs->history[idx] == NULL) {
                 break;
+            }
         }
         rs->hist_entry = idx;
     }
@@ -XXX,XX +XXX,XX @@ static void readline_up_char(ReadLineState *rs)
 
 static void readline_down_char(ReadLineState *rs)
 {
-    if (rs->hist_entry == -1)
+    if (rs->hist_entry == -1) {
         return;
+    }
     if (rs->hist_entry < READLINE_MAX_CMDS - 1 &&
         rs->history[++rs->hist_entry] != NULL) {
         pstrcpy(rs->cmd_buf, sizeof(rs->cmd_buf),
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
     char *hist_entry, *new_entry;
     int idx;
 
-    if (cmdline[0] == '\0')
+    if (cmdline[0] == '\0') {
         return;
+    }
     new_entry = NULL;
     if (rs->hist_entry != -1) {
         /* We were editing an existing history entry: replace it */
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
     /* Search cmdline in history buffers */
     for (idx = 0; idx < READLINE_MAX_CMDS; idx++) {
         hist_entry = rs->history[idx];
-        if (hist_entry == NULL)
+        if (hist_entry == NULL) {
             break;
+        }
         if (strcmp(hist_entry, cmdline) == 0) {
         same_entry:
             new_entry = hist_entry;
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
                     (READLINE_MAX_CMDS - (idx + 1)) * sizeof(char *));
             rs->history[READLINE_MAX_CMDS - 1] = NULL;
             for (; idx < READLINE_MAX_CMDS; idx++) {
-                if (rs->history[idx] == NULL)
+                if (rs->history[idx] == NULL) {
                     break;
+                }
             }
             break;
         }
@@ -XXX,XX +XXX,XX @@ static void readline_hist_add(ReadLineState *rs, const char *cmdline)
         rs->history[READLINE_MAX_CMDS - 1] = NULL;
         idx = READLINE_MAX_CMDS - 1;
     }
-    if (new_entry == NULL)
+    if (new_entry == NULL) {
         new_entry = g_strdup(cmdline);
+    }
     rs->history[idx] = new_entry;
     rs->hist_entry = -1;
 }
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
     g_free(cmdline);
 
     /* no completion found */
-    if (rs->nb_completions <= 0)
+    if (rs->nb_completions <= 0) {
         return;
+    }
     if (rs->nb_completions == 1) {
         len = strlen(rs->completions[0]);
         for (i = rs->completion_index; i < len; i++) {
             readline_insert_char(rs, rs->completions[0][i]);
         }
         /* extra space for next argument. XXX: make it more generic */
-        if (len > 0 && rs->completions[0][len - 1] != '/')
+        if (len > 0 && rs->completions[0][len - 1] != '/') {
             readline_insert_char(rs, ' ');
+        }
     } else {
         qsort(rs->completions, rs->nb_completions, sizeof(char *),
               completion_comp);
@@ -XXX,XX +XXX,XX @@ static void readline_completion(ReadLineState *rs)
             if (i == 0) {
                 max_prefix = len;
             } else {
-                if (len < max_prefix)
+                if (len < max_prefix) {
                     max_prefix = len;
+                }
                 for (j = 0; j < max_prefix; j++) {
-                    if (rs->completions[i][j] != rs->completions[0][j])
+                    if (rs->completions[i][j] != rs->completions[0][j]) {
                         max_prefix = j;
+                    }
                 }
             }
-            if (len > max_width)
+            if (len > max_width) {
                 max_width = len;
+            }
         }
         if (max_prefix > 0)
             for (i = rs->completion_index; i < max_prefix; i++) {
                 readline_insert_char(rs, rs->completions[0][i]);
             }
         max_width += 2;
-        if (max_width < 10)
+        if (max_width < 10) {
             max_width = 10;
-        else if (max_width > 80)
+        } else if (max_width > 80) {
             max_width = 80;
+        }
         nb_cols = 80 / max_width;
         j = 0;
         for (i = 0; i < rs->nb_completions; i++) {
@@ -XXX,XX +XXX,XX @@ void readline_handle_byte(ReadLineState *rs, int ch)
         case 10:
         case 13:
             rs->cmd_buf[rs->cmd_buf_size] = '\0';
-            if (!rs->read_password)
+            if (!rs->read_password) {
                 readline_hist_add(rs, rs->cmd_buf);
+            }
             rs->printf_func(rs->opaque, "\n");
             rs->cmd_buf_index = 0;
             rs->cmd_buf_size = 0;
@@ -XXX,XX +XXX,XX @@ void readline_restart(ReadLineState *rs)
 
 const char *readline_get_history(ReadLineState *rs, unsigned int index)
 {
-    if (index >= READLINE_MAX_CMDS)
+    if (index >= READLINE_MAX_CMDS) {
         return NULL;
+    }
     return rs->history[index];
 }
 
-- 
2.21.0

From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>

On a file system used by the customer, fallocate() returns an error
if the block is not properly aligned. So, bdrv_co_pwrite_zeroes()
fails. We can handle that case the same way as it is done for the
unsupported cases, namely, call to bdrv_driver_pwritev() that writes
zeroes to an image for the unaligned chunk of the block.

Suggested-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 1554474244-553661-1-git-send-email-andrey.shinkevich@virtuozzo.com
Message-Id: <1554474244-553661-1-git-send-email-andrey.shinkevich@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
             assert(!bs->supported_zero_flags);
         }
 
-        if (ret == -ENOTSUP && !(flags & BDRV_REQ_NO_FALLBACK)) {
+        if (ret < 0 && !(flags & BDRV_REQ_NO_FALLBACK)) {
             /* Fall back to bounce buffer if write zeroes is unsupported */
             BdrvRequestFlags write_flags = flags & ~BDRV_REQ_ZERO_WRITE;
 
-- 
2.21.0

From: Paolo Bonzini <pbonzini@redhat.com>

With aio=thread, adaptive polling makes latency worse rather than
better, because it delays the execution of the ThreadPool's
completion bottom half.

event_notifier_poll() does run while polling, detecting that
a bottom half was scheduled by a worker thread, but because
ctx->notifier is explicitly ignored in run_poll_handlers_once(),
scheduling the BH does not count as making progress and
run_poll_handlers() keeps running.  Fix this by recomputing
the deadline after *timeout could have changed.

With this change, ThreadPool still cannot participate in polling
but at least it does not suffer from extra latency.

Reported-by: Sergio Lopez <slp@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20190409122823.12416-1-pbonzini@redhat.com
Cc: Stefan Hajnoczi <stefanha@gmail.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: qemu-block@nongnu.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1553692145-86728-1-git-send-email-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190409122823.12416-1-pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/aio-posix.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/util/aio-posix.c b/util/aio-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -XXX,XX +XXX,XX @@ static bool run_poll_handlers_once(AioContext *ctx, int64_t *timeout)
         if (!node->deleted && node->io_poll &&
             aio_node_check(ctx, node->is_external) &&
             node->io_poll(node->opaque)) {
+            /*
+             * Polling was successful, exit try_poll_mode immediately
+             * to adjust the next polling time.
+             */
             *timeout = 0;
             if (node->opaque != &ctx->notifier) {
                 progress = true;
@@ -XXX,XX +XXX,XX @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns, int64_t *timeout)
     do {
         progress = run_poll_handlers_once(ctx, timeout);
         elapsed_time = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start_time;
-    } while (!progress && elapsed_time < max_ns
-             && !atomic_read(&ctx->poll_disable_cnt));
+        max_ns = qemu_soonest_timeout(*timeout, max_ns);
+        assert(!(max_ns && progress));
+    } while (elapsed_time < max_ns && !atomic_read(&ctx->poll_disable_cnt));
 
     /* If time has passed with no successful polling, adjust *timeout to
      * keep the same ending time.
@@ -XXX,XX +XXX,XX @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns, int64_t *timeout)
  */
 static bool try_poll_mode(AioContext *ctx, int64_t *timeout)
 {
-    /* See qemu_soonest_timeout() uint64_t hack */
-    int64_t max_ns = MIN((uint64_t)*timeout, (uint64_t)ctx->poll_ns);
+    int64_t max_ns = qemu_soonest_timeout(*timeout, ctx->poll_ns);
 
     if (max_ns && !atomic_read(&ctx->poll_disable_cnt)) {
         poll_set_started(ctx, true);
-- 
2.21.0

At KVM Forum 2018 I gave a presentation on security in QEMU:
https://www.youtube.com/watch?v=YAdRf_hwxU8 (video)
https://vmsplice.net/~stefan/stefanha-kvm-forum-2018.pdf (slides)

This patch adds a guide to secure coding practices.  This document
covers things that developers should know about security in QEMU.  It is
just a starting point that we can expand on later.  I hope it will be
useful as a resource for new contributors and will save code reviewers
from explaining the same concepts many times.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Li Qiang <liq3ea@gmail.com>
Message-id: 20190509121820.16294-2-stefanha@redhat.com
Message-Id: <20190509121820.16294-2-stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/devel/index.rst                   |   1 +
 docs/devel/secure-coding-practices.rst | 106 +++++++++++++++++++++++++
 2 files changed, 107 insertions(+)
 create mode 100644 docs/devel/secure-coding-practices.rst

diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index XXXXXXX..XXXXXXX 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -XXX,XX +XXX,XX @@ Contents:
    stable-process
    testing
    decodetree
+   secure-coding-practices
diff --git a/docs/devel/secure-coding-practices.rst b/docs/devel/secure-coding-practices.rst
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/docs/devel/secure-coding-practices.rst
@@ -XXX,XX +XXX,XX @@
+=======================
+Secure Coding Practices
+=======================
+This document covers topics that both developers and security researchers must
+be aware of so that they can develop safe code and audit existing code
+properly.
+
+Reporting Security Bugs
+-----------------------
+For details on how to report security bugs or ask questions about potential
+security bugs, see the `Security Process wiki page
+<https://wiki.qemu.org/SecurityProcess>`_.
+
+General Secure C Coding Practices
+---------------------------------
+Most CVEs (security bugs) reported against QEMU are not specific to
+virtualization or emulation.  They are simply C programming bugs.  Therefore
+it's critical to be aware of common classes of security bugs.
+
+There is a wide selection of resources available covering secure C coding.  For
+example, the `CERT C Coding Standard
+<https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Coding+Standard>`_
+covers the most important classes of security bugs.
+
+Instead of describing them in detail here, only the names of the most important
+classes of security bugs are mentioned:
+
+* Buffer overflows
+* Use-after-free and double-free
+* Integer overflows
+* Format string vulnerabilities
+
+Some of these classes of bugs can be detected by analyzers.  Static analysis is
+performed regularly by Coverity and the most obvious of these bugs are even
+reported by compilers.  Dynamic analysis is possible with valgrind, tsan, and
+asan.
+
+Input Validation
+----------------
+Inputs from the guest or external sources (e.g. network, files) cannot be
+trusted and may be invalid.  Inputs must be checked before using them in a way
+that could crash the program, expose host memory to the guest, or otherwise be
+exploitable by an attacker.
+
+The most sensitive attack surface is device emulation.  All hardware register
+accesses and data read from guest memory must be validated.  A typical example
+is a device that contains multiple units that are selectable by the guest via
+an index register::
+
+  typedef struct {
+      ProcessingUnit unit[2];
+      ...
+  } MyDeviceState;
+
+  static void mydev_writel(void *opaque, uint32_t addr, uint32_t val)
+  {
+      MyDeviceState *mydev = opaque;
+      ProcessingUnit *unit;
+
+      switch (addr) {
+      case MYDEV_SELECT_UNIT:
+          unit = &mydev->unit[val];   <-- this input wasn't validated!
+          ...
+      }
+  }
+
+If ``val`` is not in range [0, 1] then an out-of-bounds memory access will take
+place when ``unit`` is dereferenced.  The code must check that ``val`` is 0 or
+1 and handle the case where it is invalid.
+
+Unexpected Device Accesses
+--------------------------
+The guest may access device registers in unusual orders or at unexpected
+moments.  Device emulation code must not assume that the guest follows the
+typical "theory of operation" presented in driver writer manuals.  The guest
+may make nonsense accesses to device registers such as starting operations
+before the device has been fully initialized.
+
+A related issue is that device emulation code must be prepared for unexpected
+device register accesses while asynchronous operations are in progress.  A
+well-behaved guest might wait for a completion interrupt before accessing
+certain device registers.  Device emulation code must handle the case where the
+guest overwrites registers or submits further requests before an ongoing
+request completes.  Unexpected accesses must not cause memory corruption or
+leaks in QEMU.
+
+Invalid device register accesses can be reported with
+``qemu_log_mask(LOG_GUEST_ERROR, ...)``.  The ``-d guest_errors`` command-line
+option enables these log messages.
+
+Live Migration
+--------------
+Device state can be saved to disk image files and shared with other users.
+Live migration code must validate inputs when loading device state so an
+attacker cannot gain control by crafting invalid device states.  Device state
+is therefore considered untrusted even though it is typically generated by QEMU
+itself.
+
+Guest Memory Access Races
+-------------------------
+Guests with multiple vCPUs may modify guest RAM while device emulation code is
+running.  Device emulation code must copy in descriptors and other guest RAM
+structures and only process the local copy.  This prevents
+time-of-check-to-time-of-use (TOCTOU) race conditions that could cause QEMU to
+crash when a vCPU thread modifies guest RAM while device emulation is
+processing it.
-- 
2.21.0

This new chapter in the QEMU documentation covers the security
requirements that QEMU is designed to meet and principles for securely
deploying QEMU.

It is just a starting point that can be extended in the future with more
information.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Li Qiang <liq3ea@gmail.com>
Message-id: 20190509121820.16294-3-stefanha@redhat.com
Message-Id: <20190509121820.16294-3-stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 Makefile           |   2 +-
 docs/security.texi | 131 +++++++++++++++++++++++++++++++++++++++++++++
 qemu-doc.texi      |   3 ++
 3 files changed, 135 insertions(+), 1 deletion(-)
 create mode 100644 docs/security.texi

diff --git a/Makefile b/Makefile
index XXXXXXX..XXXXXXX 100644
--- a/Makefile
+++ b/Makefile
@@ -XXX,XX +XXX,XX @@ qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
 	qemu-img.texi qemu-nbd.texi qemu-options.texi qemu-option-trace.texi \
 	qemu-deprecated.texi qemu-monitor.texi qemu-img-cmds.texi qemu-ga.texi \
 	qemu-monitor-info.texi docs/qemu-block-drivers.texi \
-	docs/qemu-cpu-models.texi
+	docs/qemu-cpu-models.texi docs/security.texi
 
 docs/interop/qemu-ga-ref.dvi docs/interop/qemu-ga-ref.html \
     docs/interop/qemu-ga-ref.info docs/interop/qemu-ga-ref.pdf \
diff --git a/docs/security.texi b/docs/security.texi
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/docs/security.texi
@@ -XXX,XX +XXX,XX @@
+@node Security
+@chapter Security
+
+@section Overview
+
+This chapter explains the security requirements that QEMU is designed to meet
+and principles for securely deploying QEMU.
+
+@section Security Requirements
+
+QEMU supports many different use cases, some of which have stricter security
+requirements than others.  The community has agreed on the overall security
+requirements that users may depend on.  These requirements define what is
+considered supported from a security perspective.
+
+@subsection Virtualization Use Case
+
+The virtualization use case covers cloud and virtual private server (VPS)
+hosting, as well as traditional data center and desktop virtualization.  These
+use cases rely on hardware virtualization extensions to execute guest code
+safely on the physical CPU at close-to-native speed.
+
+The following entities are untrusted, meaning that they may be buggy or
+malicious:
+
+@itemize
+@item Guest
+@item User-facing interfaces (e.g. VNC, SPICE, WebSocket)
+@item Network protocols (e.g. NBD, live migration)
+@item User-supplied files (e.g. disk images, kernels, device trees)
+@item Passthrough devices (e.g. PCI, USB)
+@end itemize
+
+Bugs affecting these entities are evaluated on whether they can cause damage in
+real-world use cases and treated as security bugs if this is the case.
+
+@subsection Non-virtualization Use Case
+
+The non-virtualization use case covers emulation using the Tiny Code Generator
+(TCG).  In principle the TCG and device emulation code used in conjunction with
+the non-virtualization use case should meet the same security requirements as
+the virtualization use case.  However, for historical reasons much of the
+non-virtualization use case code was not written with these security
+requirements in mind.
+
+Bugs affecting the non-virtualization use case are not considered security
+bugs at this time.  Users with non-virtualization use cases must not rely on
+QEMU to provide guest isolation or any security guarantees.
+
+@section Architecture
+
+This section describes the design principles that ensure the security
+requirements are met.
+
+@subsection Guest Isolation
+
+Guest isolation is the confinement of guest code to the virtual machine.  When
+guest code gains control of execution on the host this is called escaping the
+virtual machine.  Isolation also includes resource limits such as throttling of
+CPU, memory, disk, or network.  Guests must be unable to exceed their resource
+limits.
+
+QEMU presents an attack surface to the guest in the form of emulated devices.
+The guest must not be able to gain control of QEMU.  Bugs in emulated devices
+could allow malicious guests to gain code execution in QEMU.  At this point the
+guest has escaped the virtual machine and is able to act in the context of the
+QEMU process on the host.
+
+Guests often interact with other guests and share resources with them.  A
+malicious guest must not gain control of other guests or access their data.
+Disk image files and network traffic must be protected from other guests unless
+explicitly shared between them by the user.
+
+@subsection Principle of Least Privilege
+
+The principle of least privilege states that each component only has access to
+the privileges necessary for its function.  In the case of QEMU this means that
+each process only has access to resources belonging to the guest.
+
+The QEMU process should not have access to any resources that are inaccessible
+to the guest.  This way the guest does not gain anything by escaping into the
+QEMU process since it already has access to those same resources from within
+the guest.
+
+Following the principle of least privilege immediately fulfills guest isolation
+requirements.  For example, guest A only has access to its own disk image file
+@code{a.img} and not guest B's disk image file @code{b.img}.
+
+In reality certain resources are inaccessible to the guest but must be
+available to QEMU to perform its function.  For example, host system calls are
+necessary for QEMU but are not exposed to guests.  A guest that escapes into
+the QEMU process can then begin invoking host system calls.
+
+New features must be designed to follow the principle of least privilege.
+Should this not be possible for technical reasons, the security risk must be
+clearly documented so users are aware of the trade-off of enabling the feature.
+
+@subsection Isolation mechanisms
+
+Several isolation mechanisms are available to realize this architecture of
+guest isolation and the principle of least privilege.  With the exception of
+Linux seccomp, these mechanisms are all deployed by management tools that
+launch QEMU, such as libvirt.  They are also platform-specific so they are only
+described briefly for Linux here.
+
+The fundamental isolation mechanism is that QEMU processes must run as
+unprivileged users.  Sometimes it seems more convenient to launch QEMU as
+root to give it access to host devices (e.g. @code{/dev/net/tun}) but this poses a
+huge security risk.  File descriptor passing can be used to give an otherwise
+unprivileged QEMU process access to host devices without running QEMU as root.
+It is also possible to launch QEMU as a non-root user and configure UNIX groups
+for access to @code{/dev/kvm}, @code{/dev/net/tun}, and other device nodes.
+Some Linux distros already ship with UNIX groups for these devices by default.
+
+@itemize
+@item SELinux and AppArmor make it possible to confine processes beyond the
+traditional UNIX process and file permissions model.  They restrict the QEMU
+process from accessing processes and files on the host system that are not
+needed by QEMU.
+
+@item Resource limits and cgroup controllers provide throughput and utilization
+limits on key resources such as CPU time, memory, and I/O bandwidth.
+
+@item Linux namespaces can be used to make process, file system, and other system
+resources unavailable to QEMU.  A namespaced QEMU process is restricted to only
+those resources that were granted to it.
+
+@item Linux seccomp is available via the QEMU @option{--sandbox} option.  It disables
+system calls that are not needed by QEMU, thereby reducing the host kernel
+attack surface.
+@end itemize
diff --git a/qemu-doc.texi b/qemu-doc.texi
index XXXXXXX..XXXXXXX 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -XXX,XX +XXX,XX @@
 * QEMU Guest Agent::
 * QEMU User space emulator::
 * System requirements::
+* Security::
 * Implementation notes::
 * Deprecated features::
 * Supported build platforms::
@@ -XXX,XX +XXX,XX @@ added with Linux 4.5 which is supported by the major distros. And even
 if RHEL7 has kernel 3.10, KVM there has the required functionality there
 to make it close to a 4.5 or newer kernel.
 
+@include docs/security.texi
+
 @include qemu-tech.texi
 
 @include qemu-deprecated.texi
-- 
2.21.0