Series comparison

-[Qemu-devel] [PULL for-2.10 00/15] Block patches
+[PULL 00/20] Block patches
-The following changes since commit 248b23735645f7cbb503d9be6f5bf825f2a603ab:
+The following changes since commit c1eb2ddf0f8075faddc5f7c3d39feae3e8e9d6b4:
-  Update version for v2.10.0-rc4 release (2017-08-24 17:34:26 +0100)
+  Update version for v8.0.0 release (2023-04-19 17:27:13 +0100)
-are available in the git repository at:
+are available in the Git repository at:
-  git://github.com/stefanha/qemu.git tags/block-pull-request
+  https://gitlab.com/stefanha/qemu.git tags/block-pull-request
-for you to fetch changes up to 3e4c705212abfe8c9882a00beb2d1466a8a53cec:
+for you to fetch changes up to 36e5e9b22abe56aa00ca067851555ad8127a7966:
-  qcow2: allocate cluster_cache/cluster_data on demand (2017-08-30 18:02:10 +0100)
+  tracing: install trace events file only if necessary (2023-04-20 07:39:43 -0400)
 ----------------------------------------------------------------
 Pull request
 Sam Li's zoned storage work and fixes I collected during the 8.0 freeze.
 ----------------------------------------------------------------
-----------------------------------------------------------------
+Carlos Santos (1):
   tracing: install trace events file only if necessary
-Alberto Garcia (8):
+Philippe Mathieu-Daudé (1):
-  throttle: Fix wrong variable name in the header documentation
+  block/dmg: Declare a type definition for DMG uncompress function
   throttle: Update the throttle_fix_bucket() documentation
   throttle: Make throttle_is_valid() a bit less verbose
   throttle: Remove throttle_fix_bucket() / throttle_unfix_bucket()
   throttle: Make LeakyBucket.avg and LeakyBucket.max integer types
   throttle: Make burst_length 64bit and add range checks
   throttle: Test the valid range of config values
   misc: Remove unused Error variables
-Dan Aloni (1):
+Sam Li (17):
-  nvme: Fix get/set number of queues feature, again
+  block/block-common: add zoned device structs
   block/file-posix: introduce helper functions for sysfs attributes
   block/block-backend: add block layer APIs resembling Linux
     ZonedBlockDevice ioctls
   block/raw-format: add zone operations to pass through requests
   block: add zoned BlockDriver check to block layer
   iotests: test new zone operations
   block: add some trace events for new block layer APIs
   docs/zoned-storage: add zoned device documentation
   file-posix: add tracking of the zone write pointers
   block: introduce zone append write for zoned devices
   qemu-iotests: test zone append operation
   block: add some trace events for zone append
   include: update virtio_blk headers to v6.3-rc1
   virtio-blk: add zoned storage emulation for zoned devices
   block: add accounting for zone append operation
   virtio-blk: add some trace events for zoned emulation
   docs/zoned-storage:add zoned emulation use case
-Eduardo Habkost (1):
+Thomas De Schampheleire (1):
-  oslib-posix: Print errors before aborting on qemu_alloc_stack()
+  tracetool: use relative paths for '#line' preprocessor directives
-Fred Rolland (1):
+ docs/devel/index-api.rst                     |   1 +
-  qemu-doc: Add UUID support in initiator name
+ docs/devel/zoned-storage.rst                 |  62 ++
+ qapi/block-core.json                         |  68 +-
-Stefan Hajnoczi (4):
+ qapi/block.json                              |   4 +
-  scripts: add argparse module for Python 2.6 compatibility
+ meson.build                                  |   4 +
-  docker.py: Python 2.6 argparse compatibility
+ block/dmg.h                                  |   8 +-
-  tests: migration/guestperf Python 2.6 argparse compatibility
+ include/block/accounting.h                   |   1 +
-  qcow2: allocate cluster_cache/cluster_data on demand
+ include/block/block-common.h                 |  57 ++
+ include/block/block-io.h                     |  13 +
- include/qemu/throttle.h            |    8 +-
+ include/block/block_int-common.h             |  37 +
- block/qcow.c                       |   12 +-
+ include/block/raw-aio.h                      |   8 +-
- block/qcow2-cluster.c              |   17 +
+ include/standard-headers/drm/drm_fourcc.h    |  12 +
- block/qcow2.c                      |   20 +-
+ include/standard-headers/linux/ethtool.h     |  48 +-
- dump.c                             |    4 +-
+ include/standard-headers/linux/fuse.h        |  45 +-
- hw/block/nvme.c                    |    4 +-
+ include/standard-headers/linux/pci_regs.h    |   1 +
- tests/test-throttle.c              |   80 +-
+ include/standard-headers/linux/vhost_types.h |   2 +
- util/oslib-posix.c                 |    2 +
+ include/standard-headers/linux/virtio_blk.h  | 105 +++
- util/throttle.c                    |   86 +-
+ include/sysemu/block-backend-io.h            |  27 +
- COPYING.PYTHON                     |  270 ++++
+ linux-headers/asm-arm64/kvm.h                |   1 +
- qemu-doc.texi                      |    5 +-
+ linux-headers/asm-x86/kvm.h                  |  34 +-
- scripts/argparse.py                | 2406 ++++++++++++++++++++++++++++++++++++
+ linux-headers/linux/kvm.h                    |   9 +
- tests/docker/docker.py             |    4 +-
+ linux-headers/linux/vfio.h                   |  15 +-
- tests/migration/guestperf/shell.py |    8 +-
+ linux-headers/linux/vhost.h                  |   8 +
-files changed, 2831 insertions(+), 95 deletions(-)
+ block.c                                      |  19 +
- create mode 100644 COPYING.PYTHON
+ block/block-backend.c                        | 193 ++++++
- create mode 100644 scripts/argparse.py
+ block/dmg.c                                  |   7 +-
  block/file-posix.c                           | 677 +++++++++++++++++--
  block/io.c                                   |  68 ++
  block/io_uring.c                             |   4 +
  block/linux-aio.c                            |   3 +
  block/qapi-sysemu.c                          |  11 +
  block/qapi.c                                 |  18 +
  block/raw-format.c                           |  26 +
  hw/block/virtio-blk-common.c                 |   2 +
  hw/block/virtio-blk.c                        | 405 +++++++++++
  hw/virtio/virtio-qmp.c                       |   2 +
  qemu-io-cmds.c                               | 224 ++++++
  block/trace-events                           |   4 +
  docs/system/qemu-block-drivers.rst.inc       |   6 +
  hw/block/trace-events                        |   7 +
  scripts/tracetool/backend/ftrace.py          |   4 +-
  scripts/tracetool/backend/log.py             |   4 +-
  scripts/tracetool/backend/syslog.py          |   4 +-
  tests/qemu-iotests/tests/zoned               | 105 +++
  tests/qemu-iotests/tests/zoned.out           |  69 ++
  trace/meson.build                            |   2 +-
 files changed, 2353 insertions(+), 81 deletions(-)
  create mode 100644 docs/devel/zoned-storage.rst
  create mode 100755 tests/qemu-iotests/tests/zoned
  create mode 100644 tests/qemu-iotests/tests/zoned.out
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 14/15] qemu-doc: Add UUID support in initiator name
+[PULL 01/20] block/block-common: add zoned device structs
-From: Fred Rolland <rollandf@gmail.com>
+From: Sam Li <faithilikerun@gmail.com>
-Update doc with the usage of UUID for initiator name.
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
-Related-To: https://bugzilla.redhat.com/1006468
+Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
-Signed-off-by: Fred Rolland <frolland@redhat.com>
+Reviewed-by: Hannes Reinecke <hare@suse.de>
-Message-id: 20170823084830.30500-1-frolland@redhat.com
+Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
 Acked-by: Kevin Wolf <kwolf@redhat.com>
 Message-id: 20230324090605.28361-2-faithilikerun@gmail.com
 [Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
 <philmd@linaro.org>.
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- qemu-doc.texi | 5 +++--
+ include/block/block-common.h | 43 ++++++++++++++++++++++++++++++++++++
-file changed, 3 insertions(+), 2 deletions(-)
+file changed, 43 insertions(+)
-diff --git a/qemu-doc.texi b/qemu-doc.texi
+diff --git a/include/block/block-common.h b/include/block/block-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/qemu-doc.texi
+--- a/include/block/block-common.h
-+++ b/qemu-doc.texi
++++ b/include/block/block-common.h
-@@ -XXX,XX +XXX,XX @@ in a configuration file provided via '-readconfig' or directly on the
+@@ -XXX,XX +XXX,XX @@ typedef struct BlockDriver BlockDriver;
- command line.
+ typedef struct BdrvChild BdrvChild;
+ typedef struct BdrvChildClass BdrvChildClass;
- If the initiator-name is not specified qemu will use a default name
--of 'iqn.2008-11.org.linux-kvm[:<name>'] where <name> is the name of the
++typedef enum BlockZoneOp {
-+of 'iqn.2008-11.org.linux-kvm[:<uuid>'] where <uuid> is the UUID of the
++    BLK_ZO_OPEN,
-+virtual machine. If the UUID is not specified qemu will use
++    BLK_ZO_CLOSE,
-+'iqn.2008-11.org.linux-kvm[:<name>'] where <name> is the name of the
++    BLK_ZO_FINISH,
- virtual machine.
++    BLK_ZO_RESET,
++} BlockZoneOp;
--
++
- @example
++typedef enum BlockZoneModel {
- Setting a specific initiator name to use when logging in to the target
++    BLK_Z_NONE = 0x0, /* Regular block device */
- -iscsi initiator-name=iqn.qemu.test:my-initiator
++    BLK_Z_HM = 0x1, /* Host-managed zoned block device */
 +    BLK_Z_HA = 0x2, /* Host-aware zoned block device */
 +} BlockZoneModel;
 +
 +typedef enum BlockZoneState {
 +    BLK_ZS_NOT_WP = 0x0,
 +    BLK_ZS_EMPTY = 0x1,
 +    BLK_ZS_IOPEN = 0x2,
 +    BLK_ZS_EOPEN = 0x3,
 +    BLK_ZS_CLOSED = 0x4,
 +    BLK_ZS_RDONLY = 0xD,
 +    BLK_ZS_FULL = 0xE,
 +    BLK_ZS_OFFLINE = 0xF,
 +} BlockZoneState;
 +
 +typedef enum BlockZoneType {
 +    BLK_ZT_CONV = 0x1, /* Conventional random writes supported */
 +    BLK_ZT_SWR = 0x2, /* Sequential writes required */
 +    BLK_ZT_SWP = 0x3, /* Sequential writes preferred */
 +} BlockZoneType;
 +
 +/*
 + * Zone descriptor data structure.
 + * Provides information on a zone with all position and size values in bytes.
 + */
 +typedef struct BlockZoneDescriptor {
 +    uint64_t start;
 +    uint64_t length;
 +    uint64_t cap;
 +    uint64_t wp;
 +    BlockZoneType type;
 +    BlockZoneState state;
 +} BlockZoneDescriptor;
 +
  typedef struct BlockDriverInfo {
      /* in bytes, 0 if irrelevant */
      int cluster_size;
 --
-.13.5
+.39.2

-New patch
+[PULL 02/20] block/file-posix: introduce helper functions for sysfs attributes
+From: Sam Li <faithilikerun@gmail.com>
 Use get_sysfs_str_val() to get the string value of device
 zoned model. Then get_sysfs_zoned_model() can convert it to
 BlockZoneModel type of QEMU.
 Use get_sysfs_long_val() to get the long value of zoned device
 information.
 Signed-off-by: Sam Li <faithilikerun@gmail.com>
 Reviewed-by: Hannes Reinecke <hare@suse.de>
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
 Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
 Acked-by: Kevin Wolf <kwolf@redhat.com>
 Message-id: 20230324090605.28361-3-faithilikerun@gmail.com
 [Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
 <philmd@linaro.org>.
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
  include/block/block_int-common.h |   3 +
  block/file-posix.c               | 130 ++++++++++++++++++++++---------
 files changed, 95 insertions(+), 38 deletions(-)
 diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block_int-common.h
 +++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ typedef struct BlockLimits {
       * an explicit monitor command to load the disk inside the guest).
       */
      bool has_variable_length;
 +
 +    /* device zone model */
 +    BlockZoneModel zoned;
  } BlockLimits;
  typedef struct BdrvOpBlocker BdrvOpBlocker;
 diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/file-posix.c
 +++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int hdev_get_max_hw_transfer(int fd, struct stat *st)
  #endif
  }
 -static int hdev_get_max_segments(int fd, struct stat *st)
 +/*
 + * Get a sysfs attribute value as character string.
 + */
 +static int get_sysfs_str_val(struct stat *st, const char *attribute,
 +                             char **val) {
 +#ifdef CONFIG_LINUX
 +    g_autofree char *sysfspath = NULL;
 +    int ret;
 +    size_t len;
 +
 +    if (!S_ISBLK(st->st_mode)) {
 +        return -ENOTSUP;
 +    }
 +
 +    sysfspath = g_strdup_printf("/sys/dev/block/%u:%u/queue/%s",
 +                                major(st->st_rdev), minor(st->st_rdev),
 +                                attribute);
 +    ret = g_file_get_contents(sysfspath, val, &len, NULL);
 +    if (ret == -1) {
 +        return -ENOENT;
 +    }
 +
 +    /* The file is ended with '\n' */
 +    char *p;
 +    p = *val;
 +    if (*(p + len - 1) == '\n') {
 +        *(p + len - 1) = '\0';
 +    }
 +    return ret;
 +#else
 +    return -ENOTSUP;
 +#endif
 +}
 +
 +static int get_sysfs_zoned_model(struct stat *st, BlockZoneModel *zoned)
 +{
 +    g_autofree char *val = NULL;
 +    int ret;
 +
 +    ret = get_sysfs_str_val(st, "zoned", &val);
 +    if (ret < 0) {
 +        return ret;
 +    }
 +
 +    if (strcmp(val, "host-managed") == 0) {
 +        *zoned = BLK_Z_HM;
 +    } else if (strcmp(val, "host-aware") == 0) {
 +        *zoned = BLK_Z_HA;
 +    } else if (strcmp(val, "none") == 0) {
 +        *zoned = BLK_Z_NONE;
 +    } else {
 +        return -ENOTSUP;
 +    }
 +    return 0;
 +}
 +
 +/*
 + * Get a sysfs attribute value as a long integer.
 + */
 +static long get_sysfs_long_val(struct stat *st, const char *attribute)
  {
  #ifdef CONFIG_LINUX
 -    char buf[32];
 +    g_autofree char *str = NULL;
      const char *end;
 -    char *sysfspath = NULL;
 +    long val;
 +    int ret;
 +
 +    ret = get_sysfs_str_val(st, attribute, &str);
 +    if (ret < 0) {
 +        return ret;
 +    }
 +
 +    /* The file is ended with '\n', pass 'end' to accept that. */
 +    ret = qemu_strtol(str, &end, 10, &val);
 +    if (ret == 0 && end && *end == '\0') {
 +        ret = val;
 +    }
 +    return ret;
 +#else
 +    return -ENOTSUP;
 +#endif
 +}
 +
 +static int hdev_get_max_segments(int fd, struct stat *st)
 +{
 +#ifdef CONFIG_LINUX
      int ret;
 -    int sysfd = -1;
 -    long max_segments;
      if (S_ISCHR(st->st_mode)) {
          if (ioctl(fd, SG_GET_SG_TABLESIZE, &ret) == 0) {
@@ -XXX,XX +XXX,XX @@ static int hdev_get_max_segments(int fd, struct stat *st)
          }
          return -ENOTSUP;
      }
 -
 -    if (!S_ISBLK(st->st_mode)) {
 -        return -ENOTSUP;
 -    }
 -
 -    sysfspath = g_strdup_printf("/sys/dev/block/%u:%u/queue/max_segments",
 -                                major(st->st_rdev), minor(st->st_rdev));
 -    sysfd = open(sysfspath, O_RDONLY);
 -    if (sysfd == -1) {
 -        ret = -errno;
 -        goto out;
 -    }
 -    ret = RETRY_ON_EINTR(read(sysfd, buf, sizeof(buf) - 1));
 -    if (ret < 0) {
 -        ret = -errno;
 -        goto out;
 -    } else if (ret == 0) {
 -        ret = -EIO;
 -        goto out;
 -    }
 -    buf[ret] = 0;
 -    /* The file is ended with '\n', pass 'end' to accept that. */
 -    ret = qemu_strtol(buf, &end, 10, &max_segments);
 -    if (ret == 0 && end && *end == '\n') {
 -        ret = max_segments;
 -    }
 -
 -out:
 -    if (sysfd != -1) {
 -        close(sysfd);
 -    }
 -    g_free(sysfspath);
 -    return ret;
 +    return get_sysfs_long_val(st, "max_segments");
  #else
      return -ENOTSUP;
  #endif
@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
  {
      BDRVRawState *s = bs->opaque;
      struct stat st;
 +    int ret;
 +    BlockZoneModel zoned;
      s->needs_alignment = raw_needs_alignment(bs);
      raw_probe_alignment(bs, s->fd, errp);
@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
              bs->bl.max_hw_iov = ret;
          }
      }
 +
 +    ret = get_sysfs_zoned_model(&st, &zoned);
 +    if (ret < 0) {
 +        zoned = BLK_Z_NONE;
 +    }
 +    bs->bl.zoned = zoned;
  }
  static int check_for_dasd(int fd)
 --
 .39.2

-[Qemu-devel] [PULL for-2.10 06/15] throttle: Make LeakyBucket.avg and LeakyBucket.max integer types
+[PULL 03/20] block/block-backend: add block layer APIs resembling Linux ZonedBlockDevice ioctls
-From: Alberto Garcia <berto@igalia.com>
+From: Sam Li <faithilikerun@gmail.com>
-Both the throttling limits set with the throttling.iops-* and
+Add zoned device option to host_device BlockDriver. It will be presented only
-throttling.bps-* options and their QMP equivalents defined in the
+for zoned host block devices. By adding zone management operations to the
-BlockIOThrottle struct are integer values.
+host_block_device BlockDriver, users can use the new block layer APIs
 including Report Zone and four zone management operations
 (open, close, finish, reset, reset_all).
-Those limits are also reported in the BlockDeviceInfo struct and they
+Qemu-io uses the new APIs to perform zoned storage commands of the device:
-are integers there as well.
+zone_report(zrp), zone_open(zo), zone_close(zc), zone_reset(zrs),
 zone_finish(zf).
-Therefore there's no reason to store them internally as double and do
+For example, to test zone_report, use following command:
-the conversion everytime we're setting or querying them, so this patch
+$ ./build/qemu-io --image-opts -n driver=host_device, filename=/dev/nullb0
-uses uint64_t for those types. Let's also use an unsigned type because
+-c "zrp offset nr_zones"
 we don't allow negative values anyway.
-LeakyBucket.level and LeakyBucket.burst_level do however remain double
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-because their value changes depending on the fraction of time elapsed
+Reviewed-by: Hannes Reinecke <hare@suse.de>
-since the previous I/O operation.
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
-Signed-off-by: Alberto Garcia <berto@igalia.com>
+Acked-by: Kevin Wolf <kwolf@redhat.com>
-Message-id: f29b840422767b5be2c41c2dfdbbbf6c5f8fedf8.1503580370.git.berto@igalia.com
+Message-id: 20230324090605.28361-4-faithilikerun@gmail.com
 [Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
 <philmd@linaro.org> and remove spurious ret = -errno in
 raw_co_zone_mgmt().
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- include/qemu/throttle.h | 4 ++--
+ meson.build                       |   4 +
- tests/test-throttle.c   | 3 ++-
+ include/block/block-io.h          |   9 +
- util/throttle.c         | 7 +++----
+ include/block/block_int-common.h  |  21 ++
-files changed, 7 insertions(+), 7 deletions(-)
+ include/block/raw-aio.h           |   6 +-
  include/sysemu/block-backend-io.h |  18 ++
  block/block-backend.c             | 133 +++++++++++++
  block/file-posix.c                | 306 +++++++++++++++++++++++++++++-
  block/io.c                        |  41 ++++
  qemu-io-cmds.c                    | 149 +++++++++++++++
 files changed, 684 insertions(+), 3 deletions(-)
-diff --git a/include/qemu/throttle.h b/include/qemu/throttle.h
+diff --git a/meson.build b/meson.build
 index XXXXXXX..XXXXXXX 100644
---- a/include/qemu/throttle.h
+--- a/meson.build
-+++ b/include/qemu/throttle.h
++++ b/meson.build
-@@ -XXX,XX +XXX,XX @@ typedef enum {
+@@ -XXX,XX +XXX,XX @@ config_host_data.set('CONFIG_REPLICATION', get_option('replication').allowed())
-  */
+ # has_header
+ config_host_data.set('CONFIG_EPOLL', cc.has_header('sys/epoll.h'))
- typedef struct LeakyBucket {
+ config_host_data.set('CONFIG_LINUX_MAGIC_H', cc.has_header('linux/magic.h'))
--    double  avg;              /* average goal in units per second */
++config_host_data.set('CONFIG_BLKZONED', cc.has_header('linux/blkzoned.h'))
--    double  max;              /* leaky bucket max burst in units */
+ config_host_data.set('CONFIG_VALGRIND_H', cc.has_header('valgrind/valgrind.h'))
-+    uint64_t avg;             /* average goal in units per second */
+ config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
-+    uint64_t max;             /* leaky bucket max burst in units */
+ config_host_data.set('HAVE_DRM_H', cc.has_header('libdrm/drm.h'))
-     double  level;            /* bucket level in units */
+@@ -XXX,XX +XXX,XX @@ config_host_data.set('HAVE_SIGEV_NOTIFY_THREAD_ID',
-     double  burst_level;      /* bucket level in units (for computing bursts) */
+ config_host_data.set('HAVE_STRUCT_STAT_ST_ATIM',
-     unsigned burst_length;    /* max length of the burst period, in seconds */
+                      cc.has_member('struct stat', 'st_atim',
-diff --git a/tests/test-throttle.c b/tests/test-throttle.c
+                                    prefix: '#include <sys/stat.h>'))
 +config_host_data.set('HAVE_BLK_ZONE_REP_CAPACITY',
 +                     cc.has_member('struct blk_zone', 'capacity',
 +                                   prefix: '#include <linux/blkzoned.h>'))
  # has_type
  config_host_data.set('CONFIG_IOVEC',
 diff --git a/include/block/block-io.h b/include/block/block-io.h
 index XXXXXXX..XXXXXXX 100644
---- a/tests/test-throttle.c
+--- a/include/block/block-io.h
-+++ b/tests/test-throttle.c
++++ b/include/block/block-io.h
-@@ -XXX,XX +XXX,XX @@ static void test_enabled(void)
+@@ -XXX,XX +XXX,XX @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_flush(BlockDriverState *bs);
-     for (i = 0; i < BUCKETS_COUNT; i++) {
+ int coroutine_fn GRAPH_RDLOCK bdrv_co_pdiscard(BdrvChild *child, int64_t offset,
-         throttle_config_init(&cfg);
+                                                int64_t bytes);
-         set_cfg_value(false, i, 150);
-+        g_assert(throttle_is_valid(&cfg, NULL));
++/* Report zone information of zone block device. */
-         g_assert(throttle_enabled(&cfg));
++int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_report(BlockDriverState *bs,
 +                                                  int64_t offset,
 +                                                  unsigned int *nr_zones,
 +                                                  BlockZoneDescriptor *zones);
 +int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_mgmt(BlockDriverState *bs,
 +                                                BlockZoneOp op,
 +                                                int64_t offset, int64_t len);
 +
  bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
  int bdrv_block_status(BlockDriverState *bs, int64_t offset,
                        int64_t bytes, int64_t *pnum, int64_t *map,
 diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block_int-common.h
 +++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
      int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_load_vmstate)(
          BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
 +    int coroutine_fn (*bdrv_co_zone_report)(BlockDriverState *bs,
 +            int64_t offset, unsigned int *nr_zones,
 +            BlockZoneDescriptor *zones);
 +    int coroutine_fn (*bdrv_co_zone_mgmt)(BlockDriverState *bs, BlockZoneOp op,
 +            int64_t offset, int64_t len);
 +
      /* removable device specific */
      bool coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_is_inserted)(
          BlockDriverState *bs);
@@ -XXX,XX +XXX,XX @@ typedef struct BlockLimits {
      /* device zone model */
      BlockZoneModel zoned;
 +
 +    /* zone size expressed in bytes */
 +    uint32_t zone_size;
 +
 +    /* total number of zones */
 +    uint32_t nr_zones;
 +
 +    /* maximum sectors of a zone append write operation */
 +    int64_t max_append_sectors;
 +
 +    /* maximum number of open zones */
 +    int64_t max_open_zones;
 +
 +    /* maximum number of active zones */
 +    int64_t max_active_zones;
  } BlockLimits;
  typedef struct BdrvOpBlocker BdrvOpBlocker;
 diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/raw-aio.h
 +++ b/include/block/raw-aio.h
@@ -XXX,XX +XXX,XX @@
  #define QEMU_AIO_WRITE_ZEROES 0x0020
  #define QEMU_AIO_COPY_RANGE   0x0040
  #define QEMU_AIO_TRUNCATE     0x0080
 +#define QEMU_AIO_ZONE_REPORT  0x0100
 +#define QEMU_AIO_ZONE_MGMT    0x0200
  #define QEMU_AIO_TYPE_MASK \
          (QEMU_AIO_READ | \
           QEMU_AIO_WRITE | \
@@ -XXX,XX +XXX,XX @@
           QEMU_AIO_DISCARD | \
           QEMU_AIO_WRITE_ZEROES | \
           QEMU_AIO_COPY_RANGE | \
 -         QEMU_AIO_TRUNCATE)
 +         QEMU_AIO_TRUNCATE | \
 +         QEMU_AIO_ZONE_REPORT | \
 +         QEMU_AIO_ZONE_MGMT)
  /* AIO flags */
  #define QEMU_AIO_MISALIGNED   0x1000
 diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backend-io.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/sysemu/block-backend-io.h
 +++ b/include/sysemu/block-backend-io.h
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_pwritev(BlockBackend *blk, int64_t offset,
                              BlockCompletionFunc *cb, void *opaque);
  BlockAIOCB *blk_aio_flush(BlockBackend *blk,
                            BlockCompletionFunc *cb, void *opaque);
 +BlockAIOCB *blk_aio_zone_report(BlockBackend *blk, int64_t offset,
 +                                unsigned int *nr_zones,
 +                                BlockZoneDescriptor *zones,
 +                                BlockCompletionFunc *cb, void *opaque);
 +BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
 +                              int64_t offset, int64_t len,
 +                              BlockCompletionFunc *cb, void *opaque);
  BlockAIOCB *blk_aio_pdiscard(BlockBackend *blk, int64_t offset, int64_t bytes,
                               BlockCompletionFunc *cb, void *opaque);
  void blk_aio_cancel_async(BlockAIOCB *acb);
@@ -XXX,XX +XXX,XX @@ int co_wrapper_mixed blk_pwrite_zeroes(BlockBackend *blk, int64_t offset,
  int coroutine_fn blk_co_pwrite_zeroes(BlockBackend *blk, int64_t offset,
                                        int64_t bytes, BdrvRequestFlags flags);
 +int coroutine_fn blk_co_zone_report(BlockBackend *blk, int64_t offset,
 +                                    unsigned int *nr_zones,
 +                                    BlockZoneDescriptor *zones);
 +int co_wrapper_mixed blk_zone_report(BlockBackend *blk, int64_t offset,
 +                                         unsigned int *nr_zones,
 +                                         BlockZoneDescriptor *zones);
 +int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
 +                                  int64_t offset, int64_t len);
 +int co_wrapper_mixed blk_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
 +                                       int64_t offset, int64_t len);
 +
  int co_wrapper_mixed blk_pdiscard(BlockBackend *blk, int64_t offset,
                                    int64_t bytes);
  int coroutine_fn blk_co_pdiscard(BlockBackend *blk, int64_t offset,
 diff --git a/block/block-backend.c b/block/block-backend.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/block-backend.c
 +++ b/block/block-backend.c
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_flush(BlockBackend *blk)
      return ret;
  }
 +static void coroutine_fn blk_aio_zone_report_entry(void *opaque)
 +{
 +    BlkAioEmAIOCB *acb = opaque;
 +    BlkRwCo *rwco = &acb->rwco;
 +
 +    rwco->ret = blk_co_zone_report(rwco->blk, rwco->offset,
 +                                   (unsigned int*)acb->bytes,rwco->iobuf);
 +    blk_aio_complete(acb);
 +}
 +
 +BlockAIOCB *blk_aio_zone_report(BlockBackend *blk, int64_t offset,
 +                                unsigned int *nr_zones,
 +                                BlockZoneDescriptor  *zones,
 +                                BlockCompletionFunc *cb, void *opaque)
 +{
 +    BlkAioEmAIOCB *acb;
 +    Coroutine *co;
 +    IO_CODE();
 +
 +    blk_inc_in_flight(blk);
 +    acb = blk_aio_get(&blk_aio_em_aiocb_info, blk, cb, opaque);
 +    acb->rwco = (BlkRwCo) {
 +        .blk    = blk,
 +        .offset = offset,
 +        .iobuf  = zones,
 +        .ret    = NOT_DONE,
 +    };
 +    acb->bytes = (int64_t)nr_zones,
 +    acb->has_returned = false;
 +
 +    co = qemu_coroutine_create(blk_aio_zone_report_entry, acb);
 +    aio_co_enter(blk_get_aio_context(blk), co);
 +
 +    acb->has_returned = true;
 +    if (acb->rwco.ret != NOT_DONE) {
 +        replay_bh_schedule_oneshot_event(blk_get_aio_context(blk),
 +                                         blk_aio_complete_bh, acb);
 +    }
 +
 +    return &acb->common;
 +}
 +
 +static void coroutine_fn blk_aio_zone_mgmt_entry(void *opaque)
 +{
 +    BlkAioEmAIOCB *acb = opaque;
 +    BlkRwCo *rwco = &acb->rwco;
 +
 +    rwco->ret = blk_co_zone_mgmt(rwco->blk, (BlockZoneOp)rwco->iobuf,
 +                                 rwco->offset, acb->bytes);
 +    blk_aio_complete(acb);
 +}
 +
 +BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
 +                              int64_t offset, int64_t len,
 +                              BlockCompletionFunc *cb, void *opaque) {
 +    BlkAioEmAIOCB *acb;
 +    Coroutine *co;
 +    IO_CODE();
 +
 +    blk_inc_in_flight(blk);
 +    acb = blk_aio_get(&blk_aio_em_aiocb_info, blk, cb, opaque);
 +    acb->rwco = (BlkRwCo) {
 +        .blk    = blk,
 +        .offset = offset,
 +        .iobuf  = (void *)op,
 +        .ret    = NOT_DONE,
 +    };
 +    acb->bytes = len;
 +    acb->has_returned = false;
 +
 +    co = qemu_coroutine_create(blk_aio_zone_mgmt_entry, acb);
 +    aio_co_enter(blk_get_aio_context(blk), co);
 +
 +    acb->has_returned = true;
 +    if (acb->rwco.ret != NOT_DONE) {
 +        replay_bh_schedule_oneshot_event(blk_get_aio_context(blk),
 +                                         blk_aio_complete_bh, acb);
 +    }
 +
 +    return &acb->common;
 +}
 +
 +/*
 + * Send a zone_report command.
 + * offset is a byte offset from the start of the device. No alignment
 + * required for offset.
 + * nr_zones represents IN maximum and OUT actual.
 + */
 +int coroutine_fn blk_co_zone_report(BlockBackend *blk, int64_t offset,
 +                                    unsigned int *nr_zones,
 +                                    BlockZoneDescriptor *zones)
 +{
 +    int ret;
 +    IO_CODE();
 +
 +    blk_inc_in_flight(blk); /* increase before waiting */
 +    blk_wait_while_drained(blk);
 +    if (!blk_is_available(blk)) {
 +        blk_dec_in_flight(blk);
 +        return -ENOMEDIUM;
 +    }
 +    ret = bdrv_co_zone_report(blk_bs(blk), offset, nr_zones, zones);
 +    blk_dec_in_flight(blk);
 +    return ret;
 +}
 +
 +/*
 + * Send a zone_management command.
 + * op is the zone operation;
 + * offset is the byte offset from the start of the zoned device;
 + * len is the maximum number of bytes the command should operate on. It
 + * should be aligned with the device zone size.
 + */
 +int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
 +        int64_t offset, int64_t len)
 +{
 +    int ret;
 +    IO_CODE();
 +
 +    blk_inc_in_flight(blk);
 +    blk_wait_while_drained(blk);
 +
 +    ret = blk_check_byte_request(blk, offset, len);
 +    if (ret < 0) {
 +        blk_dec_in_flight(blk);
 +        return ret;
 +    }
 +
 +    ret = bdrv_co_zone_mgmt(blk_bs(blk), op, offset, len);
 +    blk_dec_in_flight(blk);
 +    return ret;
 +}
 +
  void blk_drain(BlockBackend *blk)
  {
      BlockDriverState *bs = blk_bs(blk);
 diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/file-posix.c
 +++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@
  #include <sys/param.h>
  #include <sys/syscall.h>
  #include <sys/vfs.h>
 +#if defined(CONFIG_BLKZONED)
 +#include <linux/blkzoned.h>
 +#endif
  #include <linux/cdrom.h>
  #include <linux/fd.h>
  #include <linux/fs.h>
@@ -XXX,XX +XXX,XX @@ typedef struct RawPosixAIOData {
              PreallocMode prealloc;
              Error **errp;
          } truncate;
 +        struct {
 +            unsigned int *nr_zones;
 +            BlockZoneDescriptor *zones;
 +        } zone_report;
 +        struct {
 +            unsigned long op;
 +        } zone_mgmt;
      };
  } RawPosixAIOData;
@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
          zoned = BLK_Z_NONE;
      }
+     bs->bl.zoned = zoned;
-     for (i = 0; i < BUCKETS_COUNT; i++) {
++    if (zoned != BLK_Z_NONE) {
-         throttle_config_init(&cfg);
++        /*
-         set_cfg_value(false, i, -150);
++         * The zoned device must at least have zone size and nr_zones fields.
--        g_assert(!throttle_enabled(&cfg));
++         */
-+        g_assert(!throttle_is_valid(&cfg, NULL));
++        ret = get_sysfs_long_val(&st, "chunk_sectors");
 +        if (ret < 0) {
 +            error_setg_errno(errp, -ret, "Unable to read chunk_sectors "
 +                                         "sysfs attribute");
 +            goto out;
 +        } else if (!ret) {
 +            error_setg(errp, "Read 0 from chunk_sectors sysfs attribute");
 +            goto out;
 +        }
 +        bs->bl.zone_size = ret << BDRV_SECTOR_BITS;
 +
 +        ret = get_sysfs_long_val(&st, "nr_zones");
 +        if (ret < 0) {
 +            error_setg_errno(errp, -ret, "Unable to read nr_zones "
 +                                         "sysfs attribute");
 +            goto out;
 +        } else if (!ret) {
 +            error_setg(errp, "Read 0 from nr_zones sysfs attribute");
 +            goto out;
 +        }
 +        bs->bl.nr_zones = ret;
 +
 +        ret = get_sysfs_long_val(&st, "zone_append_max_bytes");
 +        if (ret > 0) {
 +            bs->bl.max_append_sectors = ret >> BDRV_SECTOR_BITS;
 +        }
 +
 +        ret = get_sysfs_long_val(&st, "max_open_zones");
 +        if (ret >= 0) {
 +            bs->bl.max_open_zones = ret;
 +        }
 +
 +        ret = get_sysfs_long_val(&st, "max_active_zones");
 +        if (ret >= 0) {
 +            bs->bl.max_active_zones = ret;
 +        }
 +        return;
 +    }
 +out:
 +    bs->bl.zoned = BLK_Z_NONE;
  }
  static int check_for_dasd(int fd)
@@ -XXX,XX +XXX,XX @@ static int hdev_probe_blocksizes(BlockDriverState *bs, BlockSizes *bsz)
      BDRVRawState *s = bs->opaque;
      int ret;
 -    /* If DASD, get blocksizes */
 +    /* If DASD or zoned devices, get blocksizes */
      if (check_for_dasd(s->fd) < 0) {
 -        return -ENOTSUP;
 +        /* zoned devices are not DASD */
 +        if (bs->bl.zoned == BLK_Z_NONE) {
 +            return -ENOTSUP;
 +        }
      }
      ret = probe_logical_blocksize(s->fd, &bsz->log);
      if (ret < 0) {
@@ -XXX,XX +XXX,XX @@ static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd,
  }
  #endif
 +/*
 + * parse_zone - Fill a zone descriptor
 + */
 +#if defined(CONFIG_BLKZONED)
 +static inline int parse_zone(struct BlockZoneDescriptor *zone,
 +                              const struct blk_zone *blkz) {
 +    zone->start = blkz->start << BDRV_SECTOR_BITS;
 +    zone->length = blkz->len << BDRV_SECTOR_BITS;
 +    zone->wp = blkz->wp << BDRV_SECTOR_BITS;
 +
 +#ifdef HAVE_BLK_ZONE_REP_CAPACITY
 +    zone->cap = blkz->capacity << BDRV_SECTOR_BITS;
 +#else
 +    zone->cap = blkz->len << BDRV_SECTOR_BITS;
 +#endif
 +
 +    switch (blkz->type) {
 +    case BLK_ZONE_TYPE_SEQWRITE_REQ:
 +        zone->type = BLK_ZT_SWR;
 +        break;
 +    case BLK_ZONE_TYPE_SEQWRITE_PREF:
 +        zone->type = BLK_ZT_SWP;
 +        break;
 +    case BLK_ZONE_TYPE_CONVENTIONAL:
 +        zone->type = BLK_ZT_CONV;
 +        break;
 +    default:
 +        error_report("Unsupported zone type: 0x%x", blkz->type);
 +        return -ENOTSUP;
 +    }
 +
 +    switch (blkz->cond) {
 +    case BLK_ZONE_COND_NOT_WP:
 +        zone->state = BLK_ZS_NOT_WP;
 +        break;
 +    case BLK_ZONE_COND_EMPTY:
 +        zone->state = BLK_ZS_EMPTY;
 +        break;
 +    case BLK_ZONE_COND_IMP_OPEN:
 +        zone->state = BLK_ZS_IOPEN;
 +        break;
 +    case BLK_ZONE_COND_EXP_OPEN:
 +        zone->state = BLK_ZS_EOPEN;
 +        break;
 +    case BLK_ZONE_COND_CLOSED:
 +        zone->state = BLK_ZS_CLOSED;
 +        break;
 +    case BLK_ZONE_COND_READONLY:
 +        zone->state = BLK_ZS_RDONLY;
 +        break;
 +    case BLK_ZONE_COND_FULL:
 +        zone->state = BLK_ZS_FULL;
 +        break;
 +    case BLK_ZONE_COND_OFFLINE:
 +        zone->state = BLK_ZS_OFFLINE;
 +        break;
 +    default:
 +        error_report("Unsupported zone state: 0x%x", blkz->cond);
 +        return -ENOTSUP;
 +    }
 +    return 0;
 +}
 +#endif
 +
 +#if defined(CONFIG_BLKZONED)
 +static int handle_aiocb_zone_report(void *opaque)
 +{
 +    RawPosixAIOData *aiocb = opaque;
 +    int fd = aiocb->aio_fildes;
 +    unsigned int *nr_zones = aiocb->zone_report.nr_zones;
 +    BlockZoneDescriptor *zones = aiocb->zone_report.zones;
 +    /* zoned block devices use 512-byte sectors */
 +    uint64_t sector = aiocb->aio_offset / 512;
 +
 +    struct blk_zone *blkz;
 +    size_t rep_size;
 +    unsigned int nrz;
 +    int ret, n = 0, i = 0;
 +
 +    nrz = *nr_zones;
 +    rep_size = sizeof(struct blk_zone_report) + nrz * sizeof(struct blk_zone);
 +    g_autofree struct blk_zone_report *rep = NULL;
 +    rep = g_malloc(rep_size);
 +
 +    blkz = (struct blk_zone *)(rep + 1);
 +    while (n < nrz) {
 +        memset(rep, 0, rep_size);
 +        rep->sector = sector;
 +        rep->nr_zones = nrz - n;
 +
 +        do {
 +            ret = ioctl(fd, BLKREPORTZONE, rep);
 +        } while (ret != 0 && errno == EINTR);
 +        if (ret != 0) {
 +            error_report("%d: ioctl BLKREPORTZONE at %" PRId64 " failed %d",
 +                         fd, sector, errno);
 +            return -errno;
 +        }
 +
 +        if (!rep->nr_zones) {
 +            break;
 +        }
 +
 +        for (i = 0; i < rep->nr_zones; i++, n++) {
 +            ret = parse_zone(&zones[n], &blkz[i]);
 +            if (ret != 0) {
 +                return ret;
 +            }
 +
 +            /* The next report should start after the last zone reported */
 +            sector = blkz[i].start + blkz[i].len;
 +        }
 +    }
 +
 +    *nr_zones = n;
 +    return 0;
 +}
 +#endif
 +
 +#if defined(CONFIG_BLKZONED)
 +static int handle_aiocb_zone_mgmt(void *opaque)
 +{
 +    RawPosixAIOData *aiocb = opaque;
 +    int fd = aiocb->aio_fildes;
 +    uint64_t sector = aiocb->aio_offset / 512;
 +    int64_t nr_sectors = aiocb->aio_nbytes / 512;
 +    struct blk_zone_range range;
 +    int ret;
 +
 +    /* Execute the operation */
 +    range.sector = sector;
 +    range.nr_sectors = nr_sectors;
 +    do {
 +        ret = ioctl(fd, aiocb->zone_mgmt.op, &range);
 +    } while (ret != 0 && errno == EINTR);
 +
 +    return ret;
 +}
 +#endif
 +
  static int handle_aiocb_copy_range(void *opaque)
  {
      RawPosixAIOData *aiocb = opaque;
@@ -XXX,XX +XXX,XX @@ static void raw_account_discard(BDRVRawState *s, uint64_t nbytes, int ret)
      }
  }
-diff --git a/util/throttle.c b/util/throttle.c
++/*
 + * zone report - Get a zone block device's information in the form
 + * of an array of zone descriptors.
 + * zones is an array of zone descriptors to hold zone information on reply;
 + * offset can be any byte within the entire size of the device;
 + * nr_zones is the maxium number of sectors the command should operate on.
 + */
 +#if defined(CONFIG_BLKZONED)
 +static int coroutine_fn raw_co_zone_report(BlockDriverState *bs, int64_t offset,
 +                                           unsigned int *nr_zones,
 +                                           BlockZoneDescriptor *zones) {
 +    BDRVRawState *s = bs->opaque;
 +    RawPosixAIOData acb = (RawPosixAIOData) {
 +        .bs         = bs,
 +        .aio_fildes = s->fd,
 +        .aio_type   = QEMU_AIO_ZONE_REPORT,
 +        .aio_offset = offset,
 +        .zone_report    = {
 +            .nr_zones       = nr_zones,
 +            .zones          = zones,
 +        },
 +    };
 +
 +    return raw_thread_pool_submit(bs, handle_aiocb_zone_report, &acb);
 +}
 +#endif
 +
 +/*
 + * zone management operations - Execute an operation on a zone
 + */
 +#if defined(CONFIG_BLKZONED)
 +static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
 +        int64_t offset, int64_t len) {
 +    BDRVRawState *s = bs->opaque;
 +    RawPosixAIOData acb;
 +    int64_t zone_size, zone_size_mask;
 +    const char *op_name;
 +    unsigned long zo;
 +    int ret;
 +    int64_t capacity = bs->total_sectors << BDRV_SECTOR_BITS;
 +
 +    zone_size = bs->bl.zone_size;
 +    zone_size_mask = zone_size - 1;
 +    if (offset & zone_size_mask) {
 +        error_report("sector offset %" PRId64 " is not aligned to zone size "
 +                     "%" PRId64 "", offset / 512, zone_size / 512);
 +        return -EINVAL;
 +    }
 +
 +    if (((offset + len) < capacity && len & zone_size_mask) ||
 +        offset + len > capacity) {
 +        error_report("number of sectors %" PRId64 " is not aligned to zone size"
 +                      " %" PRId64 "", len / 512, zone_size / 512);
 +        return -EINVAL;
 +    }
 +
 +    switch (op) {
 +    case BLK_ZO_OPEN:
 +        op_name = "BLKOPENZONE";
 +        zo = BLKOPENZONE;
 +        break;
 +    case BLK_ZO_CLOSE:
 +        op_name = "BLKCLOSEZONE";
 +        zo = BLKCLOSEZONE;
 +        break;
 +    case BLK_ZO_FINISH:
 +        op_name = "BLKFINISHZONE";
 +        zo = BLKFINISHZONE;
 +        break;
 +    case BLK_ZO_RESET:
 +        op_name = "BLKRESETZONE";
 +        zo = BLKRESETZONE;
 +        break;
 +    default:
 +        error_report("Unsupported zone op: 0x%x", op);
 +        return -ENOTSUP;
 +    }
 +
 +    acb = (RawPosixAIOData) {
 +        .bs             = bs,
 +        .aio_fildes     = s->fd,
 +        .aio_type       = QEMU_AIO_ZONE_MGMT,
 +        .aio_offset     = offset,
 +        .aio_nbytes     = len,
 +        .zone_mgmt  = {
 +            .op = zo,
 +        },
 +    };
 +
 +    ret = raw_thread_pool_submit(bs, handle_aiocb_zone_mgmt, &acb);
 +    if (ret != 0) {
 +        error_report("ioctl %s failed %d", op_name, ret);
 +    }
 +
 +    return ret;
 +}
 +#endif
 +
  static coroutine_fn int
  raw_do_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes,
                  bool blkdev)
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_host_device = {
  #ifdef __linux__
      .bdrv_co_ioctl          = hdev_co_ioctl,
  #endif
 +
 +    /* zoned device */
 +#if defined(CONFIG_BLKZONED)
 +    /* zone management operations */
 +    .bdrv_co_zone_report = raw_co_zone_report,
 +    .bdrv_co_zone_mgmt = raw_co_zone_mgmt,
 +#endif
  };
  #if defined(__linux__) || defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
 diff --git a/block/io.c b/block/io.c
 index XXXXXXX..XXXXXXX 100644
---- a/util/throttle.c
+--- a/block/io.c
-+++ b/util/throttle.c
++++ b/block/io.c
-@@ -XXX,XX +XXX,XX @@ int64_t throttle_compute_wait(LeakyBucket *bkt)
+@@ -XXX,XX +XXX,XX @@ out:
-         /* If bkt->max is 0 we still want to allow short bursts of I/O
+     return co.ret;
-          * from the guest, otherwise every other request will be throttled
+ }
-          * and performance will suffer considerably. */
--        bucket_size = bkt->avg / 10;
++int coroutine_fn bdrv_co_zone_report(BlockDriverState *bs, int64_t offset,
-+        bucket_size = (double) bkt->avg / 10;
++                        unsigned int *nr_zones,
-         burst_bucket_size = 0;
++                        BlockZoneDescriptor *zones)
-     } else {
++{
-         /* If we have a burst limit then we have to wait until all I/O
++    BlockDriver *drv = bs->drv;
-          * at burst rate has finished before throttling to bkt->avg */
++    CoroutineIOCompletion co = {
-         bucket_size = bkt->max * bkt->burst_length;
++            .coroutine = qemu_coroutine_self(),
--        burst_bucket_size = bkt->max / 10;
++    };
-+        burst_bucket_size = (double) bkt->max / 10;
++    IO_CODE();
-     }
++
++    bdrv_inc_in_flight(bs);
-     /* If the main bucket is full then we have to wait */
++    if (!drv || !drv->bdrv_co_zone_report || bs->bl.zoned == BLK_Z_NONE) {
-@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
++        co.ret = -ENOTSUP;
++        goto out;
-     for (i = 0; i < BUCKETS_COUNT; i++) {
++    }
-         LeakyBucket *bkt = &cfg->buckets[i];
++    co.ret = drv->bdrv_co_zone_report(bs, offset, nr_zones, zones);
--        if (bkt->avg < 0 || bkt->max < 0 ||
++out:
--            bkt->avg > THROTTLE_VALUE_MAX || bkt->max > THROTTLE_VALUE_MAX) {
++    bdrv_dec_in_flight(bs);
-+        if (bkt->avg > THROTTLE_VALUE_MAX || bkt->max > THROTTLE_VALUE_MAX) {
++    return co.ret;
-             error_setg(errp, "bps/iops/max values must be within [0, %lld]",
++}
-                        THROTTLE_VALUE_MAX);
++
-             return false;
++int coroutine_fn bdrv_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
 +        int64_t offset, int64_t len)
 +{
 +    BlockDriver *drv = bs->drv;
 +    CoroutineIOCompletion co = {
 +            .coroutine = qemu_coroutine_self(),
 +    };
 +    IO_CODE();
 +
 +    bdrv_inc_in_flight(bs);
 +    if (!drv || !drv->bdrv_co_zone_mgmt || bs->bl.zoned == BLK_Z_NONE) {
 +        co.ret = -ENOTSUP;
 +        goto out;
 +    }
 +    co.ret = drv->bdrv_co_zone_mgmt(bs, op, offset, len);
 +out:
 +    bdrv_dec_in_flight(bs);
 +    return co.ret;
 +}
 +
  void *qemu_blockalign(BlockDriverState *bs, size_t size)
  {
      IO_CODE();
 diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-io-cmds.c
 +++ b/qemu-io-cmds.c
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t flush_cmd = {
      .oneline    = "flush all in-core file state to disk",
  };
 +static inline int64_t tosector(int64_t bytes)
 +{
 +    return bytes >> BDRV_SECTOR_BITS;
 +}
 +
 +static int zone_report_f(BlockBackend *blk, int argc, char **argv)
 +{
 +    int ret;
 +    int64_t offset;
 +    unsigned int nr_zones;
 +
 +    ++optind;
 +    offset = cvtnum(argv[optind]);
 +    ++optind;
 +    nr_zones = cvtnum(argv[optind]);
 +
 +    g_autofree BlockZoneDescriptor *zones = NULL;
 +    zones = g_new(BlockZoneDescriptor, nr_zones);
 +    ret = blk_zone_report(blk, offset, &nr_zones, zones);
 +    if (ret < 0) {
 +        printf("zone report failed: %s\n", strerror(-ret));
 +    } else {
 +        for (int i = 0; i < nr_zones; ++i) {
 +            printf("start: 0x%" PRIx64 ", len 0x%" PRIx64 ", "
 +                   "cap"" 0x%" PRIx64 ", wptr 0x%" PRIx64 ", "
 +                   "zcond:%u, [type: %u]\n",
 +                    tosector(zones[i].start), tosector(zones[i].length),
 +                    tosector(zones[i].cap), tosector(zones[i].wp),
 +                    zones[i].state, zones[i].type);
 +        }
 +    }
 +    return ret;
 +}
 +
 +static const cmdinfo_t zone_report_cmd = {
 +    .name = "zone_report",
 +    .altname = "zrp",
 +    .cfunc = zone_report_f,
 +    .argmin = 2,
 +    .argmax = 2,
 +    .args = "offset number",
 +    .oneline = "report zone information",
 +};
 +
 +static int zone_open_f(BlockBackend *blk, int argc, char **argv)
 +{
 +    int ret;
 +    int64_t offset, len;
 +    ++optind;
 +    offset = cvtnum(argv[optind]);
 +    ++optind;
 +    len = cvtnum(argv[optind]);
 +    ret = blk_zone_mgmt(blk, BLK_ZO_OPEN, offset, len);
 +    if (ret < 0) {
 +        printf("zone open failed: %s\n", strerror(-ret));
 +    }
 +    return ret;
 +}
 +
 +static const cmdinfo_t zone_open_cmd = {
 +    .name = "zone_open",
 +    .altname = "zo",
 +    .cfunc = zone_open_f,
 +    .argmin = 2,
 +    .argmax = 2,
 +    .args = "offset len",
 +    .oneline = "explicit open a range of zones in zone block device",
 +};
 +
 +static int zone_close_f(BlockBackend *blk, int argc, char **argv)
 +{
 +    int ret;
 +    int64_t offset, len;
 +    ++optind;
 +    offset = cvtnum(argv[optind]);
 +    ++optind;
 +    len = cvtnum(argv[optind]);
 +    ret = blk_zone_mgmt(blk, BLK_ZO_CLOSE, offset, len);
 +    if (ret < 0) {
 +        printf("zone close failed: %s\n", strerror(-ret));
 +    }
 +    return ret;
 +}
 +
 +static const cmdinfo_t zone_close_cmd = {
 +    .name = "zone_close",
 +    .altname = "zc",
 +    .cfunc = zone_close_f,
 +    .argmin = 2,
 +    .argmax = 2,
 +    .args = "offset len",
 +    .oneline = "close a range of zones in zone block device",
 +};
 +
 +static int zone_finish_f(BlockBackend *blk, int argc, char **argv)
 +{
 +    int ret;
 +    int64_t offset, len;
 +    ++optind;
 +    offset = cvtnum(argv[optind]);
 +    ++optind;
 +    len = cvtnum(argv[optind]);
 +    ret = blk_zone_mgmt(blk, BLK_ZO_FINISH, offset, len);
 +    if (ret < 0) {
 +        printf("zone finish failed: %s\n", strerror(-ret));
 +    }
 +    return ret;
 +}
 +
 +static const cmdinfo_t zone_finish_cmd = {
 +    .name = "zone_finish",
 +    .altname = "zf",
 +    .cfunc = zone_finish_f,
 +    .argmin = 2,
 +    .argmax = 2,
 +    .args = "offset len",
 +    .oneline = "finish a range of zones in zone block device",
 +};
 +
 +static int zone_reset_f(BlockBackend *blk, int argc, char **argv)
 +{
 +    int ret;
 +    int64_t offset, len;
 +    ++optind;
 +    offset = cvtnum(argv[optind]);
 +    ++optind;
 +    len = cvtnum(argv[optind]);
 +    ret = blk_zone_mgmt(blk, BLK_ZO_RESET, offset, len);
 +    if (ret < 0) {
 +        printf("zone reset failed: %s\n", strerror(-ret));
 +    }
 +    return ret;
 +}
 +
 +static const cmdinfo_t zone_reset_cmd = {
 +    .name = "zone_reset",
 +    .altname = "zrs",
 +    .cfunc = zone_reset_f,
 +    .argmin = 2,
 +    .argmax = 2,
 +    .args = "offset len",
 +    .oneline = "reset a zone write pointer in zone block device",
 +};
 +
  static int truncate_f(BlockBackend *blk, int argc, char **argv);
  static const cmdinfo_t truncate_cmd = {
      .name       = "truncate",
@@ -XXX,XX +XXX,XX @@ static void __attribute((constructor)) init_qemuio_commands(void)
      qemuio_add_command(&aio_write_cmd);
      qemuio_add_command(&aio_flush_cmd);
      qemuio_add_command(&flush_cmd);
 +    qemuio_add_command(&zone_report_cmd);
 +    qemuio_add_command(&zone_open_cmd);
 +    qemuio_add_command(&zone_close_cmd);
 +    qemuio_add_command(&zone_finish_cmd);
 +    qemuio_add_command(&zone_reset_cmd);
      qemuio_add_command(&truncate_cmd);
      qemuio_add_command(&length_cmd);
      qemuio_add_command(&info_cmd);
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 08/15] throttle: Test the valid range of config values
+[PULL 04/20] block/raw-format: add zone operations to pass through requests
-From: Alberto Garcia <berto@igalia.com>
+From: Sam Li <faithilikerun@gmail.com>
-Signed-off-by: Alberto Garcia <berto@igalia.com>
+raw-format driver usually sits on top of file-posix driver. It needs to
-Message-id: a57dd6274e1b6dc9c28769fec4c7ea543be5c5e3.1503580370.git.berto@igalia.com
+pass through requests of zone commands.
 Signed-off-by: Sam Li <faithilikerun@gmail.com>
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
 Reviewed-by: Hannes Reinecke <hare@suse.de>
 Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
 Acked-by: Kevin Wolf <kwolf@redhat.com>
 Message-id: 20230324090605.28361-5-faithilikerun@gmail.com
 [Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
 <philmd@linaro.org>.
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- tests/test-throttle.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++
+ block/raw-format.c | 17 +++++++++++++++++
-file changed, 77 insertions(+)
+file changed, 17 insertions(+)
-diff --git a/tests/test-throttle.c b/tests/test-throttle.c
+diff --git a/block/raw-format.c b/block/raw-format.c
 index XXXXXXX..XXXXXXX 100644
---- a/tests/test-throttle.c
+--- a/block/raw-format.c
-+++ b/tests/test-throttle.c
++++ b/block/raw-format.c
-@@ -XXX,XX +XXX,XX @@ static void test_is_valid(void)
+@@ -XXX,XX +XXX,XX @@ raw_co_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes)
-     test_is_valid_for_value(1, true);
+     return bdrv_co_pdiscard(bs->file, offset, bytes);
  }
-+static void test_ranges(void)
++static int coroutine_fn GRAPH_RDLOCK
 +raw_co_zone_report(BlockDriverState *bs, int64_t offset,
 +                   unsigned int *nr_zones,
 +                   BlockZoneDescriptor *zones)
 +{
-+    int i;
++    return bdrv_co_zone_report(bs->file->bs, offset, nr_zones, zones);
 +
 +    for (i = 0; i < BUCKETS_COUNT; i++) {
 +        LeakyBucket *b = &cfg.buckets[i];
 +        throttle_config_init(&cfg);
 +
 +        /* avg = 0 means throttling is disabled, but the config is valid */
 +        b->avg = 0;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +        g_assert(!throttle_enabled(&cfg));
 +
 +        /* These are valid configurations (values <= THROTTLE_VALUE_MAX) */
 +        b->avg = 1;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = THROTTLE_VALUE_MAX;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = THROTTLE_VALUE_MAX;
 +        b->max = THROTTLE_VALUE_MAX;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +
 +        /* Values over THROTTLE_VALUE_MAX are not allowed */
 +        b->avg = THROTTLE_VALUE_MAX + 1;
 +        g_assert(!throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = THROTTLE_VALUE_MAX;
 +        b->max = THROTTLE_VALUE_MAX + 1;
 +        g_assert(!throttle_is_valid(&cfg, NULL));
 +
 +        /* burst_length must be between 1 and THROTTLE_VALUE_MAX */
 +        b->avg = 1;
 +        b->max = 1;
 +        b->burst_length = 0;
 +        g_assert(!throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = 1;
 +        b->max = 1;
 +        b->burst_length = 1;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = 1;
 +        b->max = 1;
 +        b->burst_length = THROTTLE_VALUE_MAX;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = 1;
 +        b->max = 1;
 +        b->burst_length = THROTTLE_VALUE_MAX + 1;
 +        g_assert(!throttle_is_valid(&cfg, NULL));
 +
 +        /* burst_length * max cannot exceed THROTTLE_VALUE_MAX */
 +        b->avg = 1;
 +        b->max = 2;
 +        b->burst_length = THROTTLE_VALUE_MAX / 2;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = 1;
 +        b->max = 3;
 +        b->burst_length = THROTTLE_VALUE_MAX / 2;
 +        g_assert(!throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = 1;
 +        b->max = THROTTLE_VALUE_MAX;
 +        b->burst_length = 1;
 +        g_assert(throttle_is_valid(&cfg, NULL));
 +
 +        b->avg = 1;
 +        b->max = THROTTLE_VALUE_MAX;
 +        b->burst_length = 2;
 +        g_assert(!throttle_is_valid(&cfg, NULL));
 +    }
 +}
 +
- static void test_max_is_missing_limit(void)
++static int coroutine_fn GRAPH_RDLOCK
 +raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
 +                 int64_t offset, int64_t len)
 +{
 +    return bdrv_co_zone_mgmt(bs->file->bs, op, offset, len);
 +}
 +
  static int64_t coroutine_fn GRAPH_RDLOCK
  raw_co_getlength(BlockDriverState *bs)
  {
-     int i;
+@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_raw = {
-@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
+     .bdrv_co_pwritev      = &raw_co_pwritev,
-     g_test_add_func("/throttle/config/enabled",     test_enabled);
+     .bdrv_co_pwrite_zeroes = &raw_co_pwrite_zeroes,
-     g_test_add_func("/throttle/config/conflicting", test_conflicting_config);
+     .bdrv_co_pdiscard     = &raw_co_pdiscard,
-     g_test_add_func("/throttle/config/is_valid",    test_is_valid);
++    .bdrv_co_zone_report  = &raw_co_zone_report,
-+    g_test_add_func("/throttle/config/ranges",      test_ranges);
++    .bdrv_co_zone_mgmt  = &raw_co_zone_mgmt,
-     g_test_add_func("/throttle/config/max",         test_max_is_missing_limit);
+     .bdrv_co_block_status = &raw_co_block_status,
-     g_test_add_func("/throttle/config/iops_size",
+     .bdrv_co_copy_range_from = &raw_co_copy_range_from,
-                     test_iops_size_is_missing_limit);
+     .bdrv_co_copy_range_to  = &raw_co_copy_range_to,
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 12/15] docker.py: Python 2.6 argparse compatibility
+[PULL 05/20] block: add zoned BlockDriver check to block layer
-Add the scripts/ directory to sys.path so Python 2.6 will be able to
+From: Sam Li <faithilikerun@gmail.com>
 import argparse.
-Cc: Fam Zheng <famz@redhat.com>
+Putting zoned/non-zoned BlockDrivers on top of each other is not
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+allowed.
-Acked-by: John Snow <jsnow@redhat.com>
-Acked-by: Fam Zheng <famz@redhat.com>
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-Message-id: 20170825155732.15665-3-stefanha@redhat.com
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: Hannes Reinecke <hare@suse.de>
 Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
 Acked-by: Kevin Wolf <kwolf@redhat.com>
 Message-id: 20230324090605.28361-6-faithilikerun@gmail.com
 [Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
 <philmd@linaro.org> and clarify that the check is about zoned
 BlockDrivers.
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- tests/docker/docker.py | 4 +++-
+ include/block/block_int-common.h |  5 +++++
-file changed, 3 insertions(+), 1 deletion(-)
+ block.c                          | 19 +++++++++++++++++++
  block/file-posix.c               | 12 ++++++++++++
  block/raw-format.c               |  1 +
 files changed, 37 insertions(+)
-diff --git a/tests/docker/docker.py b/tests/docker/docker.py
+diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
-index XXXXXXX..XXXXXXX 100755
+index XXXXXXX..XXXXXXX 100644
---- a/tests/docker/docker.py
+--- a/include/block/block_int-common.h
-+++ b/tests/docker/docker.py
++++ b/include/block/block_int-common.h
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
+      */
- import os
+     bool is_format;
- import sys
-+sys.path.append(os.path.join(os.path.dirname(__file__),
++    /*
-+                             '..', '..', 'scripts'))
++     * Set to true if the BlockDriver supports zoned children.
-+import argparse
++     */
- import subprocess
++    bool supports_zoned_children;
- import json
++
- import hashlib
+     /*
- import atexit
+      * Drivers not implementing bdrv_parse_filename nor bdrv_open should have
- import uuid
+      * this field set to true, except ones that are defined only by their
--import argparse
+diff --git a/block.c b/block.c
- import tempfile
+index XXXXXXX..XXXXXXX 100644
- import re
+--- a/block.c
- import signal
++++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_add_child(BlockDriverState *parent_bs, BlockDriverState *child_bs,
          return;
      }
 +    /*
 +     * Non-zoned block drivers do not follow zoned storage constraints
 +     * (i.e. sequential writes to zones). Refuse mixing zoned and non-zoned
 +     * drivers in a graph.
 +     */
 +    if (!parent_bs->drv->supports_zoned_children &&
 +        child_bs->bl.zoned == BLK_Z_HM) {
 +        /*
 +         * The host-aware model allows zoned storage constraints and random
 +         * write. Allow mixing host-aware and non-zoned drivers. Using
 +         * host-aware device as a regular device.
 +         */
 +        error_setg(errp, "Cannot add a %s child to a %s parent",
 +                   child_bs->bl.zoned == BLK_Z_HM ? "zoned" : "non-zoned",
 +                   parent_bs->drv->supports_zoned_children ?
 +                   "support zoned children" : "not support zoned children");
 +        return;
 +    }
 +
      if (!QLIST_EMPTY(&child_bs->parents)) {
          error_setg(errp, "The node %s already has a parent",
                     child_bs->node_name);
 diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/file-posix.c
 +++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
              goto fail;
          }
      }
 +#ifdef CONFIG_BLKZONED
 +    /*
 +     * The kernel page cache does not reliably work for writes to SWR zones
 +     * of zoned block device because it can not guarantee the order of writes.
 +     */
 +    if ((bs->bl.zoned != BLK_Z_NONE) &&
 +        (!(s->open_flags & O_DIRECT))) {
 +        error_setg(errp, "The driver supports zoned devices, and it requires "
 +                         "cache.direct=on, which was not specified.");
 +        return -EINVAL; /* No host kernel page cache */
 +    }
 +#endif
      if (S_ISBLK(st.st_mode)) {
  #ifdef __linux__
 diff --git a/block/raw-format.c b/block/raw-format.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/raw-format.c
 +++ b/block/raw-format.c
@@ -XXX,XX +XXX,XX @@ static void raw_child_perm(BlockDriverState *bs, BdrvChild *c,
  BlockDriver bdrv_raw = {
      .format_name          = "raw",
      .instance_size        = sizeof(BDRVRawState),
 +    .supports_zoned_children = true,
      .bdrv_probe           = &raw_probe,
      .bdrv_reopen_prepare  = &raw_reopen_prepare,
      .bdrv_reopen_commit   = &raw_reopen_commit,
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 11/15] scripts: add argparse module for Python 2.6 compatibility
+[PULL 06/20] iotests: test new zone operations
-The minimum Python version supported by QEMU is 2.6.  The argparse
+From: Sam Li <faithilikerun@gmail.com>
 standard library module was only added in Python 2.7.  Many scripts
 would like to use argparse because it supports command-line
 sub-commands.
-This patch adds argparse.  See the top of argparse.py for details.
+The new block layer APIs of zoned block devices can be tested by:
 $ tests/qemu-iotests/check zoned
 Run each zone operation on a newly created null_blk device
 and see whether it outputs the same zone information.
-Suggested-by: Daniel P. Berrange <berrange@redhat.com>
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
-Acked-by: John Snow <jsnow@redhat.com>
+Acked-by: Kevin Wolf <kwolf@redhat.com>
-Message-id: 20170825155732.15665-2-stefanha@redhat.com
+Message-id: 20230324090605.28361-7-faithilikerun@gmail.com
 [Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
 <philmd@linaro.org>.
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- COPYING.PYTHON      |  270 ++++++
+ tests/qemu-iotests/tests/zoned     | 89 ++++++++++++++++++++++++++++++
- scripts/argparse.py | 2406 +++++++++++++++++++++++++++++++++++++++++++++++++++
+ tests/qemu-iotests/tests/zoned.out | 53 ++++++++++++++++++
-files changed, 2676 insertions(+)
+files changed, 142 insertions(+)
- create mode 100644 COPYING.PYTHON
+ create mode 100755 tests/qemu-iotests/tests/zoned
- create mode 100644 scripts/argparse.py
+ create mode 100644 tests/qemu-iotests/tests/zoned.out
-diff --git a/COPYING.PYTHON b/COPYING.PYTHON
+diff --git a/tests/qemu-iotests/tests/zoned b/tests/qemu-iotests/tests/zoned
 new file mode 100755
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/tests/qemu-iotests/tests/zoned
@@ -XXX,XX +XXX,XX @@
 +#!/usr/bin/env bash
 +#
 +# Test zone management operations.
 +#
 +
 +seq="$(basename $0)"
 +echo "QA output created by $seq"
 +status=1 # failure is the default!
 +
 +_cleanup()
 +{
 +  _cleanup_test_img
 +  sudo -n rmmod null_blk
 +}
 +trap "_cleanup; exit \$status" 0 1 2 3 15
 +
 +# get standard environment, filters and checks
 +. ../common.rc
 +. ../common.filter
 +. ../common.qemu
 +
 +# This test only runs on Linux hosts with raw image files.
 +_supported_fmt raw
 +_supported_proto file
 +_supported_os Linux
 +
 +sudo -n true || \
 +    _notrun 'Password-less sudo required'
 +
 +IMG="--image-opts -n driver=host_device,filename=/dev/nullb0"
 +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
 +
 +echo "Testing a null_blk device:"
 +echo "case 1: if the operations work"
 +sudo -n modprobe null_blk nr_devices=1 zoned=1
 +sudo -n chmod 0666 /dev/nullb0
 +
 +echo "(1) report the first zone:"
 +$QEMU_IO $IMG -c "zrp 0 1"
 +echo
 +echo "report the first 10 zones"
 +$QEMU_IO $IMG -c "zrp 0 10"
 +echo
 +echo "report the last zone:"
 +$QEMU_IO $IMG -c "zrp 0x3e70000000 2" # 0x3e70000000 / 512 = 0x1f380000
 +echo
 +echo
 +echo "(2) opening the first zone"
 +$QEMU_IO $IMG -c "zo 0 268435456"  # 268435456 / 512 = 524288
 +echo "report after:"
 +$QEMU_IO $IMG -c "zrp 0 1"
 +echo
 +echo "opening the second zone"
 +$QEMU_IO $IMG -c "zo 268435456 268435456" #
 +echo "report after:"
 +$QEMU_IO $IMG -c "zrp 268435456 1"
 +echo
 +echo "opening the last zone"
 +$QEMU_IO $IMG -c "zo 0x3e70000000 268435456"
 +echo "report after:"
 +$QEMU_IO $IMG -c "zrp 0x3e70000000 2"
 +echo
 +echo
 +echo "(3) closing the first zone"
 +$QEMU_IO $IMG -c "zc 0 268435456"
 +echo "report after:"
 +$QEMU_IO $IMG -c "zrp 0 1"
 +echo
 +echo "closing the last zone"
 +$QEMU_IO $IMG -c "zc 0x3e70000000 268435456"
 +echo "report after:"
 +$QEMU_IO $IMG -c "zrp 0x3e70000000 2"
 +echo
 +echo
 +echo "(4) finishing the second zone"
 +$QEMU_IO $IMG -c "zf 268435456 268435456"
 +echo "After finishing a zone:"
 +$QEMU_IO $IMG -c "zrp 268435456 1"
 +echo
 +echo
 +echo "(5) resetting the second zone"
 +$QEMU_IO $IMG -c "zrs 268435456 268435456"
 +echo "After resetting a zone:"
 +$QEMU_IO $IMG -c "zrp 268435456 1"
 +
 +# success, all done
 +echo "*** done"
 +rm -f $seq.full
 +status=0
 diff --git a/tests/qemu-iotests/tests/zoned.out b/tests/qemu-iotests/tests/zoned.out
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
-+++ b/COPYING.PYTHON
++++ b/tests/qemu-iotests/tests/zoned.out
 @@ -XXX,XX +XXX,XX @@
-+A. HISTORY OF THE SOFTWARE
++QA output created by zoned
-+==========================
++Testing a null_blk device:
 +case 1: if the operations work
 +(1) report the first zone:
 +start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
 +
-+Python was created in the early 1990s by Guido van Rossum at Stichting
++report the first 10 zones
-+Mathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands
++start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
-+as a successor of a language called ABC.  Guido remains Python's
++start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:1, [type: 2]
-+principal author, although it includes many contributions from others.
++start: 0x100000, len 0x80000, cap 0x80000, wptr 0x100000, zcond:1, [type: 2]
 +start: 0x180000, len 0x80000, cap 0x80000, wptr 0x180000, zcond:1, [type: 2]
 +start: 0x200000, len 0x80000, cap 0x80000, wptr 0x200000, zcond:1, [type: 2]
 +start: 0x280000, len 0x80000, cap 0x80000, wptr 0x280000, zcond:1, [type: 2]
 +start: 0x300000, len 0x80000, cap 0x80000, wptr 0x300000, zcond:1, [type: 2]
 +start: 0x380000, len 0x80000, cap 0x80000, wptr 0x380000, zcond:1, [type: 2]
 +start: 0x400000, len 0x80000, cap 0x80000, wptr 0x400000, zcond:1, [type: 2]
 +start: 0x480000, len 0x80000, cap 0x80000, wptr 0x480000, zcond:1, [type: 2]
 +
-+In 1995, Guido continued his work on Python at the Corporation for
++report the last zone:
-+National Research Initiatives (CNRI, see http://www.cnri.reston.va.us)
++start: 0x1f380000, len 0x80000, cap 0x80000, wptr 0x1f380000, zcond:1, [type: 2]
 +in Reston, Virginia where he released several versions of the
 +software.
 +
 +In May 2000, Guido and the Python core development team moved to
 +BeOpen.com to form the BeOpen PythonLabs team.  In October of the same
 +year, the PythonLabs team moved to Digital Creations (now Zope
 +Corporation, see http://www.zope.com).  In 2001, the Python Software
 +Foundation (PSF, see http://www.python.org/psf/) was formed, a
 +non-profit organization created specifically to own Python-related
 +Intellectual Property.  Zope Corporation is a sponsoring member of
 +the PSF.
 +
 +All Python releases are Open Source (see http://www.opensource.org for
 +the Open Source Definition).  Historically, most, but not all, Python
 +releases have also been GPL-compatible; the table below summarizes
 +the various releases.
 +
 +    Release         Derived     Year        Owner       GPL-
 +                    from                                compatible? (1)
 +
 +    0.9.0 thru 1.2              1991-1995   CWI         yes
 +    1.3 thru 1.5.2  1.2         1995-1999   CNRI        yes
 +    1.6             1.5.2       2000        CNRI        no
 +    2.0             1.6         2000        BeOpen.com  no
 +    1.6.1           1.6         2001        CNRI        yes (2)
 +    2.1             2.0+1.6.1   2001        PSF         no
 +    2.0.1           2.0+1.6.1   2001        PSF         yes
 +    2.1.1           2.1+2.0.1   2001        PSF         yes
 +    2.2             2.1.1       2001        PSF         yes
 +    2.1.2           2.1.1       2002        PSF         yes
 +    2.1.3           2.1.2       2002        PSF         yes
 +    2.2.1           2.2         2002        PSF         yes
 +    2.2.2           2.2.1       2002        PSF         yes
 +    2.2.3           2.2.2       2003        PSF         yes
 +    2.3             2.2.2       2002-2003   PSF         yes
 +    2.3.1           2.3         2002-2003   PSF         yes
 +    2.3.2           2.3.1       2002-2003   PSF         yes
 +    2.3.3           2.3.2       2002-2003   PSF         yes
 +    2.3.4           2.3.3       2004        PSF         yes
 +    2.3.5           2.3.4       2005        PSF         yes
 +    2.4             2.3         2004        PSF         yes
 +    2.4.1           2.4         2005        PSF         yes
 +    2.4.2           2.4.1       2005        PSF         yes
 +    2.4.3           2.4.2       2006        PSF         yes
 +    2.5             2.4         2006        PSF         yes
 +    2.7             2.6         2010        PSF         yes
 +
 +Footnotes:
 +
 +(1) GPL-compatible doesn't mean that we're distributing Python under
 +    the GPL.  All Python licenses, unlike the GPL, let you distribute
 +    a modified version without making your changes open source.  The
 +    GPL-compatible licenses make it possible to combine Python with
 +    other software that is released under the GPL; the others don't.
 +
 +(2) According to Richard Stallman, 1.6.1 is not GPL-compatible,
 +    because its license has a choice of law clause.  According to
 +    CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1
 +    is "not incompatible" with the GPL.
 +
 +Thanks to the many outside volunteers who have worked under Guido's
 +direction to make these releases possible.
 +
 +
-+B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
++(2) opening the first zone
-+===============================================================
++report after:
 +start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:3, [type: 2]
 +
-+PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
++opening the second zone
-+--------------------------------------------
++report after:
 +start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:3, [type: 2]
 +
-+1. This LICENSE AGREEMENT is between the Python Software Foundation
++opening the last zone
-+("PSF"), and the Individual or Organization ("Licensee") accessing and
++report after:
-+otherwise using this software ("Python") in source or binary form and
++start: 0x1f380000, len 0x80000, cap 0x80000, wptr 0x1f380000, zcond:3, [type: 2]
 +its associated documentation.
 +
 +2. Subject to the terms and conditions of this License Agreement, PSF
 +hereby grants Licensee a nonexclusive, royalty-free, world-wide
 +license to reproduce, analyze, test, perform and/or display publicly,
 +prepare derivative works, distribute, and otherwise use Python
 +alone or in any derivative version, provided, however, that PSF's
 +License Agreement and PSF's notice of copyright, i.e., "Copyright (c)
 +2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation; All Rights
 +Reserved" are retained in Python alone or in any derivative version
 +prepared by Licensee.
 +
 +3. In the event Licensee prepares a derivative work that is based on
 +or incorporates Python or any part thereof, and wants to make
 +the derivative work available to others as provided herein, then
 +Licensee hereby agrees to include in any such work a brief summary of
 +the changes made to Python.
 +
 +4. PSF is making Python available to Licensee on an "AS IS"
 +basis.  PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
 +IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
 +DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
 +FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
 +INFRINGE ANY THIRD PARTY RIGHTS.
 +
 +5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
 +FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
 +A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
 +OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
 +
 +6. This License Agreement will automatically terminate upon a material
 +breach of its terms and conditions.
 +
 +7. Nothing in this License Agreement shall be deemed to create any
 +relationship of agency, partnership, or joint venture between PSF and
 +Licensee.  This License Agreement does not grant permission to use PSF
 +trademarks or trade name in a trademark sense to endorse or promote
 +products or services of Licensee, or any third party.
 +
 +8. By copying, installing or otherwise using Python, Licensee
 +agrees to be bound by the terms and conditions of this License
 +Agreement.
 +
 +
-+BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
++(3) closing the first zone
-+-------------------------------------------
++report after:
 +start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
 +
-+BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
++closing the last zone
-+
++report after:
-+1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
++start: 0x1f380000, len 0x80000, cap 0x80000, wptr 0x1f380000, zcond:1, [type: 2]
 +office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
 +Individual or Organization ("Licensee") accessing and otherwise using
 +this software in source or binary form and its associated
 +documentation ("the Software").
 +
 +2. Subject to the terms and conditions of this BeOpen Python License
 +Agreement, BeOpen hereby grants Licensee a non-exclusive,
 +royalty-free, world-wide license to reproduce, analyze, test, perform
 +and/or display publicly, prepare derivative works, distribute, and
 +otherwise use the Software alone or in any derivative version,
 +provided, however, that the BeOpen Python License is retained in the
 +Software, alone or in any derivative version prepared by Licensee.
 +
 +3. BeOpen is making the Software available to Licensee on an "AS IS"
 +basis.  BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
 +IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
 +DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
 +FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
 +INFRINGE ANY THIRD PARTY RIGHTS.
 +
 +4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
 +SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
 +AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
 +DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
 +
 +5. This License Agreement will automatically terminate upon a material
 +breach of its terms and conditions.
 +
 +6. This License Agreement shall be governed by and interpreted in all
 +respects by the law of the State of California, excluding conflict of
 +law provisions.  Nothing in this License Agreement shall be deemed to
 +create any relationship of agency, partnership, or joint venture
 +between BeOpen and Licensee.  This License Agreement does not grant
 +permission to use BeOpen trademarks or trade names in a trademark
 +sense to endorse or promote products or services of Licensee, or any
 +third party.  As an exception, the "BeOpen Python" logos available at
 +http://www.pythonlabs.com/logos.html may be used according to the
 +permissions granted on that web page.
 +
 +7. By copying, installing or otherwise using the software, Licensee
 +agrees to be bound by the terms and conditions of this License
 +Agreement.
 +
 +
-+CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
++(4) finishing the second zone
-+---------------------------------------
++After finishing a zone:
-+
++start: 0x80000, len 0x80000, cap 0x80000, wptr 0x100000, zcond:14, [type: 2]
 +1. This LICENSE AGREEMENT is between the Corporation for National
 +Research Initiatives, having an office at 1895 Preston White Drive,
 +Reston, VA 20191 ("CNRI"), and the Individual or Organization
 +("Licensee") accessing and otherwise using Python 1.6.1 software in
 +source or binary form and its associated documentation.
 +
 +2. Subject to the terms and conditions of this License Agreement, CNRI
 +hereby grants Licensee a nonexclusive, royalty-free, world-wide
 +license to reproduce, analyze, test, perform and/or display publicly,
 +prepare derivative works, distribute, and otherwise use Python 1.6.1
 +alone or in any derivative version, provided, however, that CNRI's
 +License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
 +1995-2001 Corporation for National Research Initiatives; All Rights
 +Reserved" are retained in Python 1.6.1 alone or in any derivative
 +version prepared by Licensee.  Alternately, in lieu of CNRI's License
 +Agreement, Licensee may substitute the following text (omitting the
 +quotes): "Python 1.6.1 is made available subject to the terms and
 +conditions in CNRI's License Agreement.  This Agreement together with
 +Python 1.6.1 may be located on the Internet using the following
 +unique, persistent identifier (known as a handle): 1895.22/1013.  This
 +Agreement may also be obtained from a proxy server on the Internet
 +using the following URL: http://hdl.handle.net/1895.22/1013".
 +
 +3. In the event Licensee prepares a derivative work that is based on
 +or incorporates Python 1.6.1 or any part thereof, and wants to make
 +the derivative work available to others as provided herein, then
 +Licensee hereby agrees to include in any such work a brief summary of
 +the changes made to Python 1.6.1.
 +
 +4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
 +basis.  CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
 +IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
 +DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
 +FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
 +INFRINGE ANY THIRD PARTY RIGHTS.
 +
 +5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
 +1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
 +A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
 +OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
 +
 +6. This License Agreement will automatically terminate upon a material
 +breach of its terms and conditions.
 +
 +7. This License Agreement shall be governed by the federal
 +intellectual property law of the United States, including without
 +limitation the federal copyright law, and, to the extent such
 +U.S. federal law does not apply, by the law of the Commonwealth of
 +Virginia, excluding Virginia's conflict of law provisions.
 +Notwithstanding the foregoing, with regard to derivative works based
 +on Python 1.6.1 that incorporate non-separable material that was
 +previously distributed under the GNU General Public License (GPL), the
 +law of the Commonwealth of Virginia shall govern this License
 +Agreement only as to issues arising under or with respect to
 +Paragraphs 4, 5, and 7 of this License Agreement.  Nothing in this
 +License Agreement shall be deemed to create any relationship of
 +agency, partnership, or joint venture between CNRI and Licensee.  This
 +License Agreement does not grant permission to use CNRI trademarks or
 +trade name in a trademark sense to endorse or promote products or
 +services of Licensee, or any third party.
 +
 +8. By clicking on the "ACCEPT" button where indicated, or by copying,
 +installing or otherwise using Python 1.6.1, Licensee agrees to be
 +bound by the terms and conditions of this License Agreement.
 +
 +        ACCEPT
 +
 +
-+CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
++(5) resetting the second zone
-+--------------------------------------------------
++After resetting a zone:
-+
++start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:1, [type: 2]
-+Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
++*** done
 +The Netherlands.  All rights reserved.
 +
 +Permission to use, copy, modify, and distribute this software and its
 +documentation for any purpose and without fee is hereby granted,
 +provided that the above copyright notice appear in all copies and that
 +both that copyright notice and this permission notice appear in
 +supporting documentation, and that the name of Stichting Mathematisch
 +Centrum or CWI not be used in advertising or publicity pertaining to
 +distribution of the software without specific, written prior
 +permission.
 +
 +STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
 +THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
 +FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
 +FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
 +OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 diff --git a/scripts/argparse.py b/scripts/argparse.py
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/scripts/argparse.py
@@ -XXX,XX +XXX,XX @@
 +# This is a local copy of the standard library argparse module taken from PyPI.
 +# It is licensed under the Python Software Foundation License.  This is a
 +# fallback for Python 2.6 which does not include this module.  Python 2.7+ and
 +# 3+ will never load this module because built-in modules are loaded before
 +# anything in sys.path.
 +#
 +# If your script is not located in the same directory as this file, import it
 +# like this:
 +#
 +#   import os
 +#   import sys
 +#   sys.path.append(os.path.join(os.path.dirname(__file__), ..., 'scripts'))
 +#   import argparse
 +
 +# Author: Steven J. Bethard <steven.bethard@gmail.com>.
 +# Maintainer: Thomas Waldmann <tw@waldmann-edv.de>
 +
 +"""Command-line parsing library
 +
 +This module is an optparse-inspired command-line parsing library that:
 +
 +    - handles both optional and positional arguments
 +    - produces highly informative usage messages
 +    - supports parsers that dispatch to sub-parsers
 +
 +The following is a simple usage example that sums integers from the
 +command-line and writes the result to a file::
 +
 +    parser = argparse.ArgumentParser(
 +        description='sum the integers at the command line')
 +    parser.add_argument(
 +        'integers', metavar='int', nargs='+', type=int,
 +        help='an integer to be summed')
 +    parser.add_argument(
 +        '--log', default=sys.stdout, type=argparse.FileType('w'),
 +        help='the file where the sum should be written')
 +    args = parser.parse_args()
 +    args.log.write('%s' % sum(args.integers))
 +    args.log.close()
 +
 +The module contains the following public classes:
 +
 +    - ArgumentParser -- The main entry point for command-line parsing. As the
 +        example above shows, the add_argument() method is used to populate
 +        the parser with actions for optional and positional arguments. Then
 +        the parse_args() method is invoked to convert the args at the
 +        command-line into an object with attributes.
 +
 +    - ArgumentError -- The exception raised by ArgumentParser objects when
 +        there are errors with the parser's actions. Errors raised while
 +        parsing the command-line are caught by ArgumentParser and emitted
 +        as command-line messages.
 +
 +    - FileType -- A factory for defining types of files to be created. As the
 +        example above shows, instances of FileType are typically passed as
 +        the type= argument of add_argument() calls.
 +
 +    - Action -- The base class for parser actions. Typically actions are
 +        selected by passing strings like 'store_true' or 'append_const' to
 +        the action= argument of add_argument(). However, for greater
 +        customization of ArgumentParser actions, subclasses of Action may
 +        be defined and passed as the action= argument.
 +
 +    - HelpFormatter, RawDescriptionHelpFormatter, RawTextHelpFormatter,
 +        ArgumentDefaultsHelpFormatter -- Formatter classes which
 +        may be passed as the formatter_class= argument to the
 +        ArgumentParser constructor. HelpFormatter is the default,
 +        RawDescriptionHelpFormatter and RawTextHelpFormatter tell the parser
 +        not to change the formatting for help text, and
 +        ArgumentDefaultsHelpFormatter adds information about argument defaults
 +        to the help.
 +
 +All other classes in this module are considered implementation details.
 +(Also note that HelpFormatter and RawDescriptionHelpFormatter are only
 +considered public as object names -- the API of the formatter objects is
 +still considered an implementation detail.)
 +"""
 +
 +__version__ = '1.4.0'  # we use our own version number independant of the
 +                       # one in stdlib and we release this on pypi.
 +
 +__external_lib__ = True  # to make sure the tests really test THIS lib,
 +                         # not the builtin one in Python stdlib
 +
 +__all__ = [
 +    'ArgumentParser',
 +    'ArgumentError',
 +    'ArgumentTypeError',
 +    'FileType',
 +    'HelpFormatter',
 +    'ArgumentDefaultsHelpFormatter',
 +    'RawDescriptionHelpFormatter',
 +    'RawTextHelpFormatter',
 +    'Namespace',
 +    'Action',
 +    'ONE_OR_MORE',
 +    'OPTIONAL',
 +    'PARSER',
 +    'REMAINDER',
 +    'SUPPRESS',
 +    'ZERO_OR_MORE',
 +]
 +
 +
 +import copy as _copy
 +import os as _os
 +import re as _re
 +import sys as _sys
 +import textwrap as _textwrap
 +
 +from gettext import gettext as _
 +
 +try:
 +    set
 +except NameError:
 +    # for python < 2.4 compatibility (sets module is there since 2.3):
 +    from sets import Set as set
 +
 +try:
 +    basestring
 +except NameError:
 +    basestring = str
 +
 +try:
 +    sorted
 +except NameError:
 +    # for python < 2.4 compatibility:
 +    def sorted(iterable, reverse=False):
 +        result = list(iterable)
 +        result.sort()
 +        if reverse:
 +            result.reverse()
 +        return result
 +
 +
 +def _callable(obj):
 +    return hasattr(obj, '__call__') or hasattr(obj, '__bases__')
 +
 +
 +SUPPRESS = '==SUPPRESS=='
 +
 +OPTIONAL = '?'
 +ZERO_OR_MORE = '*'
 +ONE_OR_MORE = '+'
 +PARSER = 'A...'
 +REMAINDER = '...'
 +_UNRECOGNIZED_ARGS_ATTR = '_unrecognized_args'
 +
 +# =============================
 +# Utility functions and classes
 +# =============================
 +
 +class _AttributeHolder(object):
 +    """Abstract base class that provides __repr__.
 +
 +    The __repr__ method returns a string in the format::
 +        ClassName(attr=name, attr=name, ...)
 +    The attributes are determined either by a class-level attribute,
 +    '_kwarg_names', or by inspecting the instance __dict__.
 +    """
 +
 +    def __repr__(self):
 +        type_name = type(self).__name__
 +        arg_strings = []
 +        for arg in self._get_args():
 +            arg_strings.append(repr(arg))
 +        for name, value in self._get_kwargs():
 +            arg_strings.append('%s=%r' % (name, value))
 +        return '%s(%s)' % (type_name, ', '.join(arg_strings))
 +
 +    def _get_kwargs(self):
 +        return sorted(self.__dict__.items())
 +
 +    def _get_args(self):
 +        return []
 +
 +
 +def _ensure_value(namespace, name, value):
 +    if getattr(namespace, name, None) is None:
 +        setattr(namespace, name, value)
 +    return getattr(namespace, name)
 +
 +
 +# ===============
 +# Formatting Help
 +# ===============
 +
 +class HelpFormatter(object):
 +    """Formatter for generating usage messages and argument help strings.
 +
 +    Only the name of this class is considered a public API. All the methods
 +    provided by the class are considered an implementation detail.
 +    """
 +
 +    def __init__(self,
 +                 prog,
 +                 indent_increment=2,
 +                 max_help_position=24,
 +                 width=None):
 +
 +        # default setting for width
 +        if width is None:
 +            try:
 +                width = int(_os.environ['COLUMNS'])
 +            except (KeyError, ValueError):
 +                width = 80
 +            width -= 2
 +
 +        self._prog = prog
 +        self._indent_increment = indent_increment
 +        self._max_help_position = max_help_position
 +        self._width = width
 +
 +        self._current_indent = 0
 +        self._level = 0
 +        self._action_max_length = 0
 +
 +        self._root_section = self._Section(self, None)
 +        self._current_section = self._root_section
 +
 +        self._whitespace_matcher = _re.compile(r'\s+')
 +        self._long_break_matcher = _re.compile(r'\n\n\n+')
 +
 +    # ===============================
 +    # Section and indentation methods
 +    # ===============================
 +    def _indent(self):
 +        self._current_indent += self._indent_increment
 +        self._level += 1
 +
 +    def _dedent(self):
 +        self._current_indent -= self._indent_increment
 +        assert self._current_indent >= 0, 'Indent decreased below 0.'
 +        self._level -= 1
 +
 +    class _Section(object):
 +
 +        def __init__(self, formatter, parent, heading=None):
 +            self.formatter = formatter
 +            self.parent = parent
 +            self.heading = heading
 +            self.items = []
 +
 +        def format_help(self):
 +            # format the indented section
 +            if self.parent is not None:
 +                self.formatter._indent()
 +            join = self.formatter._join_parts
 +            for func, args in self.items:
 +                func(*args)
 +            item_help = join([func(*args) for func, args in self.items])
 +            if self.parent is not None:
 +                self.formatter._dedent()
 +
 +            # return nothing if the section was empty
 +            if not item_help:
 +                return ''
 +
 +            # add the heading if the section was non-empty
 +            if self.heading is not SUPPRESS and self.heading is not None:
 +                current_indent = self.formatter._current_indent
 +                heading = '%*s%s:\n' % (current_indent, '', self.heading)
 +            else:
 +                heading = ''
 +
 +            # join the section-initial newline, the heading and the help
 +            return join(['\n', heading, item_help, '\n'])
 +
 +    def _add_item(self, func, args):
 +        self._current_section.items.append((func, args))
 +
 +    # ========================
 +    # Message building methods
 +    # ========================
 +    def start_section(self, heading):
 +        self._indent()
 +        section = self._Section(self, self._current_section, heading)
 +        self._add_item(section.format_help, [])
 +        self._current_section = section
 +
 +    def end_section(self):
 +        self._current_section = self._current_section.parent
 +        self._dedent()
 +
 +    def add_text(self, text):
 +        if text is not SUPPRESS and text is not None:
 +            self._add_item(self._format_text, [text])
 +
 +    def add_usage(self, usage, actions, groups, prefix=None):
 +        if usage is not SUPPRESS:
 +            args = usage, actions, groups, prefix
 +            self._add_item(self._format_usage, args)
 +
 +    def add_argument(self, action):
 +        if action.help is not SUPPRESS:
 +
 +            # find all invocations
 +            get_invocation = self._format_action_invocation
 +            invocations = [get_invocation(action)]
 +            for subaction in self._iter_indented_subactions(action):
 +                invocations.append(get_invocation(subaction))
 +
 +            # update the maximum item length
 +            invocation_length = max([len(s) for s in invocations])
 +            action_length = invocation_length + self._current_indent
 +            self._action_max_length = max(self._action_max_length,
 +                                          action_length)
 +
 +            # add the item to the list
 +            self._add_item(self._format_action, [action])
 +
 +    def add_arguments(self, actions):
 +        for action in actions:
 +            self.add_argument(action)
 +
 +    # =======================
 +    # Help-formatting methods
 +    # =======================
 +    def format_help(self):
 +        help = self._root_section.format_help()
 +        if help:
 +            help = self._long_break_matcher.sub('\n\n', help)
 +            help = help.strip('\n') + '\n'
 +        return help
 +
 +    def _join_parts(self, part_strings):
 +        return ''.join([part
 +                        for part in part_strings
 +                        if part and part is not SUPPRESS])
 +
 +    def _format_usage(self, usage, actions, groups, prefix):
 +        if prefix is None:
 +            prefix = _('usage: ')
 +
 +        # if usage is specified, use that
 +        if usage is not None:
 +            usage = usage % dict(prog=self._prog)
 +
 +        # if no optionals or positionals are available, usage is just prog
 +        elif usage is None and not actions:
 +            usage = '%(prog)s' % dict(prog=self._prog)
 +
 +        # if optionals and positionals are available, calculate usage
 +        elif usage is None:
 +            prog = '%(prog)s' % dict(prog=self._prog)
 +
 +            # split optionals from positionals
 +            optionals = []
 +            positionals = []
 +            for action in actions:
 +                if action.option_strings:
 +                    optionals.append(action)
 +                else:
 +                    positionals.append(action)
 +
 +            # build full usage string
 +            format = self._format_actions_usage
 +            action_usage = format(optionals + positionals, groups)
 +            usage = ' '.join([s for s in [prog, action_usage] if s])
 +
 +            # wrap the usage parts if it's too long
 +            text_width = self._width - self._current_indent
 +            if len(prefix) + len(usage) > text_width:
 +
 +                # break usage into wrappable parts
 +                part_regexp = r'\(.*?\)+|\[.*?\]+|\S+'
 +                opt_usage = format(optionals, groups)
 +                pos_usage = format(positionals, groups)
 +                opt_parts = _re.findall(part_regexp, opt_usage)
 +                pos_parts = _re.findall(part_regexp, pos_usage)
 +                assert ' '.join(opt_parts) == opt_usage
 +                assert ' '.join(pos_parts) == pos_usage
 +
 +                # helper for wrapping lines
 +                def get_lines(parts, indent, prefix=None):
 +                    lines = []
 +                    line = []
 +                    if prefix is not None:
 +                        line_len = len(prefix) - 1
 +                    else:
 +                        line_len = len(indent) - 1
 +                    for part in parts:
 +                        if line_len + 1 + len(part) > text_width:
 +                            lines.append(indent + ' '.join(line))
 +                            line = []
 +                            line_len = len(indent) - 1
 +                        line.append(part)
 +                        line_len += len(part) + 1
 +                    if line:
 +                        lines.append(indent + ' '.join(line))
 +                    if prefix is not None:
 +                        lines[0] = lines[0][len(indent):]
 +                    return lines
 +
 +                # if prog is short, follow it with optionals or positionals
 +                if len(prefix) + len(prog) <= 0.75 * text_width:
 +                    indent = ' ' * (len(prefix) + len(prog) + 1)
 +                    if opt_parts:
 +                        lines = get_lines([prog] + opt_parts, indent, prefix)
 +                        lines.extend(get_lines(pos_parts, indent))
 +                    elif pos_parts:
 +                        lines = get_lines([prog] + pos_parts, indent, prefix)
 +                    else:
 +                        lines = [prog]
 +
 +                # if prog is long, put it on its own line
 +                else:
 +                    indent = ' ' * len(prefix)
 +                    parts = opt_parts + pos_parts
 +                    lines = get_lines(parts, indent)
 +                    if len(lines) > 1:
 +                        lines = []
 +                        lines.extend(get_lines(opt_parts, indent))
 +                        lines.extend(get_lines(pos_parts, indent))
 +                    lines = [prog] + lines
 +
 +                # join lines into usage
 +                usage = '\n'.join(lines)
 +
 +        # prefix with 'usage:'
 +        return '%s%s\n\n' % (prefix, usage)
 +
 +    def _format_actions_usage(self, actions, groups):
 +        # find group indices and identify actions in groups
 +        group_actions = set()
 +        inserts = {}
 +        for group in groups:
 +            try:
 +                start = actions.index(group._group_actions[0])
 +            except ValueError:
 +                continue
 +            else:
 +                end = start + len(group._group_actions)
 +                if actions[start:end] == group._group_actions:
 +                    for action in group._group_actions:
 +                        group_actions.add(action)
 +                    if not group.required:
 +                        if start in inserts:
 +                            inserts[start] += ' ['
 +                        else:
 +                            inserts[start] = '['
 +                        inserts[end] = ']'
 +                    else:
 +                        if start in inserts:
 +                            inserts[start] += ' ('
 +                        else:
 +                            inserts[start] = '('
 +                        inserts[end] = ')'
 +                    for i in range(start + 1, end):
 +                        inserts[i] = '|'
 +
 +        # collect all actions format strings
 +        parts = []
 +        for i, action in enumerate(actions):
 +
 +            # suppressed arguments are marked with None
 +            # remove | separators for suppressed arguments
 +            if action.help is SUPPRESS:
 +                parts.append(None)
 +                if inserts.get(i) == '|':
 +                    inserts.pop(i)
 +                elif inserts.get(i + 1) == '|':
 +                    inserts.pop(i + 1)
 +
 +            # produce all arg strings
 +            elif not action.option_strings:
 +                part = self._format_args(action, action.dest)
 +
 +                # if it's in a group, strip the outer []
 +                if action in group_actions:
 +                    if part[0] == '[' and part[-1] == ']':
 +                        part = part[1:-1]
 +
 +                # add the action string to the list
 +                parts.append(part)
 +
 +            # produce the first way to invoke the option in brackets
 +            else:
 +                option_string = action.option_strings[0]
 +
 +                # if the Optional doesn't take a value, format is:
 +                #    -s or --long
 +                if action.nargs == 0:
 +                    part = '%s' % option_string
 +
 +                # if the Optional takes a value, format is:
 +                #    -s ARGS or --long ARGS
 +                else:
 +                    default = action.dest.upper()
 +                    args_string = self._format_args(action, default)
 +                    part = '%s %s' % (option_string, args_string)
 +
 +                # make it look optional if it's not required or in a group
 +                if not action.required and action not in group_actions:
 +                    part = '[%s]' % part
 +
 +                # add the action string to the list
 +                parts.append(part)
 +
 +        # insert things at the necessary indices
 +        for i in sorted(inserts, reverse=True):
 +            parts[i:i] = [inserts[i]]
 +
 +        # join all the action items with spaces
 +        text = ' '.join([item for item in parts if item is not None])
 +
 +        # clean up separators for mutually exclusive groups
 +        open = r'[\[(]'
 +        close = r'[\])]'
 +        text = _re.sub(r'(%s) ' % open, r'\1', text)
 +        text = _re.sub(r' (%s)' % close, r'\1', text)
 +        text = _re.sub(r'%s *%s' % (open, close), r'', text)
 +        text = _re.sub(r'\(([^|]*)\)', r'\1', text)
 +        text = text.strip()
 +
 +        # return the text
 +        return text
 +
 +    def _format_text(self, text):
 +        if '%(prog)' in text:
 +            text = text % dict(prog=self._prog)
 +        text_width = self._width - self._current_indent
 +        indent = ' ' * self._current_indent
 +        return self._fill_text(text, text_width, indent) + '\n\n'
 +
 +    def _format_action(self, action):
 +        # determine the required width and the entry label
 +        help_position = min(self._action_max_length + 2,
 +                            self._max_help_position)
 +        help_width = self._width - help_position
 +        action_width = help_position - self._current_indent - 2
 +        action_header = self._format_action_invocation(action)
 +
 +        # ho nelp; start on same line and add a final newline
 +        if not action.help:
 +            tup = self._current_indent, '', action_header
 +            action_header = '%*s%s\n' % tup
 +
 +        # short action name; start on the same line and pad two spaces
 +        elif len(action_header) <= action_width:
 +            tup = self._current_indent, '', action_width, action_header
 +            action_header = '%*s%-*s  ' % tup
 +            indent_first = 0
 +
 +        # long action name; start on the next line
 +        else:
 +            tup = self._current_indent, '', action_header
 +            action_header = '%*s%s\n' % tup
 +            indent_first = help_position
 +
 +        # collect the pieces of the action help
 +        parts = [action_header]
 +
 +        # if there was help for the action, add lines of help text
 +        if action.help:
 +            help_text = self._expand_help(action)
 +            help_lines = self._split_lines(help_text, help_width)
 +            parts.append('%*s%s\n' % (indent_first, '', help_lines[0]))
 +            for line in help_lines[1:]:
 +                parts.append('%*s%s\n' % (help_position, '', line))
 +
 +        # or add a newline if the description doesn't end with one
 +        elif not action_header.endswith('\n'):
 +            parts.append('\n')
 +
 +        # if there are any sub-actions, add their help as well
 +        for subaction in self._iter_indented_subactions(action):
 +            parts.append(self._format_action(subaction))
 +
 +        # return a single string
 +        return self._join_parts(parts)
 +
 +    def _format_action_invocation(self, action):
 +        if not action.option_strings:
 +            metavar, = self._metavar_formatter(action, action.dest)(1)
 +            return metavar
 +
 +        else:
 +            parts = []
 +
 +            # if the Optional doesn't take a value, format is:
 +            #    -s, --long
 +            if action.nargs == 0:
 +                parts.extend(action.option_strings)
 +
 +            # if the Optional takes a value, format is:
 +            #    -s ARGS, --long ARGS
 +            else:
 +                default = action.dest.upper()
 +                args_string = self._format_args(action, default)
 +                for option_string in action.option_strings:
 +                    parts.append('%s %s' % (option_string, args_string))
 +
 +            return ', '.join(parts)
 +
 +    def _metavar_formatter(self, action, default_metavar):
 +        if action.metavar is not None:
 +            result = action.metavar
 +        elif action.choices is not None:
 +            choice_strs = [str(choice) for choice in action.choices]
 +            result = '{%s}' % ','.join(choice_strs)
 +        else:
 +            result = default_metavar
 +
 +        def format(tuple_size):
 +            if isinstance(result, tuple):
 +                return result
 +            else:
 +                return (result, ) * tuple_size
 +        return format
 +
 +    def _format_args(self, action, default_metavar):
 +        get_metavar = self._metavar_formatter(action, default_metavar)
 +        if action.nargs is None:
 +            result = '%s' % get_metavar(1)
 +        elif action.nargs == OPTIONAL:
 +            result = '[%s]' % get_metavar(1)
 +        elif action.nargs == ZERO_OR_MORE:
 +            result = '[%s [%s ...]]' % get_metavar(2)
 +        elif action.nargs == ONE_OR_MORE:
 +            result = '%s [%s ...]' % get_metavar(2)
 +        elif action.nargs == REMAINDER:
 +            result = '...'
 +        elif action.nargs == PARSER:
 +            result = '%s ...' % get_metavar(1)
 +        else:
 +            formats = ['%s' for _ in range(action.nargs)]
 +            result = ' '.join(formats) % get_metavar(action.nargs)
 +        return result
 +
 +    def _expand_help(self, action):
 +        params = dict(vars(action), prog=self._prog)
 +        for name in list(params):
 +            if params[name] is SUPPRESS:
 +                del params[name]
 +        for name in list(params):
 +            if hasattr(params[name], '__name__'):
 +                params[name] = params[name].__name__
 +        if params.get('choices') is not None:
 +            choices_str = ', '.join([str(c) for c in params['choices']])
 +            params['choices'] = choices_str
 +        return self._get_help_string(action) % params
 +
 +    def _iter_indented_subactions(self, action):
 +        try:
 +            get_subactions = action._get_subactions
 +        except AttributeError:
 +            pass
 +        else:
 +            self._indent()
 +            for subaction in get_subactions():
 +                yield subaction
 +            self._dedent()
 +
 +    def _split_lines(self, text, width):
 +        text = self._whitespace_matcher.sub(' ', text).strip()
 +        return _textwrap.wrap(text, width)
 +
 +    def _fill_text(self, text, width, indent):
 +        text = self._whitespace_matcher.sub(' ', text).strip()
 +        return _textwrap.fill(text, width, initial_indent=indent,
 +                                           subsequent_indent=indent)
 +
 +    def _get_help_string(self, action):
 +        return action.help
 +
 +
 +class RawDescriptionHelpFormatter(HelpFormatter):
 +    """Help message formatter which retains any formatting in descriptions.
 +
 +    Only the name of this class is considered a public API. All the methods
 +    provided by the class are considered an implementation detail.
 +    """
 +
 +    def _fill_text(self, text, width, indent):
 +        return ''.join([indent + line for line in text.splitlines(True)])
 +
 +
 +class RawTextHelpFormatter(RawDescriptionHelpFormatter):
 +    """Help message formatter which retains formatting of all help text.
 +
 +    Only the name of this class is considered a public API. All the methods
 +    provided by the class are considered an implementation detail.
 +    """
 +
 +    def _split_lines(self, text, width):
 +        return text.splitlines()
 +
 +
 +class ArgumentDefaultsHelpFormatter(HelpFormatter):
 +    """Help message formatter which adds default values to argument help.
 +
 +    Only the name of this class is considered a public API. All the methods
 +    provided by the class are considered an implementation detail.
 +    """
 +
 +    def _get_help_string(self, action):
 +        help = action.help
 +        if '%(default)' not in action.help:
 +            if action.default is not SUPPRESS:
 +                defaulting_nargs = [OPTIONAL, ZERO_OR_MORE]
 +                if action.option_strings or action.nargs in defaulting_nargs:
 +                    help += ' (default: %(default)s)'
 +        return help
 +
 +
 +# =====================
 +# Options and Arguments
 +# =====================
 +
 +def _get_action_name(argument):
 +    if argument is None:
 +        return None
 +    elif argument.option_strings:
 +        return  '/'.join(argument.option_strings)
 +    elif argument.metavar not in (None, SUPPRESS):
 +        return argument.metavar
 +    elif argument.dest not in (None, SUPPRESS):
 +        return argument.dest
 +    else:
 +        return None
 +
 +
 +class ArgumentError(Exception):
 +    """An error from creating or using an argument (optional or positional).
 +
 +    The string value of this exception is the message, augmented with
 +    information about the argument that caused it.
 +    """
 +
 +    def __init__(self, argument, message):
 +        self.argument_name = _get_action_name(argument)
 +        self.message = message
 +
 +    def __str__(self):
 +        if self.argument_name is None:
 +            format = '%(message)s'
 +        else:
 +            format = 'argument %(argument_name)s: %(message)s'
 +        return format % dict(message=self.message,
 +                             argument_name=self.argument_name)
 +
 +
 +class ArgumentTypeError(Exception):
 +    """An error from trying to convert a command line string to a type."""
 +    pass
 +
 +
 +# ==============
 +# Action classes
 +# ==============
 +
 +class Action(_AttributeHolder):
 +    """Information about how to convert command line strings to Python objects.
 +
 +    Action objects are used by an ArgumentParser to represent the information
 +    needed to parse a single argument from one or more strings from the
 +    command line. The keyword arguments to the Action constructor are also
 +    all attributes of Action instances.
 +
 +    Keyword Arguments:
 +
 +        - option_strings -- A list of command-line option strings which
 +            should be associated with this action.
 +
 +        - dest -- The name of the attribute to hold the created object(s)
 +
 +        - nargs -- The number of command-line arguments that should be
 +            consumed. By default, one argument will be consumed and a single
 +            value will be produced.  Other values include:
 +                - N (an integer) consumes N arguments (and produces a list)
 +                - '?' consumes zero or one arguments
 +                - '*' consumes zero or more arguments (and produces a list)
 +                - '+' consumes one or more arguments (and produces a list)
 +            Note that the difference between the default and nargs=1 is that
 +            with the default, a single value will be produced, while with
 +            nargs=1, a list containing a single value will be produced.
 +
 +        - const -- The value to be produced if the option is specified and the
 +            option uses an action that takes no values.
 +
 +        - default -- The value to be produced if the option is not specified.
 +
 +        - type -- The type which the command-line arguments should be converted
 +            to, should be one of 'string', 'int', 'float', 'complex' or a
 +            callable object that accepts a single string argument. If None,
 +            'string' is assumed.
 +
 +        - choices -- A container of values that should be allowed. If not None,
 +            after a command-line argument has been converted to the appropriate
 +            type, an exception will be raised if it is not a member of this
 +            collection.
 +
 +        - required -- True if the action must always be specified at the
 +            command line. This is only meaningful for optional command-line
 +            arguments.
 +
 +        - help -- The help string describing the argument.
 +
 +        - metavar -- The name to be used for the option's argument with the
 +            help string. If None, the 'dest' value will be used as the name.
 +    """
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 nargs=None,
 +                 const=None,
 +                 default=None,
 +                 type=None,
 +                 choices=None,
 +                 required=False,
 +                 help=None,
 +                 metavar=None):
 +        self.option_strings = option_strings
 +        self.dest = dest
 +        self.nargs = nargs
 +        self.const = const
 +        self.default = default
 +        self.type = type
 +        self.choices = choices
 +        self.required = required
 +        self.help = help
 +        self.metavar = metavar
 +
 +    def _get_kwargs(self):
 +        names = [
 +            'option_strings',
 +            'dest',
 +            'nargs',
 +            'const',
 +            'default',
 +            'type',
 +            'choices',
 +            'help',
 +            'metavar',
 +        ]
 +        return [(name, getattr(self, name)) for name in names]
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        raise NotImplementedError(_('.__call__() not defined'))
 +
 +
 +class _StoreAction(Action):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 nargs=None,
 +                 const=None,
 +                 default=None,
 +                 type=None,
 +                 choices=None,
 +                 required=False,
 +                 help=None,
 +                 metavar=None):
 +        if nargs == 0:
 +            raise ValueError('nargs for store actions must be > 0; if you '
 +                             'have nothing to store, actions such as store '
 +                             'true or store const may be more appropriate')
 +        if const is not None and nargs != OPTIONAL:
 +            raise ValueError('nargs must be %r to supply const' % OPTIONAL)
 +        super(_StoreAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            nargs=nargs,
 +            const=const,
 +            default=default,
 +            type=type,
 +            choices=choices,
 +            required=required,
 +            help=help,
 +            metavar=metavar)
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        setattr(namespace, self.dest, values)
 +
 +
 +class _StoreConstAction(Action):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 const,
 +                 default=None,
 +                 required=False,
 +                 help=None,
 +                 metavar=None):
 +        super(_StoreConstAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            nargs=0,
 +            const=const,
 +            default=default,
 +            required=required,
 +            help=help)
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        setattr(namespace, self.dest, self.const)
 +
 +
 +class _StoreTrueAction(_StoreConstAction):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 default=False,
 +                 required=False,
 +                 help=None):
 +        super(_StoreTrueAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            const=True,
 +            default=default,
 +            required=required,
 +            help=help)
 +
 +
 +class _StoreFalseAction(_StoreConstAction):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 default=True,
 +                 required=False,
 +                 help=None):
 +        super(_StoreFalseAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            const=False,
 +            default=default,
 +            required=required,
 +            help=help)
 +
 +
 +class _AppendAction(Action):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 nargs=None,
 +                 const=None,
 +                 default=None,
 +                 type=None,
 +                 choices=None,
 +                 required=False,
 +                 help=None,
 +                 metavar=None):
 +        if nargs == 0:
 +            raise ValueError('nargs for append actions must be > 0; if arg '
 +                             'strings are not supplying the value to append, '
 +                             'the append const action may be more appropriate')
 +        if const is not None and nargs != OPTIONAL:
 +            raise ValueError('nargs must be %r to supply const' % OPTIONAL)
 +        super(_AppendAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            nargs=nargs,
 +            const=const,
 +            default=default,
 +            type=type,
 +            choices=choices,
 +            required=required,
 +            help=help,
 +            metavar=metavar)
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        items = _copy.copy(_ensure_value(namespace, self.dest, []))
 +        items.append(values)
 +        setattr(namespace, self.dest, items)
 +
 +
 +class _AppendConstAction(Action):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 const,
 +                 default=None,
 +                 required=False,
 +                 help=None,
 +                 metavar=None):
 +        super(_AppendConstAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            nargs=0,
 +            const=const,
 +            default=default,
 +            required=required,
 +            help=help,
 +            metavar=metavar)
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        items = _copy.copy(_ensure_value(namespace, self.dest, []))
 +        items.append(self.const)
 +        setattr(namespace, self.dest, items)
 +
 +
 +class _CountAction(Action):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest,
 +                 default=None,
 +                 required=False,
 +                 help=None):
 +        super(_CountAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            nargs=0,
 +            default=default,
 +            required=required,
 +            help=help)
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        new_count = _ensure_value(namespace, self.dest, 0) + 1
 +        setattr(namespace, self.dest, new_count)
 +
 +
 +class _HelpAction(Action):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 dest=SUPPRESS,
 +                 default=SUPPRESS,
 +                 help=None):
 +        super(_HelpAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            default=default,
 +            nargs=0,
 +            help=help)
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        parser.print_help()
 +        parser.exit()
 +
 +
 +class _VersionAction(Action):
 +
 +    def __init__(self,
 +                 option_strings,
 +                 version=None,
 +                 dest=SUPPRESS,
 +                 default=SUPPRESS,
 +                 help="show program's version number and exit"):
 +        super(_VersionAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            default=default,
 +            nargs=0,
 +            help=help)
 +        self.version = version
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        version = self.version
 +        if version is None:
 +            version = parser.version
 +        formatter = parser._get_formatter()
 +        formatter.add_text(version)
 +        parser.exit(message=formatter.format_help())
 +
 +
 +class _SubParsersAction(Action):
 +
 +    class _ChoicesPseudoAction(Action):
 +
 +        def __init__(self, name, aliases, help):
 +            metavar = dest = name
 +            if aliases:
 +                metavar += ' (%s)' % ', '.join(aliases)
 +            sup = super(_SubParsersAction._ChoicesPseudoAction, self)
 +            sup.__init__(option_strings=[], dest=dest, help=help,
 +                        metavar=metavar)
 +
 +    def __init__(self,
 +                 option_strings,
 +                 prog,
 +                 parser_class,
 +                 dest=SUPPRESS,
 +                 help=None,
 +                 metavar=None):
 +
 +        self._prog_prefix = prog
 +        self._parser_class = parser_class
 +        self._name_parser_map = {}
 +        self._choices_actions = []
 +
 +        super(_SubParsersAction, self).__init__(
 +            option_strings=option_strings,
 +            dest=dest,
 +            nargs=PARSER,
 +            choices=self._name_parser_map,
 +            help=help,
 +            metavar=metavar)
 +
 +    def add_parser(self, name, **kwargs):
 +        # set prog from the existing prefix
 +        if kwargs.get('prog') is None:
 +            kwargs['prog'] = '%s %s' % (self._prog_prefix, name)
 +
 +        aliases = kwargs.pop('aliases', ())
 +
 +        # create a pseudo-action to hold the choice help
 +        if 'help' in kwargs:
 +            help = kwargs.pop('help')
 +            choice_action = self._ChoicesPseudoAction(name, aliases, help)
 +            self._choices_actions.append(choice_action)
 +
 +        # create the parser and add it to the map
 +        parser = self._parser_class(**kwargs)
 +        self._name_parser_map[name] = parser
 +
 +        # make parser available under aliases also
 +        for alias in aliases:
 +            self._name_parser_map[alias] = parser
 +
 +        return parser
 +
 +    def _get_subactions(self):
 +        return self._choices_actions
 +
 +    def __call__(self, parser, namespace, values, option_string=None):
 +        parser_name = values[0]
 +        arg_strings = values[1:]
 +
 +        # set the parser name if requested
 +        if self.dest is not SUPPRESS:
 +            setattr(namespace, self.dest, parser_name)
 +
 +        # select the parser
 +        try:
 +            parser = self._name_parser_map[parser_name]
 +        except KeyError:
 +            tup = parser_name, ', '.join(self._name_parser_map)
 +            msg = _('unknown parser %r (choices: %s)' % tup)
 +            raise ArgumentError(self, msg)
 +
 +        # parse all the remaining options into the namespace
 +        # store any unrecognized options on the object, so that the top
 +        # level parser can decide what to do with them
 +        namespace, arg_strings = parser.parse_known_args(arg_strings, namespace)
 +        if arg_strings:
 +            vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
 +            getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)
 +
 +
 +# ==============
 +# Type classes
 +# ==============
 +
 +class FileType(object):
 +    """Factory for creating file object types
 +
 +    Instances of FileType are typically passed as type= arguments to the
 +    ArgumentParser add_argument() method.
 +
 +    Keyword Arguments:
 +        - mode -- A string indicating how the file is to be opened. Accepts the
 +            same values as the builtin open() function.
 +        - bufsize -- The file's desired buffer size. Accepts the same values as
 +            the builtin open() function.
 +    """
 +
 +    def __init__(self, mode='r', bufsize=None):
 +        self._mode = mode
 +        self._bufsize = bufsize
 +
 +    def __call__(self, string):
 +        # the special argument "-" means sys.std{in,out}
 +        if string == '-':
 +            if 'r' in self._mode:
 +                return _sys.stdin
 +            elif 'w' in self._mode:
 +                return _sys.stdout
 +            else:
 +                msg = _('argument "-" with mode %r' % self._mode)
 +                raise ValueError(msg)
 +
 +        try:
 +            # all other arguments are used as file names
 +            if self._bufsize:
 +                return open(string, self._mode, self._bufsize)
 +            else:
 +                return open(string, self._mode)
 +        except IOError:
 +            err = _sys.exc_info()[1]
 +            message = _("can't open '%s': %s")
 +            raise ArgumentTypeError(message % (string, err))
 +
 +    def __repr__(self):
 +        args = [self._mode, self._bufsize]
 +        args_str = ', '.join([repr(arg) for arg in args if arg is not None])
 +        return '%s(%s)' % (type(self).__name__, args_str)
 +
 +# ===========================
 +# Optional and Positional Parsing
 +# ===========================
 +
 +class Namespace(_AttributeHolder):
 +    """Simple object for storing attributes.
 +
 +    Implements equality by attribute names and values, and provides a simple
 +    string representation.
 +    """
 +
 +    def __init__(self, **kwargs):
 +        for name in kwargs:
 +            setattr(self, name, kwargs[name])
 +
 +    __hash__ = None
 +
 +    def __eq__(self, other):
 +        return vars(self) == vars(other)
 +
 +    def __ne__(self, other):
 +        return not (self == other)
 +
 +    def __contains__(self, key):
 +        return key in self.__dict__
 +
 +
 +class _ActionsContainer(object):
 +
 +    def __init__(self,
 +                 description,
 +                 prefix_chars,
 +                 argument_default,
 +                 conflict_handler):
 +        super(_ActionsContainer, self).__init__()
 +
 +        self.description = description
 +        self.argument_default = argument_default
 +        self.prefix_chars = prefix_chars
 +        self.conflict_handler = conflict_handler
 +
 +        # set up registries
 +        self._registries = {}
 +
 +        # register actions
 +        self.register('action', None, _StoreAction)
 +        self.register('action', 'store', _StoreAction)
 +        self.register('action', 'store_const', _StoreConstAction)
 +        self.register('action', 'store_true', _StoreTrueAction)
 +        self.register('action', 'store_false', _StoreFalseAction)
 +        self.register('action', 'append', _AppendAction)
 +        self.register('action', 'append_const', _AppendConstAction)
 +        self.register('action', 'count', _CountAction)
 +        self.register('action', 'help', _HelpAction)
 +        self.register('action', 'version', _VersionAction)
 +        self.register('action', 'parsers', _SubParsersAction)
 +
 +        # raise an exception if the conflict handler is invalid
 +        self._get_handler()
 +
 +        # action storage
 +        self._actions = []
 +        self._option_string_actions = {}
 +
 +        # groups
 +        self._action_groups = []
 +        self._mutually_exclusive_groups = []
 +
 +        # defaults storage
 +        self._defaults = {}
 +
 +        # determines whether an "option" looks like a negative number
 +        self._negative_number_matcher = _re.compile(r'^-\d+$|^-\d*\.\d+$')
 +
 +        # whether or not there are any optionals that look like negative
 +        # numbers -- uses a list so it can be shared and edited
 +        self._has_negative_number_optionals = []
 +
 +    # ====================
 +    # Registration methods
 +    # ====================
 +    def register(self, registry_name, value, object):
 +        registry = self._registries.setdefault(registry_name, {})
 +        registry[value] = object
 +
 +    def _registry_get(self, registry_name, value, default=None):
 +        return self._registries[registry_name].get(value, default)
 +
 +    # ==================================
 +    # Namespace default accessor methods
 +    # ==================================
 +    def set_defaults(self, **kwargs):
 +        self._defaults.update(kwargs)
 +
 +        # if these defaults match any existing arguments, replace
 +        # the previous default on the object with the new one
 +        for action in self._actions:
 +            if action.dest in kwargs:
 +                action.default = kwargs[action.dest]
 +
 +    def get_default(self, dest):
 +        for action in self._actions:
 +            if action.dest == dest and action.default is not None:
 +                return action.default
 +        return self._defaults.get(dest, None)
 +
 +
 +    # =======================
 +    # Adding argument actions
 +    # =======================
 +    def add_argument(self, *args, **kwargs):
 +        """
 +        add_argument(dest, ..., name=value, ...)
 +        add_argument(option_string, option_string, ..., name=value, ...)
 +        """
 +
 +        # if no positional args are supplied or only one is supplied and
 +        # it doesn't look like an option string, parse a positional
 +        # argument
 +        chars = self.prefix_chars
 +        if not args or len(args) == 1 and args[0][0] not in chars:
 +            if args and 'dest' in kwargs:
 +                raise ValueError('dest supplied twice for positional argument')
 +            kwargs = self._get_positional_kwargs(*args, **kwargs)
 +
 +        # otherwise, we're adding an optional argument
 +        else:
 +            kwargs = self._get_optional_kwargs(*args, **kwargs)
 +
 +        # if no default was supplied, use the parser-level default
 +        if 'default' not in kwargs:
 +            dest = kwargs['dest']
 +            if dest in self._defaults:
 +                kwargs['default'] = self._defaults[dest]
 +            elif self.argument_default is not None:
 +                kwargs['default'] = self.argument_default
 +
 +        # create the action object, and add it to the parser
 +        action_class = self._pop_action_class(kwargs)
 +        if not _callable(action_class):
 +            raise ValueError('unknown action "%s"' % action_class)
 +        action = action_class(**kwargs)
 +
 +        # raise an error if the action type is not callable
 +        type_func = self._registry_get('type', action.type, action.type)
 +        if not _callable(type_func):
 +            raise ValueError('%r is not callable' % type_func)
 +
 +        return self._add_action(action)
 +
 +    def add_argument_group(self, *args, **kwargs):
 +        group = _ArgumentGroup(self, *args, **kwargs)
 +        self._action_groups.append(group)
 +        return group
 +
 +    def add_mutually_exclusive_group(self, **kwargs):
 +        group = _MutuallyExclusiveGroup(self, **kwargs)
 +        self._mutually_exclusive_groups.append(group)
 +        return group
 +
 +    def _add_action(self, action):
 +        # resolve any conflicts
 +        self._check_conflict(action)
 +
 +        # add to actions list
 +        self._actions.append(action)
 +        action.container = self
 +
 +        # index the action by any option strings it has
 +        for option_string in action.option_strings:
 +            self._option_string_actions[option_string] = action
 +
 +        # set the flag if any option strings look like negative numbers
 +        for option_string in action.option_strings:
 +            if self._negative_number_matcher.match(option_string):
 +                if not self._has_negative_number_optionals:
 +                    self._has_negative_number_optionals.append(True)
 +
 +        # return the created action
 +        return action
 +
 +    def _remove_action(self, action):
 +        self._actions.remove(action)
 +
 +    def _add_container_actions(self, container):
 +        # collect groups by titles
 +        title_group_map = {}
 +        for group in self._action_groups:
 +            if group.title in title_group_map:
 +                msg = _('cannot merge actions - two groups are named %r')
 +                raise ValueError(msg % (group.title))
 +            title_group_map[group.title] = group
 +
 +        # map each action to its group
 +        group_map = {}
 +        for group in container._action_groups:
 +
 +            # if a group with the title exists, use that, otherwise
 +            # create a new group matching the container's group
 +            if group.title not in title_group_map:
 +                title_group_map[group.title] = self.add_argument_group(
 +                    title=group.title,
 +                    description=group.description,
 +                    conflict_handler=group.conflict_handler)
 +
 +            # map the actions to their new group
 +            for action in group._group_actions:
 +                group_map[action] = title_group_map[group.title]
 +
 +        # add container's mutually exclusive groups
 +        # NOTE: if add_mutually_exclusive_group ever gains title= and
 +        # description= then this code will need to be expanded as above
 +        for group in container._mutually_exclusive_groups:
 +            mutex_group = self.add_mutually_exclusive_group(
 +                required=group.required)
 +
 +            # map the actions to their new mutex group
 +            for action in group._group_actions:
 +                group_map[action] = mutex_group
 +
 +        # add all actions to this container or their group
 +        for action in container._actions:
 +            group_map.get(action, self)._add_action(action)
 +
 +    def _get_positional_kwargs(self, dest, **kwargs):
 +        # make sure required is not specified
 +        if 'required' in kwargs:
 +            msg = _("'required' is an invalid argument for positionals")
 +            raise TypeError(msg)
 +
 +        # mark positional arguments as required if at least one is
 +        # always required
 +        if kwargs.get('nargs') not in [OPTIONAL, ZERO_OR_MORE]:
 +            kwargs['required'] = True
 +        if kwargs.get('nargs') == ZERO_OR_MORE and 'default' not in kwargs:
 +            kwargs['required'] = True
 +
 +        # return the keyword arguments with no option strings
 +        return dict(kwargs, dest=dest, option_strings=[])
 +
 +    def _get_optional_kwargs(self, *args, **kwargs):
 +        # determine short and long option strings
 +        option_strings = []
 +        long_option_strings = []
 +        for option_string in args:
 +            # error on strings that don't start with an appropriate prefix
 +            if not option_string[0] in self.prefix_chars:
 +                msg = _('invalid option string %r: '
 +                        'must start with a character %r')
 +                tup = option_string, self.prefix_chars
 +                raise ValueError(msg % tup)
 +
 +            # strings starting with two prefix characters are long options
 +            option_strings.append(option_string)
 +            if option_string[0] in self.prefix_chars:
 +                if len(option_string) > 1:
 +                    if option_string[1] in self.prefix_chars:
 +                        long_option_strings.append(option_string)
 +
 +        # infer destination, '--foo-bar' -> 'foo_bar' and '-x' -> 'x'
 +        dest = kwargs.pop('dest', None)
 +        if dest is None:
 +            if long_option_strings:
 +                dest_option_string = long_option_strings[0]
 +            else:
 +                dest_option_string = option_strings[0]
 +            dest = dest_option_string.lstrip(self.prefix_chars)
 +            if not dest:
 +                msg = _('dest= is required for options like %r')
 +                raise ValueError(msg % option_string)
 +            dest = dest.replace('-', '_')
 +
 +        # return the updated keyword arguments
 +        return dict(kwargs, dest=dest, option_strings=option_strings)
 +
 +    def _pop_action_class(self, kwargs, default=None):
 +        action = kwargs.pop('action', default)
 +        return self._registry_get('action', action, action)
 +
 +    def _get_handler(self):
 +        # determine function from conflict handler string
 +        handler_func_name = '_handle_conflict_%s' % self.conflict_handler
 +        try:
 +            return getattr(self, handler_func_name)
 +        except AttributeError:
 +            msg = _('invalid conflict_resolution value: %r')
 +            raise ValueError(msg % self.conflict_handler)
 +
 +    def _check_conflict(self, action):
 +
 +        # find all options that conflict with this option
 +        confl_optionals = []
 +        for option_string in action.option_strings:
 +            if option_string in self._option_string_actions:
 +                confl_optional = self._option_string_actions[option_string]
 +                confl_optionals.append((option_string, confl_optional))
 +
 +        # resolve any conflicts
 +        if confl_optionals:
 +            conflict_handler = self._get_handler()
 +            conflict_handler(action, confl_optionals)
 +
 +    def _handle_conflict_error(self, action, conflicting_actions):
 +        message = _('conflicting option string(s): %s')
 +        conflict_string = ', '.join([option_string
 +                                     for option_string, action
 +                                     in conflicting_actions])
 +        raise ArgumentError(action, message % conflict_string)
 +
 +    def _handle_conflict_resolve(self, action, conflicting_actions):
 +
 +        # remove all conflicting options
 +        for option_string, action in conflicting_actions:
 +
 +            # remove the conflicting option
 +            action.option_strings.remove(option_string)
 +            self._option_string_actions.pop(option_string, None)
 +
 +            # if the option now has no option string, remove it from the
 +            # container holding it
 +            if not action.option_strings:
 +                action.container._remove_action(action)
 +
 +
 +class _ArgumentGroup(_ActionsContainer):
 +
 +    def __init__(self, container, title=None, description=None, **kwargs):
 +        # add any missing keyword arguments by checking the container
 +        update = kwargs.setdefault
 +        update('conflict_handler', container.conflict_handler)
 +        update('prefix_chars', container.prefix_chars)
 +        update('argument_default', container.argument_default)
 +        super_init = super(_ArgumentGroup, self).__init__
 +        super_init(description=description, **kwargs)
 +
 +        # group attributes
 +        self.title = title
 +        self._group_actions = []
 +
 +        # share most attributes with the container
 +        self._registries = container._registries
 +        self._actions = container._actions
 +        self._option_string_actions = container._option_string_actions
 +        self._defaults = container._defaults
 +        self._has_negative_number_optionals = \
 +            container._has_negative_number_optionals
 +
 +    def _add_action(self, action):
 +        action = super(_ArgumentGroup, self)._add_action(action)
 +        self._group_actions.append(action)
 +        return action
 +
 +    def _remove_action(self, action):
 +        super(_ArgumentGroup, self)._remove_action(action)
 +        self._group_actions.remove(action)
 +
 +
 +class _MutuallyExclusiveGroup(_ArgumentGroup):
 +
 +    def __init__(self, container, required=False):
 +        super(_MutuallyExclusiveGroup, self).__init__(container)
 +        self.required = required
 +        self._container = container
 +
 +    def _add_action(self, action):
 +        if action.required:
 +            msg = _('mutually exclusive arguments must be optional')
 +            raise ValueError(msg)
 +        action = self._container._add_action(action)
 +        self._group_actions.append(action)
 +        return action
 +
 +    def _remove_action(self, action):
 +        self._container._remove_action(action)
 +        self._group_actions.remove(action)
 +
 +
 +class ArgumentParser(_AttributeHolder, _ActionsContainer):
 +    """Object for parsing command line strings into Python objects.
 +
 +    Keyword Arguments:
 +        - prog -- The name of the program (default: sys.argv[0])
 +        - usage -- A usage message (default: auto-generated from arguments)
 +        - description -- A description of what the program does
 +        - epilog -- Text following the argument descriptions
 +        - parents -- Parsers whose arguments should be copied into this one
 +        - formatter_class -- HelpFormatter class for printing help messages
 +        - prefix_chars -- Characters that prefix optional arguments
 +        - fromfile_prefix_chars -- Characters that prefix files containing
 +            additional arguments
 +        - argument_default -- The default value for all arguments
 +        - conflict_handler -- String indicating how to handle conflicts
 +        - add_help -- Add a -h/-help option
 +    """
 +
 +    def __init__(self,
 +                 prog=None,
 +                 usage=None,
 +                 description=None,
 +                 epilog=None,
 +                 version=None,
 +                 parents=[],
 +                 formatter_class=HelpFormatter,
 +                 prefix_chars='-',
 +                 fromfile_prefix_chars=None,
 +                 argument_default=None,
 +                 conflict_handler='error',
 +                 add_help=True):
 +
 +        if version is not None:
 +            import warnings
 +            warnings.warn(
 +                """The "version" argument to ArgumentParser is deprecated. """
 +                """Please use """
 +                """"add_argument(..., action='version', version="N", ...)" """
 +                """instead""", DeprecationWarning)
 +
 +        superinit = super(ArgumentParser, self).__init__
 +        superinit(description=description,
 +                  prefix_chars=prefix_chars,
 +                  argument_default=argument_default,
 +                  conflict_handler=conflict_handler)
 +
 +        # default setting for prog
 +        if prog is None:
 +            prog = _os.path.basename(_sys.argv[0])
 +
 +        self.prog = prog
 +        self.usage = usage
 +        self.epilog = epilog
 +        self.version = version
 +        self.formatter_class = formatter_class
 +        self.fromfile_prefix_chars = fromfile_prefix_chars
 +        self.add_help = add_help
 +
 +        add_group = self.add_argument_group
 +        self._positionals = add_group(_('positional arguments'))
 +        self._optionals = add_group(_('optional arguments'))
 +        self._subparsers = None
 +
 +        # register types
 +        def identity(string):
 +            return string
 +        self.register('type', None, identity)
 +
 +        # add help and version arguments if necessary
 +        # (using explicit default to override global argument_default)
 +        if '-' in prefix_chars:
 +            default_prefix = '-'
 +        else:
 +            default_prefix = prefix_chars[0]
 +        if self.add_help:
 +            self.add_argument(
 +                default_prefix+'h', default_prefix*2+'help',
 +                action='help', default=SUPPRESS,
 +                help=_('show this help message and exit'))
 +        if self.version:
 +            self.add_argument(
 +                default_prefix+'v', default_prefix*2+'version',
 +                action='version', default=SUPPRESS,
 +                version=self.version,
 +                help=_("show program's version number and exit"))
 +
 +        # add parent arguments and defaults
 +        for parent in parents:
 +            self._add_container_actions(parent)
 +            try:
 +                defaults = parent._defaults
 +            except AttributeError:
 +                pass
 +            else:
 +                self._defaults.update(defaults)
 +
 +    # =======================
 +    # Pretty __repr__ methods
 +    # =======================
 +    def _get_kwargs(self):
 +        names = [
 +            'prog',
 +            'usage',
 +            'description',
 +            'version',
 +            'formatter_class',
 +            'conflict_handler',
 +            'add_help',
 +        ]
 +        return [(name, getattr(self, name)) for name in names]
 +
 +    # ==================================
 +    # Optional/Positional adding methods
 +    # ==================================
 +    def add_subparsers(self, **kwargs):
 +        if self._subparsers is not None:
 +            self.error(_('cannot have multiple subparser arguments'))
 +
 +        # add the parser class to the arguments if it's not present
 +        kwargs.setdefault('parser_class', type(self))
 +
 +        if 'title' in kwargs or 'description' in kwargs:
 +            title = _(kwargs.pop('title', 'subcommands'))
 +            description = _(kwargs.pop('description', None))
 +            self._subparsers = self.add_argument_group(title, description)
 +        else:
 +            self._subparsers = self._positionals
 +
 +        # prog defaults to the usage message of this parser, skipping
 +        # optional arguments and with no "usage:" prefix
 +        if kwargs.get('prog') is None:
 +            formatter = self._get_formatter()
 +            positionals = self._get_positional_actions()
 +            groups = self._mutually_exclusive_groups
 +            formatter.add_usage(self.usage, positionals, groups, '')
 +            kwargs['prog'] = formatter.format_help().strip()
 +
 +        # create the parsers action and add it to the positionals list
 +        parsers_class = self._pop_action_class(kwargs, 'parsers')
 +        action = parsers_class(option_strings=[], **kwargs)
 +        self._subparsers._add_action(action)
 +
 +        # return the created parsers action
 +        return action
 +
 +    def _add_action(self, action):
 +        if action.option_strings:
 +            self._optionals._add_action(action)
 +        else:
 +            self._positionals._add_action(action)
 +        return action
 +
 +    def _get_optional_actions(self):
 +        return [action
 +                for action in self._actions
 +                if action.option_strings]
 +
 +    def _get_positional_actions(self):
 +        return [action
 +                for action in self._actions
 +                if not action.option_strings]
 +
 +    # =====================================
 +    # Command line argument parsing methods
 +    # =====================================
 +    def parse_args(self, args=None, namespace=None):
 +        args, argv = self.parse_known_args(args, namespace)
 +        if argv:
 +            msg = _('unrecognized arguments: %s')
 +            self.error(msg % ' '.join(argv))
 +        return args
 +
 +    def parse_known_args(self, args=None, namespace=None):
 +        # args default to the system args
 +        if args is None:
 +            args = _sys.argv[1:]
 +
 +        # default Namespace built from parser defaults
 +        if namespace is None:
 +            namespace = Namespace()
 +
 +        # add any action defaults that aren't present
 +        for action in self._actions:
 +            if action.dest is not SUPPRESS:
 +                if not hasattr(namespace, action.dest):
 +                    if action.default is not SUPPRESS:
 +                        setattr(namespace, action.dest, action.default)
 +
 +        # add any parser defaults that aren't present
 +        for dest in self._defaults:
 +            if not hasattr(namespace, dest):
 +                setattr(namespace, dest, self._defaults[dest])
 +
 +        # parse the arguments and exit if there are any errors
 +        try:
 +            namespace, args = self._parse_known_args(args, namespace)
 +            if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR):
 +                args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))
 +                delattr(namespace, _UNRECOGNIZED_ARGS_ATTR)
 +            return namespace, args
 +        except ArgumentError:
 +            err = _sys.exc_info()[1]
 +            self.error(str(err))
 +
 +    def _parse_known_args(self, arg_strings, namespace):
 +        # replace arg strings that are file references
 +        if self.fromfile_prefix_chars is not None:
 +            arg_strings = self._read_args_from_files(arg_strings)
 +
 +        # map all mutually exclusive arguments to the other arguments
 +        # they can't occur with
 +        action_conflicts = {}
 +        for mutex_group in self._mutually_exclusive_groups:
 +            group_actions = mutex_group._group_actions
 +            for i, mutex_action in enumerate(mutex_group._group_actions):
 +                conflicts = action_conflicts.setdefault(mutex_action, [])
 +                conflicts.extend(group_actions[:i])
 +                conflicts.extend(group_actions[i + 1:])
 +
 +        # find all option indices, and determine the arg_string_pattern
 +        # which has an 'O' if there is an option at an index,
 +        # an 'A' if there is an argument, or a '-' if there is a '--'
 +        option_string_indices = {}
 +        arg_string_pattern_parts = []
 +        arg_strings_iter = iter(arg_strings)
 +        for i, arg_string in enumerate(arg_strings_iter):
 +
 +            # all args after -- are non-options
 +            if arg_string == '--':
 +                arg_string_pattern_parts.append('-')
 +                for arg_string in arg_strings_iter:
 +                    arg_string_pattern_parts.append('A')
 +
 +            # otherwise, add the arg to the arg strings
 +            # and note the index if it was an option
 +            else:
 +                option_tuple = self._parse_optional(arg_string)
 +                if option_tuple is None:
 +                    pattern = 'A'
 +                else:
 +                    option_string_indices[i] = option_tuple
 +                    pattern = 'O'
 +                arg_string_pattern_parts.append(pattern)
 +
 +        # join the pieces together to form the pattern
 +        arg_strings_pattern = ''.join(arg_string_pattern_parts)
 +
 +        # converts arg strings to the appropriate and then takes the action
 +        seen_actions = set()
 +        seen_non_default_actions = set()
 +
 +        def take_action(action, argument_strings, option_string=None):
 +            seen_actions.add(action)
 +            argument_values = self._get_values(action, argument_strings)
 +
 +            # error if this argument is not allowed with other previously
 +            # seen arguments, assuming that actions that use the default
 +            # value don't really count as "present"
 +            if argument_values is not action.default:
 +                seen_non_default_actions.add(action)
 +                for conflict_action in action_conflicts.get(action, []):
 +                    if conflict_action in seen_non_default_actions:
 +                        msg = _('not allowed with argument %s')
 +                        action_name = _get_action_name(conflict_action)
 +                        raise ArgumentError(action, msg % action_name)
 +
 +            # take the action if we didn't receive a SUPPRESS value
 +            # (e.g. from a default)
 +            if argument_values is not SUPPRESS:
 +                action(self, namespace, argument_values, option_string)
 +
 +        # function to convert arg_strings into an optional action
 +        def consume_optional(start_index):
 +
 +            # get the optional identified at this index
 +            option_tuple = option_string_indices[start_index]
 +            action, option_string, explicit_arg = option_tuple
 +
 +            # identify additional optionals in the same arg string
 +            # (e.g. -xyz is the same as -x -y -z if no args are required)
 +            match_argument = self._match_argument
 +            action_tuples = []
 +            while True:
 +
 +                # if we found no optional action, skip it
 +                if action is None:
 +                    extras.append(arg_strings[start_index])
 +                    return start_index + 1
 +
 +                # if there is an explicit argument, try to match the
 +                # optional's string arguments to only this
 +                if explicit_arg is not None:
 +                    arg_count = match_argument(action, 'A')
 +
 +                    # if the action is a single-dash option and takes no
 +                    # arguments, try to parse more single-dash options out
 +                    # of the tail of the option string
 +                    chars = self.prefix_chars
 +                    if arg_count == 0 and option_string[1] not in chars:
 +                        action_tuples.append((action, [], option_string))
 +                        char = option_string[0]
 +                        option_string = char + explicit_arg[0]
 +                        new_explicit_arg = explicit_arg[1:] or None
 +                        optionals_map = self._option_string_actions
 +                        if option_string in optionals_map:
 +                            action = optionals_map[option_string]
 +                            explicit_arg = new_explicit_arg
 +                        else:
 +                            msg = _('ignored explicit argument %r')
 +                            raise ArgumentError(action, msg % explicit_arg)
 +
 +                    # if the action expect exactly one argument, we've
 +                    # successfully matched the option; exit the loop
 +                    elif arg_count == 1:
 +                        stop = start_index + 1
 +                        args = [explicit_arg]
 +                        action_tuples.append((action, args, option_string))
 +                        break
 +
 +                    # error if a double-dash option did not use the
 +                    # explicit argument
 +                    else:
 +                        msg = _('ignored explicit argument %r')
 +                        raise ArgumentError(action, msg % explicit_arg)
 +
 +                # if there is no explicit argument, try to match the
 +                # optional's string arguments with the following strings
 +                # if successful, exit the loop
 +                else:
 +                    start = start_index + 1
 +                    selected_patterns = arg_strings_pattern[start:]
 +                    arg_count = match_argument(action, selected_patterns)
 +                    stop = start + arg_count
 +                    args = arg_strings[start:stop]
 +                    action_tuples.append((action, args, option_string))
 +                    break
 +
 +            # add the Optional to the list and return the index at which
 +            # the Optional's string args stopped
 +            assert action_tuples
 +            for action, args, option_string in action_tuples:
 +                take_action(action, args, option_string)
 +            return stop
 +
 +        # the list of Positionals left to be parsed; this is modified
 +        # by consume_positionals()
 +        positionals = self._get_positional_actions()
 +
 +        # function to convert arg_strings into positional actions
 +        def consume_positionals(start_index):
 +            # match as many Positionals as possible
 +            match_partial = self._match_arguments_partial
 +            selected_pattern = arg_strings_pattern[start_index:]
 +            arg_counts = match_partial(positionals, selected_pattern)
 +
 +            # slice off the appropriate arg strings for each Positional
 +            # and add the Positional and its args to the list
 +            for action, arg_count in zip(positionals, arg_counts):
 +                args = arg_strings[start_index: start_index + arg_count]
 +                start_index += arg_count
 +                take_action(action, args)
 +
 +            # slice off the Positionals that we just parsed and return the
 +            # index at which the Positionals' string args stopped
 +            positionals[:] = positionals[len(arg_counts):]
 +            return start_index
 +
 +        # consume Positionals and Optionals alternately, until we have
 +        # passed the last option string
 +        extras = []
 +        start_index = 0
 +        if option_string_indices:
 +            max_option_string_index = max(option_string_indices)
 +        else:
 +            max_option_string_index = -1
 +        while start_index <= max_option_string_index:
 +
 +            # consume any Positionals preceding the next option
 +            next_option_string_index = min([
 +                index
 +                for index in option_string_indices
 +                if index >= start_index])
 +            if start_index != next_option_string_index:
 +                positionals_end_index = consume_positionals(start_index)
 +
 +                # only try to parse the next optional if we didn't consume
 +                # the option string during the positionals parsing
 +                if positionals_end_index > start_index:
 +                    start_index = positionals_end_index
 +                    continue
 +                else:
 +                    start_index = positionals_end_index
 +
 +            # if we consumed all the positionals we could and we're not
 +            # at the index of an option string, there were extra arguments
 +            if start_index not in option_string_indices:
 +                strings = arg_strings[start_index:next_option_string_index]
 +                extras.extend(strings)
 +                start_index = next_option_string_index
 +
 +            # consume the next optional and any arguments for it
 +            start_index = consume_optional(start_index)
 +
 +        # consume any positionals following the last Optional
 +        stop_index = consume_positionals(start_index)
 +
 +        # if we didn't consume all the argument strings, there were extras
 +        extras.extend(arg_strings[stop_index:])
 +
 +        # if we didn't use all the Positional objects, there were too few
 +        # arg strings supplied.
 +        if positionals:
 +            self.error(_('too few arguments'))
 +
 +        # make sure all required actions were present, and convert defaults.
 +        for action in self._actions:
 +            if action not in seen_actions:
 +                if action.required:
 +                    name = _get_action_name(action)
 +                    self.error(_('argument %s is required') % name)
 +                else:
 +                    # Convert action default now instead of doing it before
 +                    # parsing arguments to avoid calling convert functions
 +                    # twice (which may fail) if the argument was given, but
 +                    # only if it was defined already in the namespace
 +                    if (action.default is not None and
 +                            isinstance(action.default, basestring) and
 +                            hasattr(namespace, action.dest) and
 +                            action.default is getattr(namespace, action.dest)):
 +                        setattr(namespace, action.dest,
 +                                self._get_value(action, action.default))
 +
 +        # make sure all required groups had one option present
 +        for group in self._mutually_exclusive_groups:
 +            if group.required:
 +                for action in group._group_actions:
 +                    if action in seen_non_default_actions:
 +                        break
 +
 +                # if no actions were used, report the error
 +                else:
 +                    names = [_get_action_name(action)
 +                             for action in group._group_actions
 +                             if action.help is not SUPPRESS]
 +                    msg = _('one of the arguments %s is required')
 +                    self.error(msg % ' '.join(names))
 +
 +        # return the updated namespace and the extra arguments
 +        return namespace, extras
 +
 +    def _read_args_from_files(self, arg_strings):
 +        # expand arguments referencing files
 +        new_arg_strings = []
 +        for arg_string in arg_strings:
 +
 +            # for regular arguments, just add them back into the list
 +            if arg_string[0] not in self.fromfile_prefix_chars:
 +                new_arg_strings.append(arg_string)
 +
 +            # replace arguments referencing files with the file content
 +            else:
 +                try:
 +                    args_file = open(arg_string[1:])
 +                    try:
 +                        arg_strings = []
 +                        for arg_line in args_file.read().splitlines():
 +                            for arg in self.convert_arg_line_to_args(arg_line):
 +                                arg_strings.append(arg)
 +                        arg_strings = self._read_args_from_files(arg_strings)
 +                        new_arg_strings.extend(arg_strings)
 +                    finally:
 +                        args_file.close()
 +                except IOError:
 +                    err = _sys.exc_info()[1]
 +                    self.error(str(err))
 +
 +        # return the modified argument list
 +        return new_arg_strings
 +
 +    def convert_arg_line_to_args(self, arg_line):
 +        return [arg_line]
 +
 +    def _match_argument(self, action, arg_strings_pattern):
 +        # match the pattern for this action to the arg strings
 +        nargs_pattern = self._get_nargs_pattern(action)
 +        match = _re.match(nargs_pattern, arg_strings_pattern)
 +
 +        # raise an exception if we weren't able to find a match
 +        if match is None:
 +            nargs_errors = {
 +                None: _('expected one argument'),
 +                OPTIONAL: _('expected at most one argument'),
 +                ONE_OR_MORE: _('expected at least one argument'),
 +            }
 +            default = _('expected %s argument(s)') % action.nargs
 +            msg = nargs_errors.get(action.nargs, default)
 +            raise ArgumentError(action, msg)
 +
 +        # return the number of arguments matched
 +        return len(match.group(1))
 +
 +    def _match_arguments_partial(self, actions, arg_strings_pattern):
 +        # progressively shorten the actions list by slicing off the
 +        # final actions until we find a match
 +        result = []
 +        for i in range(len(actions), 0, -1):
 +            actions_slice = actions[:i]
 +            pattern = ''.join([self._get_nargs_pattern(action)
 +                               for action in actions_slice])
 +            match = _re.match(pattern, arg_strings_pattern)
 +            if match is not None:
 +                result.extend([len(string) for string in match.groups()])
 +                break
 +
 +        # return the list of arg string counts
 +        return result
 +
 +    def _parse_optional(self, arg_string):
 +        # if it's an empty string, it was meant to be a positional
 +        if not arg_string:
 +            return None
 +
 +        # if it doesn't start with a prefix, it was meant to be positional
 +        if not arg_string[0] in self.prefix_chars:
 +            return None
 +
 +        # if the option string is present in the parser, return the action
 +        if arg_string in self._option_string_actions:
 +            action = self._option_string_actions[arg_string]
 +            return action, arg_string, None
 +
 +        # if it's just a single character, it was meant to be positional
 +        if len(arg_string) == 1:
 +            return None
 +
 +        # if the option string before the "=" is present, return the action
 +        if '=' in arg_string:
 +            option_string, explicit_arg = arg_string.split('=', 1)
 +            if option_string in self._option_string_actions:
 +                action = self._option_string_actions[option_string]
 +                return action, option_string, explicit_arg
 +
 +        # search through all possible prefixes of the option string
 +        # and all actions in the parser for possible interpretations
 +        option_tuples = self._get_option_tuples(arg_string)
 +
 +        # if multiple actions match, the option string was ambiguous
 +        if len(option_tuples) > 1:
 +            options = ', '.join([option_string
 +                for action, option_string, explicit_arg in option_tuples])
 +            tup = arg_string, options
 +            self.error(_('ambiguous option: %s could match %s') % tup)
 +
 +        # if exactly one action matched, this segmentation is good,
 +        # so return the parsed action
 +        elif len(option_tuples) == 1:
 +            option_tuple, = option_tuples
 +            return option_tuple
 +
 +        # if it was not found as an option, but it looks like a negative
 +        # number, it was meant to be positional
 +        # unless there are negative-number-like options
 +        if self._negative_number_matcher.match(arg_string):
 +            if not self._has_negative_number_optionals:
 +                return None
 +
 +        # if it contains a space, it was meant to be a positional
 +        if ' ' in arg_string:
 +            return None
 +
 +        # it was meant to be an optional but there is no such option
 +        # in this parser (though it might be a valid option in a subparser)
 +        return None, arg_string, None
 +
 +    def _get_option_tuples(self, option_string):
 +        result = []
 +
 +        # option strings starting with two prefix characters are only
 +        # split at the '='
 +        chars = self.prefix_chars
 +        if option_string[0] in chars and option_string[1] in chars:
 +            if '=' in option_string:
 +                option_prefix, explicit_arg = option_string.split('=', 1)
 +            else:
 +                option_prefix = option_string
 +                explicit_arg = None
 +            for option_string in self._option_string_actions:
 +                if option_string.startswith(option_prefix):
 +                    action = self._option_string_actions[option_string]
 +                    tup = action, option_string, explicit_arg
 +                    result.append(tup)
 +
 +        # single character options can be concatenated with their arguments
 +        # but multiple character options always have to have their argument
 +        # separate
 +        elif option_string[0] in chars and option_string[1] not in chars:
 +            option_prefix = option_string
 +            explicit_arg = None
 +            short_option_prefix = option_string[:2]
 +            short_explicit_arg = option_string[2:]
 +
 +            for option_string in self._option_string_actions:
 +                if option_string == short_option_prefix:
 +                    action = self._option_string_actions[option_string]
 +                    tup = action, option_string, short_explicit_arg
 +                    result.append(tup)
 +                elif option_string.startswith(option_prefix):
 +                    action = self._option_string_actions[option_string]
 +                    tup = action, option_string, explicit_arg
 +                    result.append(tup)
 +
 +        # shouldn't ever get here
 +        else:
 +            self.error(_('unexpected option string: %s') % option_string)
 +
 +        # return the collected option tuples
 +        return result
 +
 +    def _get_nargs_pattern(self, action):
 +        # in all examples below, we have to allow for '--' args
 +        # which are represented as '-' in the pattern
 +        nargs = action.nargs
 +
 +        # the default (None) is assumed to be a single argument
 +        if nargs is None:
 +            nargs_pattern = '(-*A-*)'
 +
 +        # allow zero or one arguments
 +        elif nargs == OPTIONAL:
 +            nargs_pattern = '(-*A?-*)'
 +
 +        # allow zero or more arguments
 +        elif nargs == ZERO_OR_MORE:
 +            nargs_pattern = '(-*[A-]*)'
 +
 +        # allow one or more arguments
 +        elif nargs == ONE_OR_MORE:
 +            nargs_pattern = '(-*A[A-]*)'
 +
 +        # allow any number of options or arguments
 +        elif nargs == REMAINDER:
 +            nargs_pattern = '([-AO]*)'
 +
 +        # allow one argument followed by any number of options or arguments
 +        elif nargs == PARSER:
 +            nargs_pattern = '(-*A[-AO]*)'
 +
 +        # all others should be integers
 +        else:
 +            nargs_pattern = '(-*%s-*)' % '-*'.join('A' * nargs)
 +
 +        # if this is an optional action, -- is not allowed
 +        if action.option_strings:
 +            nargs_pattern = nargs_pattern.replace('-*', '')
 +            nargs_pattern = nargs_pattern.replace('-', '')
 +
 +        # return the pattern
 +        return nargs_pattern
 +
 +    # ========================
 +    # Value conversion methods
 +    # ========================
 +    def _get_values(self, action, arg_strings):
 +        # for everything but PARSER args, strip out '--'
 +        if action.nargs not in [PARSER, REMAINDER]:
 +            arg_strings = [s for s in arg_strings if s != '--']
 +
 +        # optional argument produces a default when not present
 +        if not arg_strings and action.nargs == OPTIONAL:
 +            if action.option_strings:
 +                value = action.const
 +            else:
 +                value = action.default
 +            if isinstance(value, basestring):
 +                value = self._get_value(action, value)
 +                self._check_value(action, value)
 +
 +        # when nargs='*' on a positional, if there were no command-line
 +        # args, use the default if it is anything other than None
 +        elif (not arg_strings and action.nargs == ZERO_OR_MORE and
 +              not action.option_strings):
 +            if action.default is not None:
 +                value = action.default
 +            else:
 +                value = arg_strings
 +            self._check_value(action, value)
 +
 +        # single argument or optional argument produces a single value
 +        elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:
 +            arg_string, = arg_strings
 +            value = self._get_value(action, arg_string)
 +            self._check_value(action, value)
 +
 +        # REMAINDER arguments convert all values, checking none
 +        elif action.nargs == REMAINDER:
 +            value = [self._get_value(action, v) for v in arg_strings]
 +
 +        # PARSER arguments convert all values, but check only the first
 +        elif action.nargs == PARSER:
 +            value = [self._get_value(action, v) for v in arg_strings]
 +            self._check_value(action, value[0])
 +
 +        # all other types of nargs produce a list
 +        else:
 +            value = [self._get_value(action, v) for v in arg_strings]
 +            for v in value:
 +                self._check_value(action, v)
 +
 +        # return the converted value
 +        return value
 +
 +    def _get_value(self, action, arg_string):
 +        type_func = self._registry_get('type', action.type, action.type)
 +        if not _callable(type_func):
 +            msg = _('%r is not callable')
 +            raise ArgumentError(action, msg % type_func)
 +
 +        # convert the value to the appropriate type
 +        try:
 +            result = type_func(arg_string)
 +
 +        # ArgumentTypeErrors indicate errors
 +        except ArgumentTypeError:
 +            name = getattr(action.type, '__name__', repr(action.type))
 +            msg = str(_sys.exc_info()[1])
 +            raise ArgumentError(action, msg)
 +
 +        # TypeErrors or ValueErrors also indicate errors
 +        except (TypeError, ValueError):
 +            name = getattr(action.type, '__name__', repr(action.type))
 +            msg = _('invalid %s value: %r')
 +            raise ArgumentError(action, msg % (name, arg_string))
 +
 +        # return the converted value
 +        return result
 +
 +    def _check_value(self, action, value):
 +        # converted value must be one of the choices (if specified)
 +        if action.choices is not None and value not in action.choices:
 +            tup = value, ', '.join(map(repr, action.choices))
 +            msg = _('invalid choice: %r (choose from %s)') % tup
 +            raise ArgumentError(action, msg)
 +
 +    # =======================
 +    # Help-formatting methods
 +    # =======================
 +    def format_usage(self):
 +        formatter = self._get_formatter()
 +        formatter.add_usage(self.usage, self._actions,
 +                            self._mutually_exclusive_groups)
 +        return formatter.format_help()
 +
 +    def format_help(self):
 +        formatter = self._get_formatter()
 +
 +        # usage
 +        formatter.add_usage(self.usage, self._actions,
 +                            self._mutually_exclusive_groups)
 +
 +        # description
 +        formatter.add_text(self.description)
 +
 +        # positionals, optionals and user-defined groups
 +        for action_group in self._action_groups:
 +            formatter.start_section(action_group.title)
 +            formatter.add_text(action_group.description)
 +            formatter.add_arguments(action_group._group_actions)
 +            formatter.end_section()
 +
 +        # epilog
 +        formatter.add_text(self.epilog)
 +
 +        # determine help from format above
 +        return formatter.format_help()
 +
 +    def format_version(self):
 +        import warnings
 +        warnings.warn(
 +            'The format_version method is deprecated -- the "version" '
 +            'argument to ArgumentParser is no longer supported.',
 +            DeprecationWarning)
 +        formatter = self._get_formatter()
 +        formatter.add_text(self.version)
 +        return formatter.format_help()
 +
 +    def _get_formatter(self):
 +        return self.formatter_class(prog=self.prog)
 +
 +    # =====================
 +    # Help-printing methods
 +    # =====================
 +    def print_usage(self, file=None):
 +        if file is None:
 +            file = _sys.stdout
 +        self._print_message(self.format_usage(), file)
 +
 +    def print_help(self, file=None):
 +        if file is None:
 +            file = _sys.stdout
 +        self._print_message(self.format_help(), file)
 +
 +    def print_version(self, file=None):
 +        import warnings
 +        warnings.warn(
 +            'The print_version method is deprecated -- the "version" '
 +            'argument to ArgumentParser is no longer supported.',
 +            DeprecationWarning)
 +        self._print_message(self.format_version(), file)
 +
 +    def _print_message(self, message, file=None):
 +        if message:
 +            if file is None:
 +                file = _sys.stderr
 +            file.write(message)
 +
 +    # ===============
 +    # Exiting methods
 +    # ===============
 +    def exit(self, status=0, message=None):
 +        if message:
 +            self._print_message(message, _sys.stderr)
 +        _sys.exit(status)
 +
 +    def error(self, message):
 +        """error(message: string)
 +
 +        Prints a usage message incorporating the message to stderr and
 +        exits.
 +
 +        If you override this in a subclass, it should not return -- it
 +        should either exit or raise an exception.
 +        """
 +        self.print_usage(_sys.stderr)
 +        self.exit(2, _('%s: error: %s\n') % (self.prog, message))
 --
-.13.5
+.39.2

-New patch
+[PULL 07/20] block: add some trace events for new block layer APIs
+From: Sam Li <faithilikerun@gmail.com>
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
+Acked-by: Kevin Wolf <kwolf@redhat.com>
+Message-id: 20230324090605.28361-8-faithilikerun@gmail.com
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+---
+ block/file-posix.c | 3 +++
+ block/trace-events | 2 ++
+files changed, 5 insertions(+)
+diff --git a/block/file-posix.c b/block/file-posix.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/file-posix.c
++++ b/block/file-posix.c
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_report(BlockDriverState *bs, int64_t offset,
+         },
+     };
++    trace_zbd_zone_report(bs, *nr_zones, offset >> BDRV_SECTOR_BITS);
+     return raw_thread_pool_submit(bs, handle_aiocb_zone_report, &acb);
+ }
+ #endif
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
+         },
+     };
++    trace_zbd_zone_mgmt(bs, op_name, offset >> BDRV_SECTOR_BITS,
++                        len >> BDRV_SECTOR_BITS);
+     ret = raw_thread_pool_submit(bs, handle_aiocb_zone_mgmt, &acb);
+     if (ret != 0) {
+         error_report("ioctl %s failed %d", op_name, ret);
+diff --git a/block/trace-events b/block/trace-events
+index XXXXXXX..XXXXXXX 100644
+--- a/block/trace-events
++++ b/block/trace-events
+@@ -XXX,XX +XXX,XX @@ file_FindEjectableOpticalMedia(const char *media) "Matching using %s"
+ file_setup_cdrom(const char *partition) "Using %s as optical disc"
+ file_hdev_is_sg(int type, int version) "SG device found: type=%d, version=%d"
+ file_flush_fdatasync_failed(int err) "errno %d"
++zbd_zone_report(void *bs, unsigned int nr_zones, int64_t sector) "bs %p report %d zones starting at sector offset 0x%" PRIx64 ""
++zbd_zone_mgmt(void *bs, const char *op_name, int64_t sector, int64_t len) "bs %p %s starts at sector offset 0x%" PRIx64 " over a range of 0x%" PRIx64 " sectors"
+ # ssh.c
+ sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
+--
+.39.2

-[Qemu-devel] [PULL for-2.10 10/15] misc: Remove unused Error variables
+[PULL 08/20] docs/zoned-storage: add zoned device documentation
-From: Alberto Garcia <berto@igalia.com>
+From: Sam Li <faithilikerun@gmail.com>
-There's a few cases which we're passing an Error pointer to a function
+Add the documentation about the zoned device support to virtio-blk
-only to discard it immediately afterwards without checking it. In
+emulation.
 these cases we can simply remove the variable and pass NULL instead.
-Signed-off-by: Alberto Garcia <berto@igalia.com>
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
-Message-id: 20170829120836.16091-1-berto@igalia.com
+Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
 Acked-by: Kevin Wolf <kwolf@redhat.com>
 Message-id: 20230324090605.28361-9-faithilikerun@gmail.com
 [Add index-api.rst to fix "zoned-storage.rst:document isn't included in
 any toctree" error.
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/qcow.c  | 12 +++---------
+ docs/devel/index-api.rst               |  1 +
- block/qcow2.c |  8 ++------
+ docs/devel/zoned-storage.rst           | 43 ++++++++++++++++++++++++++
- dump.c        |  4 +---
+ docs/system/qemu-block-drivers.rst.inc |  6 ++++
-files changed, 6 insertions(+), 18 deletions(-)
+files changed, 50 insertions(+)
  create mode 100644 docs/devel/zoned-storage.rst
-diff --git a/block/qcow.c b/block/qcow.c
+diff --git a/docs/devel/index-api.rst b/docs/devel/index-api.rst
 index XXXXXXX..XXXXXXX 100644
---- a/block/qcow.c
+--- a/docs/devel/index-api.rst
-+++ b/block/qcow.c
++++ b/docs/devel/index-api.rst
-@@ -XXX,XX +XXX,XX @@ static uint64_t get_cluster_offset(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ generated from in-code annotations to function prototypes.
-                     start_sect = (offset & ~(s->cluster_size - 1)) >> 9;
+    memory
-                     for(i = 0; i < s->cluster_sectors; i++) {
+    modules
-                         if (i < n_start || i >= n_end) {
+    ui
--                            Error *err = NULL;
++   zoned-storage
-                             memset(s->cluster_data, 0x00, 512);
+diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst
-                             if (qcrypto_block_encrypt(s->crypto, start_sect + i,
+new file mode 100644
-                                                       s->cluster_data,
+index XXXXXXX..XXXXXXX
-                                                       BDRV_SECTOR_SIZE,
+--- /dev/null
--                                                      &err) < 0) {
++++ b/docs/devel/zoned-storage.rst
--                                error_free(err);
+@@ -XXX,XX +XXX,XX @@
-+                                                      NULL) < 0) {
++=============
-                                 errno = EIO;
++zoned-storage
-                                 return -1;
++=============
-                             }
++
-@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_readv(BlockDriverState *bs, int64_t sector_num,
++Zoned Block Devices (ZBDs) divide the LBA space into block regions called zones
-     QEMUIOVector hd_qiov;
++that are larger than the LBA size. They can only allow sequential writes, which
-     uint8_t *buf;
++can reduce write amplification in SSDs, and potentially lead to higher
-     void *orig_buf;
++throughput and increased capacity. More details about ZBDs can be found at:
--    Error *err = NULL;
++
++https://zonedstorage.io/docs/introduction/zoned-storage
-     if (qiov->niov > 1) {
++
-         buf = orig_buf = qemu_try_blockalign(bs, qiov->size);
++1. Block layer APIs for zoned storage
-@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_readv(BlockDriverState *bs, int64_t sector_num,
++-------------------------------------
-             if (bs->encrypted) {
++QEMU block layer supports three zoned storage models:
-                 assert(s->crypto);
++- BLK_Z_HM: The host-managed zoned model only allows sequential writes access
-                 if (qcrypto_block_decrypt(s->crypto, sector_num, buf,
++to zones. It supports ZBD-specific I/O commands that can be used by a host to
--                                          n * BDRV_SECTOR_SIZE, &err) < 0) {
++manage the zones of a device.
-+                                          n * BDRV_SECTOR_SIZE, NULL) < 0) {
++- BLK_Z_HA: The host-aware zoned model allows random write operations in
-                     goto fail;
++zones, making it backward compatible with regular block devices.
-                 }
++- BLK_Z_NONE: The non-zoned model has no zones support. It includes both
-             }
++regular and drive-managed ZBD devices. ZBD-specific I/O commands are not
-@@ -XXX,XX +XXX,XX @@ done:
++supported.
-     return ret;
++
++The block device information resides inside BlockDriverState. QEMU uses
- fail:
++BlockLimits struct(BlockDriverState::bl) that is continuously accessed by the
--    error_free(err);
++block layer while processing I/O requests. A BlockBackend has a root pointer to
-     ret = -EIO;
++a BlockDriverState graph(for example, raw format on top of file-posix). The
-     goto done;
++zoned storage information can be propagated from the leaf BlockDriverState all
- }
++the way up to the BlockBackend. If the zoned storage model in file-posix is
-@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t sector_num,
++set to BLK_Z_HM, then block drivers will declare support for zoned host device.
-             break;
++
-         }
++The block layer APIs support commands needed for zoned storage devices,
-         if (bs->encrypted) {
++including report zones, four zone operations, and zone append.
--            Error *err = NULL;
++
-             assert(s->crypto);
++2. Emulating zoned storage controllers
-             if (qcrypto_block_encrypt(s->crypto, sector_num, buf,
++--------------------------------------
--                                      n * BDRV_SECTOR_SIZE, &err) < 0) {
++When the BlockBackend's BlockLimits model reports a zoned storage device, users
--                error_free(err);
++like the virtio-blk emulation or the qemu-io-cmds.c utility can use block layer
-+                                      n * BDRV_SECTOR_SIZE, NULL) < 0) {
++APIs for zoned storage emulation or testing.
-                 ret = -EIO;
++
-                 break;
++For example, to test zone_report on a null_blk device using qemu-io is:
-             }
++$ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0
-diff --git a/block/qcow2.c b/block/qcow2.c
++-c "zrp offset nr_zones"
 diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
 index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2.c
+--- a/docs/system/qemu-block-drivers.rst.inc
-+++ b/block/qcow2.c
++++ b/docs/system/qemu-block-drivers.rst.inc
-@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset,
+@@ -XXX,XX +XXX,XX @@ Hard disks
-                 assert(s->crypto);
+   you may corrupt your host data (use the ``-snapshot`` command
-                 assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
+   line option or modify the device permissions accordingly).
-                 assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
--                Error *err = NULL;
++Zoned block devices
-                 if (qcrypto_block_decrypt(s->crypto,
++  Zoned block devices can be passed through to the guest if the emulated storage
-                                           (s->crypt_physical_offset ?
++  controller supports zoned storage. Use ``--blockdev host_device,
-                                            cluster_offset + offset_in_cluster :
++  node-name=drive0,filename=/dev/nullb0,cache.direct=on`` to pass through
-                                            offset) >> BDRV_SECTOR_BITS,
++  ``/dev/nullb0`` as ``drive0``.
-                                           cluster_data,
++
-                                           cur_bytes,
+ Windows
--                                          &err) < 0) {
+ ^^^^^^^
 -                    error_free(err);
 +                                          NULL) < 0) {
                      ret = -EIO;
                      goto fail;
                  }
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset,
          qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes);
          if (bs->encrypted) {
 -            Error *err = NULL;
              assert(s->crypto);
              if (!cluster_data) {
                  cluster_data = qemu_try_blockalign(bs->file->bs,
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset,
                                         cluster_offset + offset_in_cluster :
                                         offset) >> BDRV_SECTOR_BITS,
                                        cluster_data,
 -                                      cur_bytes, &err) < 0) {
 -                error_free(err);
 +                                      cur_bytes, NULL) < 0) {
                  ret = -EIO;
                  goto fail;
              }
 diff --git a/dump.c b/dump.c
 index XXXXXXX..XXXXXXX 100644
 --- a/dump.c
 +++ b/dump.c
@@ -XXX,XX +XXX,XX @@ static void dump_process(DumpState *s, Error **errp)
  static void *dump_thread(void *data)
  {
 -    Error *err = NULL;
      DumpState *s = (DumpState *)data;
 -    dump_process(s, &err);
 -    error_free(err);
 +    dump_process(s, NULL);
      return NULL;
  }
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 13/15] tests: migration/guestperf Python 2.6 argparse compatibility
+[PULL 09/20] block/dmg: Declare a type definition for DMG uncompress function
-Add the scripts/ directory to sys.path so Python 2.6 will be able to
+From: Philippe Mathieu-Daudé <philmd@linaro.org>
 import argparse.
-Cc: Daniel P. Berrange <berrange@redhat.com>
+Introduce the BdrvDmgUncompressFunc type defintion. To emphasis
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+dmg_uncompress_bz2 and dmg_uncompress_lzfse are pointer to functions,
-Acked-by: John Snow <jsnow@redhat.com>
+declare them using this new typedef.
-Acked-by: Fam Zheng <famz@redhat.com>
-Message-id: 20170825155732.15665-4-stefanha@redhat.com
+Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
 Message-id: 20230320152610.32052-1-philmd@linaro.org
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- tests/migration/guestperf/shell.py | 8 +++++---
+ block/dmg.h | 8 ++++----
-file changed, 5 insertions(+), 3 deletions(-)
+ block/dmg.c | 7 ++-----
 files changed, 6 insertions(+), 9 deletions(-)
-diff --git a/tests/migration/guestperf/shell.py b/tests/migration/guestperf/shell.py
+diff --git a/block/dmg.h b/block/dmg.h
 index XXXXXXX..XXXXXXX 100644
---- a/tests/migration/guestperf/shell.py
+--- a/block/dmg.h
-+++ b/tests/migration/guestperf/shell.py
++++ b/block/dmg.h
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVDMGState {
      z_stream zstream;
  } BDRVDMGState;
 -extern int (*dmg_uncompress_bz2)(char *next_in, unsigned int avail_in,
 -                                 char *next_out, unsigned int avail_out);
 +typedef int BdrvDmgUncompressFunc(char *next_in, unsigned int avail_in,
 +                                  char *next_out, unsigned int avail_out);
 -extern int (*dmg_uncompress_lzfse)(char *next_in, unsigned int avail_in,
 -                                   char *next_out, unsigned int avail_out);
 +extern BdrvDmgUncompressFunc *dmg_uncompress_bz2;
 +extern BdrvDmgUncompressFunc *dmg_uncompress_lzfse;
  #endif
 diff --git a/block/dmg.c b/block/dmg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/dmg.c
 +++ b/block/dmg.c
 @@ -XXX,XX +XXX,XX @@
- #
+ #include "qemu/memalign.h"
+ #include "dmg.h"
--import argparse
+-int (*dmg_uncompress_bz2)(char *next_in, unsigned int avail_in,
--import fnmatch
+-                          char *next_out, unsigned int avail_out);
- import os
+-
- import os.path
+-int (*dmg_uncompress_lzfse)(char *next_in, unsigned int avail_in,
--import platform
+-                            char *next_out, unsigned int avail_out);
- import sys
++BdrvDmgUncompressFunc *dmg_uncompress_bz2;
-+sys.path.append(os.path.join(os.path.dirname(__file__),
++BdrvDmgUncompressFunc *dmg_uncompress_lzfse;
-+                             '..', '..', '..', 'scripts'))
-+import argparse
+ enum {
-+import fnmatch
+     /* Limit chunk sizes to prevent unreasonable amounts of memory being used
 +import platform
  from guestperf.hardware import Hardware
  from guestperf.engine import Engine
 --
-.13.5
+.39.2

-New patch
+[PULL 10/20] tracetool: use relative paths for '#line' preprocessor directives
+From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
+The event filename is an absolute path. Convert it to a relative path when
+writing '#line' directives, to preserve reproducibility of the generated
+output when different base paths are used.
+Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-Id: <20230406080045.21696-1-thomas.de_schampheleire@nokia.com>
+---
+ scripts/tracetool/backend/ftrace.py | 4 +++-
+ scripts/tracetool/backend/log.py    | 4 +++-
+ scripts/tracetool/backend/syslog.py | 4 +++-
+files changed, 9 insertions(+), 3 deletions(-)
+diff --git a/scripts/tracetool/backend/ftrace.py b/scripts/tracetool/backend/ftrace.py
+index XXXXXXX..XXXXXXX 100644
+--- a/scripts/tracetool/backend/ftrace.py
++++ b/scripts/tracetool/backend/ftrace.py
+@@ -XXX,XX +XXX,XX @@
+ __email__      = "stefanha@redhat.com"
++import os.path
++
+ from tracetool import out
+@@ -XXX,XX +XXX,XX @@ def generate_h(event, group):
+         args=event.args,
+         event_id="TRACE_" + event.name.upper(),
+         event_lineno=event.lineno,
+-        event_filename=event.filename,
++        event_filename=os.path.relpath(event.filename),
+         fmt=event.fmt.rstrip("\n"),
+         argnames=argnames)
+diff --git a/scripts/tracetool/backend/log.py b/scripts/tracetool/backend/log.py
+index XXXXXXX..XXXXXXX 100644
+--- a/scripts/tracetool/backend/log.py
++++ b/scripts/tracetool/backend/log.py
+@@ -XXX,XX +XXX,XX @@
+ __email__      = "stefanha@redhat.com"
++import os.path
++
+ from tracetool import out
+@@ -XXX,XX +XXX,XX @@ def generate_h(event, group):
+         '    }',
+         cond=cond,
+         event_lineno=event.lineno,
+-        event_filename=event.filename,
++        event_filename=os.path.relpath(event.filename),
+         name=event.name,
+         fmt=event.fmt.rstrip("\n"),
+         argnames=argnames)
+diff --git a/scripts/tracetool/backend/syslog.py b/scripts/tracetool/backend/syslog.py
+index XXXXXXX..XXXXXXX 100644
+--- a/scripts/tracetool/backend/syslog.py
++++ b/scripts/tracetool/backend/syslog.py
+@@ -XXX,XX +XXX,XX @@
+ __email__      = "stefanha@redhat.com"
++import os.path
++
+ from tracetool import out
+@@ -XXX,XX +XXX,XX @@ def generate_h(event, group):
+         '    }',
+         cond=cond,
+         event_lineno=event.lineno,
+-        event_filename=event.filename,
++        event_filename=os.path.relpath(event.filename),
+         name=event.name,
+         fmt=event.fmt.rstrip("\n"),
+         argnames=argnames)
+--
+.39.2

-[Qemu-devel] [PULL for-2.10 15/15] qcow2: allocate cluster_cache/cluster_data on demand
+[PULL 11/20] file-posix: add tracking of the zone write pointers
-Most qcow2 files are uncompressed so it is wasteful to allocate (32 + 1)
+From: Sam Li <faithilikerun@gmail.com>
 * cluster_size + 512 bytes upfront.  Allocate s->cluster_cache and
 s->cluster_data when the first read operation is performance on a
 compressed cluster.
-The buffers are freed in .bdrv_close().  .bdrv_open() no longer has any
+Since Linux doesn't have a user API to issue zone append operations to
-code paths that can allocate these buffers, so remove the free functions
+zoned devices from user space, the file-posix driver is modified to add
-in the error code path.
+zone append emulation using regular writes. To do this, the file-posix
 driver tracks the wp location of all zones of the device. It uses an
 array of uint64_t. The most significant bit of each wp location indicates
 if the zone type is conventional zones.
-This patch can result in significant memory savings when many qcow2
+The zones wp can be changed due to the following operations issued:
-disks are attached or backing file chains are long:
+- zone reset: change the wp to the start offset of that zone
 - zone finish: change to the end location of that zone
 - write to a zone
 - zone append
-Before 12.81% (1,023,193,088B)
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-After   5.36% (393,893,888B)
+Message-id: 20230407081657.17947-2-faithilikerun@gmail.com
 Reported-by: Alexey Kardashevskiy <aik@ozlabs.ru>
 Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru>
 Reviewed-by: Eric Blake <eblake@redhat.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 Message-id: 20170821135530.32344-1-stefanha@redhat.com
 Cc: Kevin Wolf <kwolf@redhat.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/qcow2-cluster.c | 17 +++++++++++++++++
+ include/block/block-common.h     |  14 +++
- block/qcow2.c         | 12 ------------
+ include/block/block_int-common.h |   5 +
-files changed, 17 insertions(+), 12 deletions(-)
+ block/file-posix.c               | 173 ++++++++++++++++++++++++++++++-
 files changed, 189 insertions(+), 3 deletions(-)
-diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
+diff --git a/include/block/block-common.h b/include/block/block-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2-cluster.c
+--- a/include/block/block-common.h
-+++ b/block/qcow2-cluster.c
++++ b/include/block/block-common.h
-@@ -XXX,XX +XXX,XX @@ int qcow2_decompress_cluster(BlockDriverState *bs, uint64_t cluster_offset)
+@@ -XXX,XX +XXX,XX @@ typedef struct BlockZoneDescriptor {
-         nb_csectors = ((cluster_offset >> s->csize_shift) & s->csize_mask) + 1;
+     BlockZoneState state;
-         sector_offset = coffset & 511;
+ } BlockZoneDescriptor;
-         csize = nb_csectors * 512 - sector_offset;
-+
++/*
-+        /* Allocate buffers on first decompress operation, most images are
++ * Track write pointers of a zone in bytes.
-+         * uncompressed and the memory overhead can be avoided.  The buffers
++ */
-+         * are freed in .bdrv_close().
++typedef struct BlockZoneWps {
-+         */
++    CoMutex colock;
-+        if (!s->cluster_data) {
++    uint64_t wp[];
-+            /* one more sector for decompressed data alignment */
++} BlockZoneWps;
-+            s->cluster_data = qemu_try_blockalign(bs->file->bs,
++
-+                    QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size + 512);
+ typedef struct BlockDriverInfo {
-+            if (!s->cluster_data) {
+     /* in bytes, 0 if irrelevant */
-+                return -ENOMEM;
+     int cluster_size;
@@ -XXX,XX +XXX,XX @@ typedef enum {
  #define BDRV_SECTOR_BITS   9
  #define BDRV_SECTOR_SIZE   (1ULL << BDRV_SECTOR_BITS)
 +/*
 + * Get the first most significant bit of wp. If it is zero, then
 + * the zone type is SWR.
 + */
 +#define BDRV_ZT_IS_CONV(wp)    (wp & (1ULL << 63))
 +
  #define BDRV_REQUEST_MAX_SECTORS MIN_CONST(SIZE_MAX >> BDRV_SECTOR_BITS, \
                                             INT_MAX >> BDRV_SECTOR_BITS)
  #define BDRV_REQUEST_MAX_BYTES (BDRV_REQUEST_MAX_SECTORS << BDRV_SECTOR_BITS)
 diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block_int-common.h
 +++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ typedef struct BlockLimits {
      /* maximum number of active zones */
      int64_t max_active_zones;
 +
 +    int64_t write_granularity;
  } BlockLimits;
  typedef struct BdrvOpBlocker BdrvOpBlocker;
@@ -XXX,XX +XXX,XX @@ struct BlockDriverState {
      CoMutex bsc_modify_lock;
      /* Always non-NULL, but must only be dereferenced under an RCU read guard */
      BdrvBlockStatusCache *block_status_cache;
 +
 +    /* array of write pointers' location of each zone in the zoned device. */
 +    BlockZoneWps *wps;
  };
  struct BlockBackendRootState {
 diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/file-posix.c
 +++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int hdev_get_max_segments(int fd, struct stat *st)
  #endif
  }
 +#if defined(CONFIG_BLKZONED)
 +/*
 + * If the reset_all flag is true, then the wps of zone whose state is
 + * not readonly or offline should be all reset to the start sector.
 + * Else, take the real wp of the device.
 + */
 +static int get_zones_wp(BlockDriverState *bs, int fd, int64_t offset,
 +                        unsigned int nrz, bool reset_all)
 +{
 +    struct blk_zone *blkz;
 +    size_t rep_size;
 +    uint64_t sector = offset >> BDRV_SECTOR_BITS;
 +    BlockZoneWps *wps = bs->wps;
 +    int j = offset / bs->bl.zone_size;
 +    int ret, n = 0, i = 0;
 +    rep_size = sizeof(struct blk_zone_report) + nrz * sizeof(struct blk_zone);
 +    g_autofree struct blk_zone_report *rep = NULL;
 +
 +    rep = g_malloc(rep_size);
 +    blkz = (struct blk_zone *)(rep + 1);
 +    while (n < nrz) {
 +        memset(rep, 0, rep_size);
 +        rep->sector = sector;
 +        rep->nr_zones = nrz - n;
 +
 +        do {
 +            ret = ioctl(fd, BLKREPORTZONE, rep);
 +        } while (ret != 0 && errno == EINTR);
 +        if (ret != 0) {
 +            error_report("%d: ioctl BLKREPORTZONE at %" PRId64 " failed %d",
 +                    fd, offset, errno);
 +            return -errno;
 +        }
 +
 +        if (!rep->nr_zones) {
 +            break;
 +        }
 +
 +        for (i = 0; i < rep->nr_zones; ++i, ++n, ++j) {
 +            /*
 +             * The wp tracking cares only about sequential writes required and
 +             * sequential write preferred zones so that the wp can advance to
 +             * the right location.
 +             * Use the most significant bit of the wp location to indicate the
 +             * zone type: 0 for SWR/SWP zones and 1 for conventional zones.
 +             */
 +            if (blkz[i].type == BLK_ZONE_TYPE_CONVENTIONAL) {
 +                wps->wp[j] |= 1ULL << 63;
 +            } else {
 +                switch(blkz[i].cond) {
 +                case BLK_ZONE_COND_FULL:
 +                case BLK_ZONE_COND_READONLY:
 +                    /* Zone not writable */
 +                    wps->wp[j] = (blkz[i].start + blkz[i].len) << BDRV_SECTOR_BITS;
 +                    break;
 +                case BLK_ZONE_COND_OFFLINE:
 +                    /* Zone not writable nor readable */
 +                    wps->wp[j] = (blkz[i].start) << BDRV_SECTOR_BITS;
 +                    break;
 +                default:
 +                    if (reset_all) {
 +                        wps->wp[j] = blkz[i].start << BDRV_SECTOR_BITS;
 +                    } else {
 +                        wps->wp[j] = blkz[i].wp << BDRV_SECTOR_BITS;
 +                    }
 +                    break;
 +                }
 +            }
 +        }
-+        if (!s->cluster_cache) {
++        sector = blkz[i - 1].start + blkz[i - 1].len;
-+            s->cluster_cache = g_malloc(s->cluster_size);
++    }
-+        }
++
-+
++    return 0;
-         BLKDBG_EVENT(bs->file, BLKDBG_READ_COMPRESSED);
++}
-         ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data,
++
-                         nb_csectors);
++static void update_zones_wp(BlockDriverState *bs, int fd, int64_t offset,
-diff --git a/block/qcow2.c b/block/qcow2.c
++                            unsigned int nrz)
-index XXXXXXX..XXXXXXX 100644
++{
---- a/block/qcow2.c
++    if (get_zones_wp(bs, fd, offset, nrz, 0) < 0) {
-+++ b/block/qcow2.c
++        error_report("update zone wp failed");
-@@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags,
++    }
-         goto fail;
++}
-     }
++#endif
++
--    s->cluster_cache = g_malloc(s->cluster_size);
+ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
--    /* one more sector for decompressed data alignment */
+ {
--    s->cluster_data = qemu_try_blockalign(bs->file->bs, QCOW_MAX_CRYPT_CLUSTERS
+     BDRVRawState *s = bs->opaque;
--                                                    * s->cluster_size + 512);
+@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
--    if (s->cluster_data == NULL) {
+         if (ret >= 0) {
--        error_setg(errp, "Could not allocate temporary cluster buffer");
+             bs->bl.max_active_zones = ret;
--        ret = -ENOMEM;
+         }
--        goto fail;
++
--    }
++        ret = get_sysfs_long_val(&st, "physical_block_size");
--
++        if (ret >= 0) {
-     s->cluster_cache_offset = -1;
++            bs->bl.write_granularity = ret;
-     s->flags = flags;
++        }
++
-@@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags,
++        /* The refresh_limits() function can be called multiple times. */
-     if (s->refcount_block_cache) {
++        g_free(bs->wps);
-         qcow2_cache_destroy(bs, s->refcount_block_cache);
++        bs->wps = g_malloc(sizeof(BlockZoneWps) +
-     }
++                sizeof(int64_t) * bs->bl.nr_zones);
--    g_free(s->cluster_cache);
++        ret = get_zones_wp(bs, s->fd, 0, bs->bl.nr_zones, 0);
--    qemu_vfree(s->cluster_data);
++        if (ret < 0) {
-     qcrypto_block_free(s->crypto);
++            error_setg_errno(errp, -ret, "report wps failed");
-     qapi_free_QCryptoBlockOpenOptions(s->crypto_opts);
++            bs->wps = NULL;
 +            return;
 +        }
 +        qemu_co_mutex_init(&bs->wps->colock);
          return;
      }
  out:
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
  {
      BDRVRawState *s = bs->opaque;
      RawPosixAIOData acb;
 +    int ret;
      if (fd_open(bs) < 0)
          return -EIO;
 +#if defined(CONFIG_BLKZONED)
 +    if (type & QEMU_AIO_WRITE && bs->wps) {
 +        qemu_co_mutex_lock(&bs->wps->colock);
 +    }
 +#endif
      /*
       * When using O_DIRECT, the request must be aligned to be able to use
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
      } else if (s->use_linux_io_uring) {
          LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs));
          assert(qiov->size == bytes);
 -        return luring_co_submit(bs, aio, s->fd, offset, qiov, type);
 +        ret = luring_co_submit(bs, aio, s->fd, offset, qiov, type);
 +        goto out;
  #endif
  #ifdef CONFIG_LINUX_AIO
      } else if (s->use_linux_aio) {
          LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
          assert(qiov->size == bytes);
 -        return laio_co_submit(bs, aio, s->fd, offset, qiov, type,
 +        ret = laio_co_submit(bs, aio, s->fd, offset, qiov, type,
                                s->aio_max_batch);
 +        goto out;
  #endif
      }
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
      };
      assert(qiov->size == bytes);
 -    return raw_thread_pool_submit(bs, handle_aiocb_rw, &acb);
 +    ret = raw_thread_pool_submit(bs, handle_aiocb_rw, &acb);
 +
 +out:
 +#if defined(CONFIG_BLKZONED)
 +    BlockZoneWps *wps = bs->wps;
 +    if (ret == 0) {
 +        if (type & QEMU_AIO_WRITE && wps && bs->bl.zone_size) {
 +            uint64_t *wp = &wps->wp[offset / bs->bl.zone_size];
 +            if (!BDRV_ZT_IS_CONV(*wp)) {
 +                /* Advance the wp if needed */
 +                if (offset + bytes > *wp) {
 +                    *wp = offset + bytes;
 +                }
 +            }
 +        }
 +    } else {
 +        if (type & QEMU_AIO_WRITE) {
 +            update_zones_wp(bs, s->fd, 0, 1);
 +        }
 +    }
 +
 +    if (type & QEMU_AIO_WRITE && wps) {
 +        qemu_co_mutex_unlock(&wps->colock);
 +    }
 +#endif
 +    return ret;
  }
  static int coroutine_fn raw_co_preadv(BlockDriverState *bs, int64_t offset,
@@ -XXX,XX +XXX,XX @@ static void raw_close(BlockDriverState *bs)
      BDRVRawState *s = bs->opaque;
      if (s->fd >= 0) {
 +#if defined(CONFIG_BLKZONED)
 +        g_free(bs->wps);
 +#endif
          qemu_close(s->fd);
          s->fd = -1;
      }
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
      const char *op_name;
      unsigned long zo;
      int ret;
 +    BlockZoneWps *wps = bs->wps;
      int64_t capacity = bs->total_sectors << BDRV_SECTOR_BITS;
      zone_size = bs->bl.zone_size;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
          return -EINVAL;
      }
 +    QEMU_LOCK_GUARD(&wps->colock);
 +    uint32_t i = offset / bs->bl.zone_size;
 +    uint32_t nrz = len / bs->bl.zone_size;
 +    uint64_t *wp = &wps->wp[i];
 +    if (BDRV_ZT_IS_CONV(*wp) && len != capacity) {
 +        error_report("zone mgmt operations are not allowed for conventional zones");
 +        return -EIO;
 +    }
 +
      switch (op) {
      case BLK_ZO_OPEN:
          op_name = "BLKOPENZONE";
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
                          len >> BDRV_SECTOR_BITS);
      ret = raw_thread_pool_submit(bs, handle_aiocb_zone_mgmt, &acb);
      if (ret != 0) {
 +        update_zones_wp(bs, s->fd, offset, i);
          error_report("ioctl %s failed %d", op_name, ret);
 +        return ret;
 +    }
 +
 +    if (zo == BLKRESETZONE && len == capacity) {
 +        ret = get_zones_wp(bs, s->fd, 0, bs->bl.nr_zones, 1);
 +        if (ret < 0) {
 +            error_report("reporting single wp failed");
 +            return ret;
 +        }
 +    } else if (zo == BLKRESETZONE) {
 +        for (int j = 0; j < nrz; ++j) {
 +            wp[j] = offset + j * zone_size;
 +        }
 +    } else if (zo == BLKFINISHZONE) {
 +        for (int j = 0; j < nrz; ++j) {
 +            /* The zoned device allows the last zone smaller that the
 +             * zone size. */
 +            wp[j] = MIN(offset + (j + 1) * zone_size, offset + len);
 +        }
      }
      return ret;
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 09/15] oslib-posix: Print errors before aborting on qemu_alloc_stack()
+[PULL 12/20] block: introduce zone append write for zoned devices
-From: Eduardo Habkost <ehabkost@redhat.com>
+From: Sam Li <faithilikerun@gmail.com>
-If QEMU is running on a system that's out of memory and mmap()
+A zone append command is a write operation that specifies the first
-fails, QEMU aborts with no error message at all, making it hard
+logical block of a zone as the write position. When writing to a zoned
-to debug the reason for the failure.
+block device using zone append, the byte offset of the call may point at
 any position within the zone to which the data is being appended. Upon
 completion the device will respond with the position where the data has
 been written in the zone.
-Add perror() calls that will print error information before
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-aborting.
+Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
-Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
+Message-id: 20230407081657.17947-3-faithilikerun@gmail.com
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Message-id: 20170829212053.6003-1-ehabkost@redhat.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- util/oslib-posix.c | 2 ++
+ include/block/block-io.h          |  4 +++
-file changed, 2 insertions(+)
+ include/block/block_int-common.h  |  3 ++
  include/block/raw-aio.h           |  4 ++-
  include/sysemu/block-backend-io.h |  9 +++++
  block/block-backend.c             | 60 +++++++++++++++++++++++++++++++
  block/file-posix.c                | 58 ++++++++++++++++++++++++++----
  block/io.c                        | 27 ++++++++++++++
  block/io_uring.c                  |  4 +++
  block/linux-aio.c                 |  3 ++
  block/raw-format.c                |  8 +++++
 files changed, 172 insertions(+), 8 deletions(-)
-diff --git a/util/oslib-posix.c b/util/oslib-posix.c
+diff --git a/include/block/block-io.h b/include/block/block-io.h
 index XXXXXXX..XXXXXXX 100644
---- a/util/oslib-posix.c
+--- a/include/block/block-io.h
-+++ b/util/oslib-posix.c
++++ b/include/block/block-io.h
-@@ -XXX,XX +XXX,XX @@ void *qemu_alloc_stack(size_t *sz)
+@@ -XXX,XX +XXX,XX @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_report(BlockDriverState *bs,
-     ptr = mmap(NULL, *sz, PROT_READ | PROT_WRITE,
+ int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_mgmt(BlockDriverState *bs,
-                MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+                                                 BlockZoneOp op,
-     if (ptr == MAP_FAILED) {
+                                                 int64_t offset, int64_t len);
-+        perror("failed to allocate memory for stack");
++int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_append(BlockDriverState *bs,
-         abort();
++                                                  int64_t *offset,
 +                                                  QEMUIOVector *qiov,
 +                                                  BdrvRequestFlags flags);
  bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
  int bdrv_block_status(BlockDriverState *bs, int64_t offset,
 diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block_int-common.h
 +++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
              BlockZoneDescriptor *zones);
      int coroutine_fn (*bdrv_co_zone_mgmt)(BlockDriverState *bs, BlockZoneOp op,
              int64_t offset, int64_t len);
 +    int coroutine_fn (*bdrv_co_zone_append)(BlockDriverState *bs,
 +            int64_t *offset, QEMUIOVector *qiov,
 +            BdrvRequestFlags flags);
      /* removable device specific */
      bool coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_is_inserted)(
 diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/raw-aio.h
 +++ b/include/block/raw-aio.h
@@ -XXX,XX +XXX,XX @@
  #define QEMU_AIO_TRUNCATE     0x0080
  #define QEMU_AIO_ZONE_REPORT  0x0100
  #define QEMU_AIO_ZONE_MGMT    0x0200
 +#define QEMU_AIO_ZONE_APPEND  0x0400
  #define QEMU_AIO_TYPE_MASK \
          (QEMU_AIO_READ | \
           QEMU_AIO_WRITE | \
@@ -XXX,XX +XXX,XX @@
           QEMU_AIO_COPY_RANGE | \
           QEMU_AIO_TRUNCATE | \
           QEMU_AIO_ZONE_REPORT | \
 -         QEMU_AIO_ZONE_MGMT)
 +         QEMU_AIO_ZONE_MGMT | \
 +         QEMU_AIO_ZONE_APPEND)
  /* AIO flags */
  #define QEMU_AIO_MISALIGNED   0x1000
 diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backend-io.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/sysemu/block-backend-io.h
 +++ b/include/sysemu/block-backend-io.h
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_zone_report(BlockBackend *blk, int64_t offset,
  BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
                                int64_t offset, int64_t len,
                                BlockCompletionFunc *cb, void *opaque);
 +BlockAIOCB *blk_aio_zone_append(BlockBackend *blk, int64_t *offset,
 +                                QEMUIOVector *qiov, BdrvRequestFlags flags,
 +                                BlockCompletionFunc *cb, void *opaque);
  BlockAIOCB *blk_aio_pdiscard(BlockBackend *blk, int64_t offset, int64_t bytes,
                               BlockCompletionFunc *cb, void *opaque);
  void blk_aio_cancel_async(BlockAIOCB *acb);
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
                                    int64_t offset, int64_t len);
  int co_wrapper_mixed blk_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
                                         int64_t offset, int64_t len);
 +int coroutine_fn blk_co_zone_append(BlockBackend *blk, int64_t *offset,
 +                                    QEMUIOVector *qiov,
 +                                    BdrvRequestFlags flags);
 +int co_wrapper_mixed blk_zone_append(BlockBackend *blk, int64_t *offset,
 +                                         QEMUIOVector *qiov,
 +                                         BdrvRequestFlags flags);
  int co_wrapper_mixed blk_pdiscard(BlockBackend *blk, int64_t offset,
                                    int64_t bytes);
 diff --git a/block/block-backend.c b/block/block-backend.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/block-backend.c
 +++ b/block/block-backend.c
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
      return &acb->common;
  }
 +static void coroutine_fn blk_aio_zone_append_entry(void *opaque)
 +{
 +    BlkAioEmAIOCB *acb = opaque;
 +    BlkRwCo *rwco = &acb->rwco;
 +
 +    rwco->ret = blk_co_zone_append(rwco->blk, (int64_t *)acb->bytes,
 +                                   rwco->iobuf, rwco->flags);
 +    blk_aio_complete(acb);
 +}
 +
 +BlockAIOCB *blk_aio_zone_append(BlockBackend *blk, int64_t *offset,
 +                                QEMUIOVector *qiov, BdrvRequestFlags flags,
 +                                BlockCompletionFunc *cb, void *opaque) {
 +    BlkAioEmAIOCB *acb;
 +    Coroutine *co;
 +    IO_CODE();
 +
 +    blk_inc_in_flight(blk);
 +    acb = blk_aio_get(&blk_aio_em_aiocb_info, blk, cb, opaque);
 +    acb->rwco = (BlkRwCo) {
 +        .blk    = blk,
 +        .ret    = NOT_DONE,
 +        .flags  = flags,
 +        .iobuf  = qiov,
 +    };
 +    acb->bytes = (int64_t)offset;
 +    acb->has_returned = false;
 +
 +    co = qemu_coroutine_create(blk_aio_zone_append_entry, acb);
 +    aio_co_enter(blk_get_aio_context(blk), co);
 +    acb->has_returned = true;
 +    if (acb->rwco.ret != NOT_DONE) {
 +        replay_bh_schedule_oneshot_event(blk_get_aio_context(blk),
 +                                         blk_aio_complete_bh, acb);
 +    }
 +
 +    return &acb->common;
 +}
 +
  /*
   * Send a zone_report command.
   * offset is a byte offset from the start of the device. No alignment
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
      return ret;
  }
 +/*
 + * Send a zone_append command.
 + */
 +int coroutine_fn blk_co_zone_append(BlockBackend *blk, int64_t *offset,
 +        QEMUIOVector *qiov, BdrvRequestFlags flags)
 +{
 +    int ret;
 +    IO_CODE();
 +
 +    blk_inc_in_flight(blk);
 +    blk_wait_while_drained(blk);
 +    if (!blk_is_available(blk)) {
 +        blk_dec_in_flight(blk);
 +        return -ENOMEDIUM;
 +    }
 +
 +    ret = bdrv_co_zone_append(blk_bs(blk), offset, qiov, flags);
 +    blk_dec_in_flight(blk);
 +    return ret;
 +}
 +
  void blk_drain(BlockBackend *blk)
  {
      BlockDriverState *bs = blk_bs(blk);
 diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/file-posix.c
 +++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVRawState {
      bool has_write_zeroes:1;
      bool use_linux_aio:1;
      bool use_linux_io_uring:1;
 +    int64_t *offset; /* offset of zone append operation */
      int page_cache_inconsistent; /* errno from fdatasync failure */
      bool has_fallocate;
      bool needs_alignment;
@@ -XXX,XX +XXX,XX @@ static ssize_t handle_aiocb_rw_vector(RawPosixAIOData *aiocb)
      ssize_t len;
      len = RETRY_ON_EINTR(
 -        (aiocb->aio_type & QEMU_AIO_WRITE) ?
 +        (aiocb->aio_type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) ?
              qemu_pwritev(aiocb->aio_fildes,
                             aiocb->io.iov,
                             aiocb->io.niov,
@@ -XXX,XX +XXX,XX @@ static ssize_t handle_aiocb_rw_linear(RawPosixAIOData *aiocb, char *buf)
      ssize_t len;
      while (offset < aiocb->aio_nbytes) {
 -        if (aiocb->aio_type & QEMU_AIO_WRITE) {
 +        if (aiocb->aio_type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) {
              len = pwrite(aiocb->aio_fildes,
                           (const char *)buf + offset,
                           aiocb->aio_nbytes - offset,
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_rw(void *opaque)
      }
-@@ -XXX,XX +XXX,XX @@ void *qemu_alloc_stack(size_t *sz)
+     nbytes = handle_aiocb_rw_linear(aiocb, buf);
-     guardpage = ptr;
+-    if (!(aiocb->aio_type & QEMU_AIO_WRITE)) {
 +    if (!(aiocb->aio_type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND))) {
          char *p = buf;
          size_t count = aiocb->aio_nbytes, copy;
          int i;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
      if (fd_open(bs) < 0)
          return -EIO;
  #if defined(CONFIG_BLKZONED)
 -    if (type & QEMU_AIO_WRITE && bs->wps) {
 +    if ((type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) && bs->wps) {
          qemu_co_mutex_lock(&bs->wps->colock);
 +        if (type & QEMU_AIO_ZONE_APPEND && bs->bl.zone_size) {
 +            int index = offset / bs->bl.zone_size;
 +            offset = bs->wps->wp[index];
 +        }
      }
  #endif
-     if (mprotect(guardpage, pagesz, PROT_NONE) != 0) {
-+        perror("failed to set up stack guard page");
+@@ -XXX,XX +XXX,XX @@ out:
-         abort();
+ #if defined(CONFIG_BLKZONED)
      BlockZoneWps *wps = bs->wps;
      if (ret == 0) {
 -        if (type & QEMU_AIO_WRITE && wps && bs->bl.zone_size) {
 +        if ((type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND))
 +            && wps && bs->bl.zone_size) {
              uint64_t *wp = &wps->wp[offset / bs->bl.zone_size];
              if (!BDRV_ZT_IS_CONV(*wp)) {
 +                if (type & QEMU_AIO_ZONE_APPEND) {
 +                    *s->offset = *wp;
 +                }
                  /* Advance the wp if needed */
                  if (offset + bytes > *wp) {
                      *wp = offset + bytes;
@@ -XXX,XX +XXX,XX @@ out:
              }
          }
      } else {
 -        if (type & QEMU_AIO_WRITE) {
 +        if (type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) {
              update_zones_wp(bs, s->fd, 0, 1);
          }
      }
+-    if (type & QEMU_AIO_WRITE && wps) {
++    if ((type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) && wps) {
+         qemu_co_mutex_unlock(&wps->colock);
+     }
+ #endif
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
+ }
+ #endif
++#if defined(CONFIG_BLKZONED)
++static int coroutine_fn raw_co_zone_append(BlockDriverState *bs,
++                                           int64_t *offset,
++                                           QEMUIOVector *qiov,
++                                           BdrvRequestFlags flags) {
++    assert(flags == 0);
++    int64_t zone_size_mask = bs->bl.zone_size - 1;
++    int64_t iov_len = 0;
++    int64_t len = 0;
++    BDRVRawState *s = bs->opaque;
++    s->offset = offset;
++
++    if (*offset & zone_size_mask) {
++        error_report("sector offset %" PRId64 " is not aligned to zone size "
++                     "%" PRId32 "", *offset / 512, bs->bl.zone_size / 512);
++        return -EINVAL;
++    }
++
++    int64_t wg = bs->bl.write_granularity;
++    int64_t wg_mask = wg - 1;
++    for (int i = 0; i < qiov->niov; i++) {
++        iov_len = qiov->iov[i].iov_len;
++        if (iov_len & wg_mask) {
++            error_report("len of IOVector[%d] %" PRId64 " is not aligned to "
++                         "block size %" PRId64 "", i, iov_len, wg);
++            return -EINVAL;
++        }
++        len += iov_len;
++    }
++
++    return raw_co_prw(bs, *offset, len, qiov, QEMU_AIO_ZONE_APPEND);
++}
++#endif
++
+ static coroutine_fn int
+ raw_do_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes,
+                 bool blkdev)
+@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_host_device = {
+     /* zone management operations */
+     .bdrv_co_zone_report = raw_co_zone_report,
+     .bdrv_co_zone_mgmt = raw_co_zone_mgmt,
++    .bdrv_co_zone_append = raw_co_zone_append,
+ #endif
+ };
+diff --git a/block/io.c b/block/io.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/io.c
++++ b/block/io.c
+@@ -XXX,XX +XXX,XX @@ out:
+     return co.ret;
+ }
++int coroutine_fn bdrv_co_zone_append(BlockDriverState *bs, int64_t *offset,
++                        QEMUIOVector *qiov,
++                        BdrvRequestFlags flags)
++{
++    int ret;
++    BlockDriver *drv = bs->drv;
++    CoroutineIOCompletion co = {
++            .coroutine = qemu_coroutine_self(),
++    };
++    IO_CODE();
++
++    ret = bdrv_check_qiov_request(*offset, qiov->size, qiov, 0, NULL);
++    if (ret < 0) {
++        return ret;
++    }
++
++    bdrv_inc_in_flight(bs);
++    if (!drv || !drv->bdrv_co_zone_append || bs->bl.zoned == BLK_Z_NONE) {
++        co.ret = -ENOTSUP;
++        goto out;
++    }
++    co.ret = drv->bdrv_co_zone_append(bs, offset, qiov, flags);
++out:
++    bdrv_dec_in_flight(bs);
++    return co.ret;
++}
++
+ void *qemu_blockalign(BlockDriverState *bs, size_t size)
+ {
+     IO_CODE();
+diff --git a/block/io_uring.c b/block/io_uring.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/io_uring.c
++++ b/block/io_uring.c
+@@ -XXX,XX +XXX,XX @@ static int luring_do_submit(int fd, LuringAIOCB *luringcb, LuringState *s,
+         io_uring_prep_writev(sqes, fd, luringcb->qiov->iov,
+                              luringcb->qiov->niov, offset);
+         break;
++    case QEMU_AIO_ZONE_APPEND:
++        io_uring_prep_writev(sqes, fd, luringcb->qiov->iov,
++                             luringcb->qiov->niov, offset);
++        break;
+     case QEMU_AIO_READ:
+         io_uring_prep_readv(sqes, fd, luringcb->qiov->iov,
+                             luringcb->qiov->niov, offset);
+diff --git a/block/linux-aio.c b/block/linux-aio.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/linux-aio.c
++++ b/block/linux-aio.c
+@@ -XXX,XX +XXX,XX @@ static int laio_do_submit(int fd, struct qemu_laiocb *laiocb, off_t offset,
+     case QEMU_AIO_WRITE:
+         io_prep_pwritev(iocbs, fd, qiov->iov, qiov->niov, offset);
+         break;
++    case QEMU_AIO_ZONE_APPEND:
++        io_prep_pwritev(iocbs, fd, qiov->iov, qiov->niov, offset);
++        break;
+     case QEMU_AIO_READ:
+         io_prep_preadv(iocbs, fd, qiov->iov, qiov->niov, offset);
+         break;
+diff --git a/block/raw-format.c b/block/raw-format.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/raw-format.c
++++ b/block/raw-format.c
+@@ -XXX,XX +XXX,XX @@ raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
+     return bdrv_co_zone_mgmt(bs->file->bs, op, offset, len);
+ }
++static int coroutine_fn GRAPH_RDLOCK
++raw_co_zone_append(BlockDriverState *bs,int64_t *offset, QEMUIOVector *qiov,
++                   BdrvRequestFlags flags)
++{
++    return bdrv_co_zone_append(bs->file->bs, offset, qiov, flags);
++}
++
+ static int64_t coroutine_fn GRAPH_RDLOCK
+ raw_co_getlength(BlockDriverState *bs)
+ {
+@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_raw = {
+     .bdrv_co_pdiscard     = &raw_co_pdiscard,
+     .bdrv_co_zone_report  = &raw_co_zone_report,
+     .bdrv_co_zone_mgmt  = &raw_co_zone_mgmt,
++    .bdrv_co_zone_append = &raw_co_zone_append,
+     .bdrv_co_block_status = &raw_co_block_status,
+     .bdrv_co_copy_range_from = &raw_co_copy_range_from,
+     .bdrv_co_copy_range_to  = &raw_co_copy_range_to,
 --
-.13.5
+.39.2

-New patch
+[PULL 13/20] qemu-iotests: test zone append operation
+From: Sam Li <faithilikerun@gmail.com>
+The patch tests zone append writes by reporting the zone wp after
+the completion of the call. "zap -p" option can print the sector
+offset value after completion, which should be the start sector
+where the append write begins.
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-id: 20230407081657.17947-4-faithilikerun@gmail.com
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+---
+ qemu-io-cmds.c                     | 75 ++++++++++++++++++++++++++++++
+ tests/qemu-iotests/tests/zoned     | 16 +++++++
+ tests/qemu-iotests/tests/zoned.out | 16 +++++++
+files changed, 107 insertions(+)
+diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
+index XXXXXXX..XXXXXXX 100644
+--- a/qemu-io-cmds.c
++++ b/qemu-io-cmds.c
+@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t zone_reset_cmd = {
+     .oneline = "reset a zone write pointer in zone block device",
+ };
++static int do_aio_zone_append(BlockBackend *blk, QEMUIOVector *qiov,
++                              int64_t *offset, int flags, int *total)
++{
++    int async_ret = NOT_DONE;
++
++    blk_aio_zone_append(blk, offset, qiov, flags, aio_rw_done, &async_ret);
++    while (async_ret == NOT_DONE) {
++        main_loop_wait(false);
++    }
++
++    *total = qiov->size;
++    return async_ret < 0 ? async_ret : 1;
++}
++
++static int zone_append_f(BlockBackend *blk, int argc, char **argv)
++{
++    int ret;
++    bool pflag = false;
++    int flags = 0;
++    int total = 0;
++    int64_t offset;
++    char *buf;
++    int c, nr_iov;
++    int pattern = 0xcd;
++    QEMUIOVector qiov;
++
++    if (optind > argc - 3) {
++        return -EINVAL;
++    }
++
++    if ((c = getopt(argc, argv, "p")) != -1) {
++        pflag = true;
++    }
++
++    offset = cvtnum(argv[optind]);
++    if (offset < 0) {
++        print_cvtnum_err(offset, argv[optind]);
++        return offset;
++    }
++    optind++;
++    nr_iov = argc - optind;
++    buf = create_iovec(blk, &qiov, &argv[optind], nr_iov, pattern,
++                       flags & BDRV_REQ_REGISTERED_BUF);
++    if (buf == NULL) {
++        return -EINVAL;
++    }
++    ret = do_aio_zone_append(blk, &qiov, &offset, flags, &total);
++    if (ret < 0) {
++        printf("zone append failed: %s\n", strerror(-ret));
++        goto out;
++    }
++
++    if (pflag) {
++        printf("After zap done, the append sector is 0x%" PRIx64 "\n",
++               tosector(offset));
++    }
++
++out:
++    qemu_io_free(blk, buf, qiov.size,
++                 flags & BDRV_REQ_REGISTERED_BUF);
++    qemu_iovec_destroy(&qiov);
++    return ret;
++}
++
++static const cmdinfo_t zone_append_cmd = {
++    .name = "zone_append",
++    .altname = "zap",
++    .cfunc = zone_append_f,
++    .argmin = 3,
++    .argmax = 4,
++    .args = "offset len [len..]",
++    .oneline = "append write a number of bytes at a specified offset",
++};
++
+ static int truncate_f(BlockBackend *blk, int argc, char **argv);
+ static const cmdinfo_t truncate_cmd = {
+     .name       = "truncate",
+@@ -XXX,XX +XXX,XX @@ static void __attribute((constructor)) init_qemuio_commands(void)
+     qemuio_add_command(&zone_close_cmd);
+     qemuio_add_command(&zone_finish_cmd);
+     qemuio_add_command(&zone_reset_cmd);
++    qemuio_add_command(&zone_append_cmd);
+     qemuio_add_command(&truncate_cmd);
+     qemuio_add_command(&length_cmd);
+     qemuio_add_command(&info_cmd);
+diff --git a/tests/qemu-iotests/tests/zoned b/tests/qemu-iotests/tests/zoned
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/tests/zoned
++++ b/tests/qemu-iotests/tests/zoned
+@@ -XXX,XX +XXX,XX @@ echo "(5) resetting the second zone"
+ $QEMU_IO $IMG -c "zrs 268435456 268435456"
+ echo "After resetting a zone:"
+ $QEMU_IO $IMG -c "zrp 268435456 1"
++echo
++echo
++echo "(6) append write" # the physical block size of the device is 4096
++$QEMU_IO $IMG -c "zrp 0 1"
++$QEMU_IO $IMG -c "zap -p 0 0x1000 0x2000"
++echo "After appending the first zone firstly:"
++$QEMU_IO $IMG -c "zrp 0 1"
++$QEMU_IO $IMG -c "zap -p 0 0x1000 0x2000"
++echo "After appending the first zone secondly:"
++$QEMU_IO $IMG -c "zrp 0 1"
++$QEMU_IO $IMG -c "zap -p 268435456 0x1000 0x2000"
++echo "After appending the second zone firstly:"
++$QEMU_IO $IMG -c "zrp 268435456 1"
++$QEMU_IO $IMG -c "zap -p 268435456 0x1000 0x2000"
++echo "After appending the second zone secondly:"
++$QEMU_IO $IMG -c "zrp 268435456 1"
+ # success, all done
+ echo "*** done"
+diff --git a/tests/qemu-iotests/tests/zoned.out b/tests/qemu-iotests/tests/zoned.out
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/tests/zoned.out
++++ b/tests/qemu-iotests/tests/zoned.out
+@@ -XXX,XX +XXX,XX @@ start: 0x80000, len 0x80000, cap 0x80000, wptr 0x100000, zcond:14, [type: 2]
+ (5) resetting the second zone
+ After resetting a zone:
+ start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:1, [type: 2]
++
++
++(6) append write
++start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
++After zap done, the append sector is 0x0
++After appending the first zone firstly:
++start: 0x0, len 0x80000, cap 0x80000, wptr 0x18, zcond:2, [type: 2]
++After zap done, the append sector is 0x18
++After appending the first zone secondly:
++start: 0x0, len 0x80000, cap 0x80000, wptr 0x30, zcond:2, [type: 2]
++After zap done, the append sector is 0x80000
++After appending the second zone firstly:
++start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80018, zcond:2, [type: 2]
++After zap done, the append sector is 0x80018
++After appending the second zone secondly:
++start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80030, zcond:2, [type: 2]
+ *** done
+--
+.39.2

-New patch
+[PULL 14/20] block: add some trace events for zone append
+From: Sam Li <faithilikerun@gmail.com>
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
+Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-id: 20230407081657.17947-5-faithilikerun@gmail.com
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+---
+ block/file-posix.c | 3 +++
+ block/trace-events | 2 ++
+files changed, 5 insertions(+)
+diff --git a/block/file-posix.c b/block/file-posix.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/file-posix.c
++++ b/block/file-posix.c
+@@ -XXX,XX +XXX,XX @@ out:
+             if (!BDRV_ZT_IS_CONV(*wp)) {
+                 if (type & QEMU_AIO_ZONE_APPEND) {
+                     *s->offset = *wp;
++                    trace_zbd_zone_append_complete(bs, *s->offset
++                        >> BDRV_SECTOR_BITS);
+                 }
+                 /* Advance the wp if needed */
+                 if (offset + bytes > *wp) {
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_append(BlockDriverState *bs,
+         len += iov_len;
+     }
++    trace_zbd_zone_append(bs, *offset >> BDRV_SECTOR_BITS);
+     return raw_co_prw(bs, *offset, len, qiov, QEMU_AIO_ZONE_APPEND);
+ }
+ #endif
+diff --git a/block/trace-events b/block/trace-events
+index XXXXXXX..XXXXXXX 100644
+--- a/block/trace-events
++++ b/block/trace-events
+@@ -XXX,XX +XXX,XX @@ file_hdev_is_sg(int type, int version) "SG device found: type=%d, version=%d"
+ file_flush_fdatasync_failed(int err) "errno %d"
+ zbd_zone_report(void *bs, unsigned int nr_zones, int64_t sector) "bs %p report %d zones starting at sector offset 0x%" PRIx64 ""
+ zbd_zone_mgmt(void *bs, const char *op_name, int64_t sector, int64_t len) "bs %p %s starts at sector offset 0x%" PRIx64 " over a range of 0x%" PRIx64 " sectors"
++zbd_zone_append(void *bs, int64_t sector) "bs %p append at sector offset 0x%" PRIx64 ""
++zbd_zone_append_complete(void *bs, int64_t sector) "bs %p returns append sector 0x%" PRIx64 ""
+ # ssh.c
+ sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
+--
+.39.2

-[Qemu-devel] [PULL for-2.10 03/15] throttle: Update the throttle_fix_bucket() documentation
+[PULL 15/20] include: update virtio_blk headers to v6.3-rc1
-From: Alberto Garcia <berto@igalia.com>
+From: Sam Li <faithilikerun@gmail.com>
-The way the throttling algorithm works is that requests start being
+Use scripts/update-linux-headers.sh to update headers to 6.3-rc1.
 throttled once the bucket level exceeds the burst limit. When we get
 there the bucket leaks at the level set by the user (bkt->avg), and
 that leak rate is what prevents guest I/O from exceeding the desired
 limit.
-If we don't allow bursts (i.e. bkt->max == 0) then we can start
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-throttling requests immediately. The problem with keeping the
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
-threshold at 0 is that it only allows one request at a time, and as
+Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
-soon as there's a bit of I/O from the guest every other request will
+Message-id: 20230407082528.18841-2-faithilikerun@gmail.com
 be throttled and performance will suffer considerably. That can even
 make the guest unable to reach the throttle limit if that limit is
 high enough, and that happens regardless of the block scheduler used
 by the guest.
 Increasing that threshold gives flexibility to the guest, allowing it
 to perform short bursts of I/O before being throttled. Increasing the
 threshold too much does not make a difference in the long run (because
 it's the leak rate what defines the actual throughput) but it does
 allow the guest to perform longer initial bursts and exceed the
 throttle limit for a short while.
 A burst value of bkt->avg / 10 allows the guest to perform 100ms'
 worth of I/O at the target rate without being throttled.
 Signed-off-by: Alberto Garcia <berto@igalia.com>
 Message-id: 31aae6645f0d1fbf3860fb2b528b757236f0c0a7.1503580370.git.berto@igalia.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- util/throttle.c | 11 +++--------
+ include/standard-headers/drm/drm_fourcc.h    |  12 +++
-file changed, 3 insertions(+), 8 deletions(-)
+ include/standard-headers/linux/ethtool.h     |  48 ++++++++-
  include/standard-headers/linux/fuse.h        |  45 +++++++-
  include/standard-headers/linux/pci_regs.h    |   1 +
  include/standard-headers/linux/vhost_types.h |   2 +
  include/standard-headers/linux/virtio_blk.h  | 105 +++++++++++++++++++
  linux-headers/asm-arm64/kvm.h                |   1 +
  linux-headers/asm-x86/kvm.h                  |  34 +++++-
  linux-headers/linux/kvm.h                    |   9 ++
  linux-headers/linux/vfio.h                   |  15 +--
  linux-headers/linux/vhost.h                  |   8 ++
 files changed, 270 insertions(+), 10 deletions(-)
-diff --git a/util/throttle.c b/util/throttle.c
+diff --git a/include/standard-headers/drm/drm_fourcc.h b/include/standard-headers/drm/drm_fourcc.h
 index XXXXXXX..XXXXXXX 100644
---- a/util/throttle.c
+--- a/include/standard-headers/drm/drm_fourcc.h
-+++ b/util/throttle.c
++++ b/include/standard-headers/drm/drm_fourcc.h
-@@ -XXX,XX +XXX,XX @@ static void throttle_fix_bucket(LeakyBucket *bkt)
+@@ -XXX,XX +XXX,XX @@ extern "C" {
-     /* zero bucket level */
+  *
-     bkt->level = bkt->burst_level = 0;
+  * The authoritative list of format modifier codes is found in
+  * `include/uapi/drm/drm_fourcc.h`
--    /* The following is done to cope with the Linux CFQ block scheduler
++ *
--     * which regroup reads and writes by block of 100ms in the guest.
++ * Open Source User Waiver
--     * When they are two process one making reads and one making writes cfq
++ * -----------------------
--     * make a pattern looking like the following:
++ *
--     * WWWWWWWWWWWRRRRRRRRRRRRRRWWWWWWWWWWWWWwRRRRRRRRRRRRRRRRR
++ * Because this is the authoritative source for pixel formats and modifiers
--     * Having a max burst value of 100ms of the average will help smooth the
++ * referenced by GL, Vulkan extensions and other standards and hence used both
--     * throttling
++ * by open source and closed source driver stacks, the usual requirement for an
--     */
++ * upstream in-kernel or open source userspace user does not apply.
-+    /* If bkt->max is 0 we still want to allow short bursts of I/O
++ *
-+     * from the guest, otherwise every other request will be throttled
++ * To ensure, as much as feasible, compatibility across stacks and avoid
-+     * and performance will suffer considerably. */
++ * confusion with incompatible enumerations stakeholders for all relevant driver
-     min = bkt->avg / 10;
++ * stacks should approve additions.
-     if (bkt->avg && !bkt->max) {
+  */
-         bkt->max = min;
  #define fourcc_code(a, b, c, d) ((uint32_t)(a) | ((uint32_t)(b) << 8) | \
 diff --git a/include/standard-headers/linux/ethtool.h b/include/standard-headers/linux/ethtool.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/standard-headers/linux/ethtool.h
 +++ b/include/standard-headers/linux/ethtool.h
@@ -XXX,XX +XXX,XX @@ enum ethtool_stringset {
      ETH_SS_COUNT
  };
 +/**
 + * enum ethtool_mac_stats_src - source of ethtool MAC statistics
 + * @ETHTOOL_MAC_STATS_SRC_AGGREGATE:
 + *    if device supports a MAC merge layer, this retrieves the aggregate
 + *    statistics of the eMAC and pMAC. Otherwise, it retrieves just the
 + *    statistics of the single (express) MAC.
 + * @ETHTOOL_MAC_STATS_SRC_EMAC:
 + *    if device supports a MM layer, this retrieves the eMAC statistics.
 + *    Otherwise, it retrieves the statistics of the single (express) MAC.
 + * @ETHTOOL_MAC_STATS_SRC_PMAC:
 + *    if device supports a MM layer, this retrieves the pMAC statistics.
 + */
 +enum ethtool_mac_stats_src {
 +    ETHTOOL_MAC_STATS_SRC_AGGREGATE,
 +    ETHTOOL_MAC_STATS_SRC_EMAC,
 +    ETHTOOL_MAC_STATS_SRC_PMAC,
 +};
 +
  /**
   * enum ethtool_module_power_mode_policy - plug-in module power mode policy
   * @ETHTOOL_MODULE_POWER_MODE_POLICY_HIGH: Module is always in high power mode.
@@ -XXX,XX +XXX,XX @@ enum ethtool_podl_pse_pw_d_status {
      ETHTOOL_PODL_PSE_PW_D_STATUS_ERROR,
  };
 +/**
 + * enum ethtool_mm_verify_status - status of MAC Merge Verify function
 + * @ETHTOOL_MM_VERIFY_STATUS_UNKNOWN:
 + *    verification status is unknown
 + * @ETHTOOL_MM_VERIFY_STATUS_INITIAL:
 + *    the 802.3 Verify State diagram is in the state INIT_VERIFICATION
 + * @ETHTOOL_MM_VERIFY_STATUS_VERIFYING:
 + *    the Verify State diagram is in the state VERIFICATION_IDLE,
 + *    SEND_VERIFY or WAIT_FOR_RESPONSE
 + * @ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED:
 + *    indicates that the Verify State diagram is in the state VERIFIED
 + * @ETHTOOL_MM_VERIFY_STATUS_FAILED:
 + *    the Verify State diagram is in the state VERIFY_FAIL
 + * @ETHTOOL_MM_VERIFY_STATUS_DISABLED:
 + *    verification of preemption operation is disabled
 + */
 +enum ethtool_mm_verify_status {
 +    ETHTOOL_MM_VERIFY_STATUS_UNKNOWN,
 +    ETHTOOL_MM_VERIFY_STATUS_INITIAL,
 +    ETHTOOL_MM_VERIFY_STATUS_VERIFYING,
 +    ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED,
 +    ETHTOOL_MM_VERIFY_STATUS_FAILED,
 +    ETHTOOL_MM_VERIFY_STATUS_DISABLED,
 +};
 +
  /**
   * struct ethtool_gstrings - string set for data tagging
   * @cmd: Command number = %ETHTOOL_GSTRINGS
@@ -XXX,XX +XXX,XX @@ struct ethtool_rxnfc {
          uint32_t            rule_cnt;
          uint32_t            rss_context;
      };
 -    uint32_t                rule_locs[0];
 +    uint32_t                rule_locs[];
  };
@@ -XXX,XX +XXX,XX @@ enum ethtool_link_mode_bit_indices {
      ETHTOOL_LINK_MODE_800000baseDR8_2_Full_BIT     = 96,
      ETHTOOL_LINK_MODE_800000baseSR8_Full_BIT     = 97,
      ETHTOOL_LINK_MODE_800000baseVR8_Full_BIT     = 98,
 +    ETHTOOL_LINK_MODE_10baseT1S_Full_BIT         = 99,
 +    ETHTOOL_LINK_MODE_10baseT1S_Half_BIT         = 100,
 +    ETHTOOL_LINK_MODE_10baseT1S_P2MP_Half_BIT     = 101,
      /* must be last entry */
      __ETHTOOL_LINK_MODE_MASK_NBITS
 diff --git a/include/standard-headers/linux/fuse.h b/include/standard-headers/linux/fuse.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/standard-headers/linux/fuse.h
 +++ b/include/standard-headers/linux/fuse.h
@@ -XXX,XX +XXX,XX @@
   *  7.38
   *  - add FUSE_EXPIRE_ONLY flag to fuse_notify_inval_entry
   *  - add FOPEN_PARALLEL_DIRECT_WRITES
 + *  - add total_extlen to fuse_in_header
 + *  - add FUSE_MAX_NR_SECCTX
 + *  - add extension header
 + *  - add FUSE_EXT_GROUPS
 + *  - add FUSE_CREATE_SUPP_GROUP
   */
  #ifndef _LINUX_FUSE_H
@@ -XXX,XX +XXX,XX @@ struct fuse_file_lock {
   * FUSE_SECURITY_CTX:    add security context to create, mkdir, symlink, and
   *            mknod
   * FUSE_HAS_INODE_DAX:  use per inode DAX
 + * FUSE_CREATE_SUPP_GROUP: add supplementary group info to create, mkdir,
 + *            symlink and mknod (single group that matches parent)
   */
  #define FUSE_ASYNC_READ        (1 << 0)
  #define FUSE_POSIX_LOCKS    (1 << 1)
@@ -XXX,XX +XXX,XX @@ struct fuse_file_lock {
  /* bits 32..63 get shifted down 32 bits into the flags2 field */
  #define FUSE_SECURITY_CTX    (1ULL << 32)
  #define FUSE_HAS_INODE_DAX    (1ULL << 33)
 +#define FUSE_CREATE_SUPP_GROUP    (1ULL << 34)
  /**
   * CUSE INIT request/reply flags
@@ -XXX,XX +XXX,XX @@ struct fuse_file_lock {
   */
  #define FUSE_EXPIRE_ONLY        (1 << 0)
 +/**
 + * extension type
 + * FUSE_MAX_NR_SECCTX: maximum value of &fuse_secctx_header.nr_secctx
 + * FUSE_EXT_GROUPS: &fuse_supp_groups extension
 + */
 +enum fuse_ext_type {
 +    /* Types 0..31 are reserved for fuse_secctx_header */
 +    FUSE_MAX_NR_SECCTX    = 31,
 +    FUSE_EXT_GROUPS        = 32,
 +};
 +
  enum fuse_opcode {
      FUSE_LOOKUP        = 1,
      FUSE_FORGET        = 2,  /* no reply */
@@ -XXX,XX +XXX,XX @@ struct fuse_in_header {
      uint32_t    uid;
      uint32_t    gid;
      uint32_t    pid;
 -    uint32_t    padding;
 +    uint16_t    total_extlen; /* length of extensions in 8byte units */
 +    uint16_t    padding;
  };
  struct fuse_out_header {
@@ -XXX,XX +XXX,XX @@ struct fuse_secctx_header {
      uint32_t    nr_secctx;
  };
 +/**
 + * struct fuse_ext_header - extension header
 + * @size: total size of this extension including this header
 + * @type: type of extension
 + *
 + * This is made compatible with fuse_secctx_header by using type values >
 + * FUSE_MAX_NR_SECCTX
 + */
 +struct fuse_ext_header {
 +    uint32_t    size;
 +    uint32_t    type;
 +};
 +
 +/**
 + * struct fuse_supp_groups - Supplementary group extension
 + * @nr_groups: number of supplementary groups
 + * @groups: flexible array of group IDs
 + */
 +struct fuse_supp_groups {
 +    uint32_t    nr_groups;
 +    uint32_t    groups[];
 +};
 +
  #endif /* _LINUX_FUSE_H */
 diff --git a/include/standard-headers/linux/pci_regs.h b/include/standard-headers/linux/pci_regs.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/standard-headers/linux/pci_regs.h
 +++ b/include/standard-headers/linux/pci_regs.h
@@ -XXX,XX +XXX,XX @@
  #define  PCI_EXP_LNKCTL2_TX_MARGIN    0x0380 /* Transmit Margin */
  #define  PCI_EXP_LNKCTL2_HASD        0x0020 /* HW Autonomous Speed Disable */
  #define PCI_EXP_LNKSTA2        0x32    /* Link Status 2 */
 +#define  PCI_EXP_LNKSTA2_FLIT        0x0400 /* Flit Mode Status */
  #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2    0x32    /* end of v2 EPs w/ link */
  #define PCI_EXP_SLTCAP2        0x34    /* Slot Capabilities 2 */
  #define  PCI_EXP_SLTCAP2_IBPD    0x00000001 /* In-band PD Disable Supported */
 diff --git a/include/standard-headers/linux/vhost_types.h b/include/standard-headers/linux/vhost_types.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/standard-headers/linux/vhost_types.h
 +++ b/include/standard-headers/linux/vhost_types.h
@@ -XXX,XX +XXX,XX @@ struct vhost_vdpa_iova_range {
  #define VHOST_BACKEND_F_IOTLB_ASID  0x3
  /* Device can be suspended */
  #define VHOST_BACKEND_F_SUSPEND  0x4
 +/* Device can be resumed */
 +#define VHOST_BACKEND_F_RESUME  0x5
  #endif
 diff --git a/include/standard-headers/linux/virtio_blk.h b/include/standard-headers/linux/virtio_blk.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/standard-headers/linux/virtio_blk.h
 +++ b/include/standard-headers/linux/virtio_blk.h
@@ -XXX,XX +XXX,XX @@
  #define VIRTIO_BLK_F_DISCARD    13    /* DISCARD is supported */
  #define VIRTIO_BLK_F_WRITE_ZEROES    14    /* WRITE ZEROES is supported */
  #define VIRTIO_BLK_F_SECURE_ERASE    16 /* Secure Erase is supported */
 +#define VIRTIO_BLK_F_ZONED        17    /* Zoned block device */
  /* Legacy feature bits */
  #ifndef VIRTIO_BLK_NO_LEGACY
@@ -XXX,XX +XXX,XX @@ struct virtio_blk_config {
      /* Secure erase commands must be aligned to this number of sectors. */
      __virtio32 secure_erase_sector_alignment;
 +    /* Zoned block device characteristics (if VIRTIO_BLK_F_ZONED) */
 +    struct virtio_blk_zoned_characteristics {
 +        uint32_t zone_sectors;
 +        uint32_t max_open_zones;
 +        uint32_t max_active_zones;
 +        uint32_t max_append_sectors;
 +        uint32_t write_granularity;
 +        uint8_t model;
 +        uint8_t unused2[3];
 +    } zoned;
  } QEMU_PACKED;
  /*
@@ -XXX,XX +XXX,XX @@ struct virtio_blk_config {
  /* Secure erase command */
  #define VIRTIO_BLK_T_SECURE_ERASE    14
 +/* Zone append command */
 +#define VIRTIO_BLK_T_ZONE_APPEND    15
 +
 +/* Report zones command */
 +#define VIRTIO_BLK_T_ZONE_REPORT    16
 +
 +/* Open zone command */
 +#define VIRTIO_BLK_T_ZONE_OPEN      18
 +
 +/* Close zone command */
 +#define VIRTIO_BLK_T_ZONE_CLOSE     20
 +
 +/* Finish zone command */
 +#define VIRTIO_BLK_T_ZONE_FINISH    22
 +
 +/* Reset zone command */
 +#define VIRTIO_BLK_T_ZONE_RESET     24
 +
 +/* Reset All zones command */
 +#define VIRTIO_BLK_T_ZONE_RESET_ALL 26
 +
  #ifndef VIRTIO_BLK_NO_LEGACY
  /* Barrier before this op. */
  #define VIRTIO_BLK_T_BARRIER    0x80000000
@@ -XXX,XX +XXX,XX @@ struct virtio_blk_outhdr {
      __virtio64 sector;
  };
 +/*
 + * Supported zoned device models.
 + */
 +
 +/* Regular block device */
 +#define VIRTIO_BLK_Z_NONE      0
 +/* Host-managed zoned device */
 +#define VIRTIO_BLK_Z_HM        1
 +/* Host-aware zoned device */
 +#define VIRTIO_BLK_Z_HA        2
 +
 +/*
 + * Zone descriptor. A part of VIRTIO_BLK_T_ZONE_REPORT command reply.
 + */
 +struct virtio_blk_zone_descriptor {
 +    /* Zone capacity */
 +    uint64_t z_cap;
 +    /* The starting sector of the zone */
 +    uint64_t z_start;
 +    /* Zone write pointer position in sectors */
 +    uint64_t z_wp;
 +    /* Zone type */
 +    uint8_t z_type;
 +    /* Zone state */
 +    uint8_t z_state;
 +    uint8_t reserved[38];
 +};
 +
 +struct virtio_blk_zone_report {
 +    uint64_t nr_zones;
 +    uint8_t reserved[56];
 +    struct virtio_blk_zone_descriptor zones[];
 +};
 +
 +/*
 + * Supported zone types.
 + */
 +
 +/* Conventional zone */
 +#define VIRTIO_BLK_ZT_CONV         1
 +/* Sequential Write Required zone */
 +#define VIRTIO_BLK_ZT_SWR          2
 +/* Sequential Write Preferred zone */
 +#define VIRTIO_BLK_ZT_SWP          3
 +
 +/*
 + * Zone states that are available for zones of all types.
 + */
 +
 +/* Not a write pointer (conventional zones only) */
 +#define VIRTIO_BLK_ZS_NOT_WP       0
 +/* Empty */
 +#define VIRTIO_BLK_ZS_EMPTY        1
 +/* Implicitly Open */
 +#define VIRTIO_BLK_ZS_IOPEN        2
 +/* Explicitly Open */
 +#define VIRTIO_BLK_ZS_EOPEN        3
 +/* Closed */
 +#define VIRTIO_BLK_ZS_CLOSED       4
 +/* Read-Only */
 +#define VIRTIO_BLK_ZS_RDONLY       13
 +/* Full */
 +#define VIRTIO_BLK_ZS_FULL         14
 +/* Offline */
 +#define VIRTIO_BLK_ZS_OFFLINE      15
 +
  /* Unmap this range (only valid for write zeroes command) */
  #define VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP    0x00000001
@@ -XXX,XX +XXX,XX @@ struct virtio_scsi_inhdr {
  #define VIRTIO_BLK_S_OK        0
  #define VIRTIO_BLK_S_IOERR    1
  #define VIRTIO_BLK_S_UNSUPP    2
 +
 +/* Error codes that are specific to zoned block devices */
 +#define VIRTIO_BLK_S_ZONE_INVALID_CMD     3
 +#define VIRTIO_BLK_S_ZONE_UNALIGNED_WP    4
 +#define VIRTIO_BLK_S_ZONE_OPEN_RESOURCE   5
 +#define VIRTIO_BLK_S_ZONE_ACTIVE_RESOURCE 6
 +
  #endif /* _LINUX_VIRTIO_BLK_H */
 diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h
 index XXXXXXX..XXXXXXX 100644
 --- a/linux-headers/asm-arm64/kvm.h
 +++ b/linux-headers/asm-arm64/kvm.h
@@ -XXX,XX +XXX,XX @@ struct kvm_regs {
  #define KVM_ARM_VCPU_SVE        4 /* enable SVE for this CPU */
  #define KVM_ARM_VCPU_PTRAUTH_ADDRESS    5 /* VCPU uses address authentication */
  #define KVM_ARM_VCPU_PTRAUTH_GENERIC    6 /* VCPU uses generic authentication */
 +#define KVM_ARM_VCPU_HAS_EL2        7 /* Support nested virtualization */
  struct kvm_vcpu_init {
      __u32 target;
 diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
 index XXXXXXX..XXXXXXX 100644
 --- a/linux-headers/asm-x86/kvm.h
 +++ b/linux-headers/asm-x86/kvm.h
@@ -XXX,XX +XXX,XX @@
  #include <linux/types.h>
  #include <linux/ioctl.h>
 +#include <linux/stddef.h>
  #define KVM_PIO_PAGE_OFFSET 1
  #define KVM_COALESCED_MMIO_PAGE_OFFSET 2
@@ -XXX,XX +XXX,XX @@ struct kvm_nested_state {
       * KVM_{GET,PUT}_NESTED_STATE ioctl values.
       */
      union {
 -        struct kvm_vmx_nested_state_data vmx[0];
 -        struct kvm_svm_nested_state_data svm[0];
 +        __DECLARE_FLEX_ARRAY(struct kvm_vmx_nested_state_data, vmx);
 +        __DECLARE_FLEX_ARRAY(struct kvm_svm_nested_state_data, svm);
      } data;
  };
@@ -XXX,XX +XXX,XX @@ struct kvm_pmu_event_filter {
  #define KVM_PMU_EVENT_ALLOW 0
  #define KVM_PMU_EVENT_DENY 1
 +#define KVM_PMU_EVENT_FLAG_MASKED_EVENTS BIT(0)
 +#define KVM_PMU_EVENT_FLAGS_VALID_MASK (KVM_PMU_EVENT_FLAG_MASKED_EVENTS)
 +
 +/*
 + * Masked event layout.
 + * Bits   Description
 + * ----   -----------
 + * 7:0    event select (low bits)
 + * 15:8   umask match
 + * 31:16  unused
 + * 35:32  event select (high bits)
 + * 36:54  unused
 + * 55     exclude bit
 + * 63:56  umask mask
 + */
 +
 +#define KVM_PMU_ENCODE_MASKED_ENTRY(event_select, mask, match, exclude) \
 +    (((event_select) & 0xFFULL) | (((event_select) & 0XF00ULL) << 24) | \
 +    (((mask) & 0xFFULL) << 56) | \
 +    (((match) & 0xFFULL) << 8) | \
 +    ((__u64)(!!(exclude)) << 55))
 +
 +#define KVM_PMU_MASKED_ENTRY_EVENT_SELECT \
 +    (GENMASK_ULL(7, 0) | GENMASK_ULL(35, 32))
 +#define KVM_PMU_MASKED_ENTRY_UMASK_MASK        (GENMASK_ULL(63, 56))
 +#define KVM_PMU_MASKED_ENTRY_UMASK_MATCH    (GENMASK_ULL(15, 8))
 +#define KVM_PMU_MASKED_ENTRY_EXCLUDE        (BIT_ULL(55))
 +#define KVM_PMU_MASKED_ENTRY_UMASK_MASK_SHIFT    (56)
 +
  /* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
  #define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
  #define   KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
 diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
 index XXXXXXX..XXXXXXX 100644
 --- a/linux-headers/linux/kvm.h
 +++ b/linux-headers/linux/kvm.h
@@ -XXX,XX +XXX,XX @@ struct kvm_s390_mem_op {
          struct {
              __u8 ar;    /* the access register number */
              __u8 key;    /* access key, ignored if flag unset */
 +            __u8 pad1[6];    /* ignored */
 +            __u64 old_addr;    /* ignored if cmpxchg flag unset */
          };
          __u32 sida_offset; /* offset into the sida */
          __u8 reserved[32]; /* ignored */
@@ -XXX,XX +XXX,XX @@ struct kvm_s390_mem_op {
  #define KVM_S390_MEMOP_SIDA_WRITE    3
  #define KVM_S390_MEMOP_ABSOLUTE_READ    4
  #define KVM_S390_MEMOP_ABSOLUTE_WRITE    5
 +#define KVM_S390_MEMOP_ABSOLUTE_CMPXCHG    6
 +
  /* flags for kvm_s390_mem_op->flags */
  #define KVM_S390_MEMOP_F_CHECK_ONLY        (1ULL << 0)
  #define KVM_S390_MEMOP_F_INJECT_EXCEPTION    (1ULL << 1)
  #define KVM_S390_MEMOP_F_SKEY_PROTECTION    (1ULL << 2)
 +/* flags specifying extension support via KVM_CAP_S390_MEM_OP_EXTENSION */
 +#define KVM_S390_MEMOP_EXTENSION_CAP_BASE    (1 << 0)
 +#define KVM_S390_MEMOP_EXTENSION_CAP_CMPXCHG    (1 << 1)
 +
  /* for KVM_INTERRUPT */
  struct kvm_interrupt {
      /* in */
@@ -XXX,XX +XXX,XX @@ struct kvm_ppc_resize_hpt {
  #define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223
  #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
  #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
 +#define KVM_CAP_PMU_EVENT_MASKED_EVENTS 226
  #ifdef KVM_CAP_IRQ_ROUTING
 diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
 index XXXXXXX..XXXXXXX 100644
 --- a/linux-headers/linux/vfio.h
 +++ b/linux-headers/linux/vfio.h
@@ -XXX,XX +XXX,XX @@
  /* Supports VFIO_DMA_UNMAP_FLAG_ALL */
  #define VFIO_UNMAP_ALL            9
 -/* Supports the vaddr flag for DMA map and unmap */
 +/*
 + * Supports the vaddr flag for DMA map and unmap.  Not supported for mediated
 + * devices, so this capability is subject to change as groups are added or
 + * removed.
 + */
  #define VFIO_UPDATE_VADDR        10
  /*
@@ -XXX,XX +XXX,XX @@ struct vfio_iommu_type1_info_dma_avail {
   * Map process virtual addresses to IO virtual addresses using the
   * provided struct vfio_dma_map. Caller sets argsz. READ &/ WRITE required.
   *
 - * If flags & VFIO_DMA_MAP_FLAG_VADDR, update the base vaddr for iova, and
 - * unblock translation of host virtual addresses in the iova range.  The vaddr
 + * If flags & VFIO_DMA_MAP_FLAG_VADDR, update the base vaddr for iova. The vaddr
   * must have previously been invalidated with VFIO_DMA_UNMAP_FLAG_VADDR.  To
   * maintain memory consistency within the user application, the updated vaddr
   * must address the same memory object as originally mapped.  Failure to do so
@@ -XXX,XX +XXX,XX @@ struct vfio_bitmap {
   * must be 0.  This cannot be combined with the get-dirty-bitmap flag.
   *
   * If flags & VFIO_DMA_UNMAP_FLAG_VADDR, do not unmap, but invalidate host
 - * virtual addresses in the iova range.  Tasks that attempt to translate an
 - * iova's vaddr will block.  DMA to already-mapped pages continues.  This
 - * cannot be combined with the get-dirty-bitmap flag.
 + * virtual addresses in the iova range.  DMA to already-mapped pages continues.
 + * Groups may not be added to the container while any addresses are invalid.
 + * This cannot be combined with the get-dirty-bitmap flag.
   */
  struct vfio_iommu_type1_dma_unmap {
      __u32    argsz;
 diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h
 index XXXXXXX..XXXXXXX 100644
 --- a/linux-headers/linux/vhost.h
 +++ b/linux-headers/linux/vhost.h
@@ -XXX,XX +XXX,XX @@
   */
  #define VHOST_VDPA_SUSPEND        _IO(VHOST_VIRTIO, 0x7D)
 +/* Resume a device so it can resume processing virtqueue requests
 + *
 + * After the return of this ioctl the device will have restored all the
 + * necessary states and it is fully operational to continue processing the
 + * virtqueue descriptors.
 + */
 +#define VHOST_VDPA_RESUME        _IO(VHOST_VIRTIO, 0x7E)
 +
  #endif
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 07/15] throttle: Make burst_length 64bit and add range checks
+[PULL 16/20] virtio-blk: add zoned storage emulation for zoned devices
-From: Alberto Garcia <berto@igalia.com>
+From: Sam Li <faithilikerun@gmail.com>
-LeakyBucket.burst_length is defined as an unsigned integer but the
+This patch extends virtio-blk emulation to handle zoned device commands
-code never checks for overflows and it only makes sure that the value
+by calling the new block layer APIs to perform zoned device I/O on
-is not 0.
+behalf of the guest. It supports Report Zone, four zone oparations (open,
 close, finish, reset), and Append Zone.
-In practice this means that the user can set something like
+The VIRTIO_BLK_F_ZONED feature bit will only be set if the host does
-throttling.iops-total-max-length=4294967300 despite being larger than
+support zoned block devices. Regular block devices(conventional zones)
-UINT_MAX and the final value after casting to unsigned int will be 4.
+will not be set.
-This patch changes the data type to uint64_t. This does not increase
+The guest os can use blktests, fio to test those commands on zoned devices.
-the storage size of LeakyBucket, and allows us to assign the value
+Furthermore, using zonefs to test zone append write is also supported.
 directly from qemu_opt_get_number() or BlockIOThrottle and then do the
 checks directly in throttle_is_valid().
-The value of burst_length does not have a specific upper limit,
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-but since the bucket size is defined by max * burst_length we have
+Message-id: 20230407082528.18841-3-faithilikerun@gmail.com
 to prevent overflows. Instead of going for UINT64_MAX or something
 similar this patch reuses THROTTLE_VALUE_MAX, which allows I/O bursts
 of 1 GiB/s for 10 days in a row.
 Signed-off-by: Alberto Garcia <berto@igalia.com>
 Message-id: 1b2e3049803f71cafb2e1fa1be4fb47147a0d398.1503580370.git.berto@igalia.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- include/qemu/throttle.h | 2 +-
+ hw/block/virtio-blk-common.c |   2 +
- util/throttle.c         | 5 +++++
+ hw/block/virtio-blk.c        | 389 +++++++++++++++++++++++++++++++++++
-files changed, 6 insertions(+), 1 deletion(-)
+ hw/virtio/virtio-qmp.c       |   2 +
 files changed, 393 insertions(+)
-diff --git a/include/qemu/throttle.h b/include/qemu/throttle.h
+diff --git a/hw/block/virtio-blk-common.c b/hw/block/virtio-blk-common.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/qemu/throttle.h
+--- a/hw/block/virtio-blk-common.c
-+++ b/include/qemu/throttle.h
++++ b/hw/block/virtio-blk-common.c
-@@ -XXX,XX +XXX,XX @@ typedef struct LeakyBucket {
+@@ -XXX,XX +XXX,XX @@ static const VirtIOFeature feature_sizes[] = {
-     uint64_t max;             /* leaky bucket max burst in units */
+      .end = endof(struct virtio_blk_config, discard_sector_alignment)},
-     double  level;            /* bucket level in units */
+     {.flags = 1ULL << VIRTIO_BLK_F_WRITE_ZEROES,
-     double  burst_level;      /* bucket level in units (for computing bursts) */
+      .end = endof(struct virtio_blk_config, write_zeroes_may_unmap)},
--    unsigned burst_length;    /* max length of the burst period, in seconds */
++    {.flags = 1ULL << VIRTIO_BLK_F_ZONED,
-+    uint64_t burst_length;    /* max length of the burst period, in seconds */
++     .end = endof(struct virtio_blk_config, zoned)},
- } LeakyBucket;
+     {}
+ };
- /* The following structure is used to configure a ThrottleState
-diff --git a/util/throttle.c b/util/throttle.c
+diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
 index XXXXXXX..XXXXXXX 100644
---- a/util/throttle.c
+--- a/hw/block/virtio-blk.c
-+++ b/util/throttle.c
++++ b/hw/block/virtio-blk.c
-@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
+@@ -XXX,XX +XXX,XX @@
-             return false;
+ #include "qemu/module.h"
-         }
+ #include "qemu/error-report.h"
+ #include "qemu/main-loop.h"
-+        if (bkt->max && bkt->burst_length > THROTTLE_VALUE_MAX / bkt->max) {
++#include "block/block_int.h"
-+            error_setg(errp, "burst length too high for this burst rate");
+ #include "trace.h"
  #include "hw/block/block.h"
  #include "hw/qdev-properties.h"
@@ -XXX,XX +XXX,XX @@ err:
      return err_status;
  }
 +typedef struct ZoneCmdData {
 +    VirtIOBlockReq *req;
 +    struct iovec *in_iov;
 +    unsigned in_num;
 +    union {
 +        struct {
 +            unsigned int nr_zones;
 +            BlockZoneDescriptor *zones;
 +        } zone_report_data;
 +        struct {
 +            int64_t offset;
 +        } zone_append_data;
 +    };
 +} ZoneCmdData;
 +
 +/*
 + * check zoned_request: error checking before issuing requests. If all checks
 + * passed, return true.
 + * append: true if only zone append requests issued.
 + */
 +static bool check_zoned_request(VirtIOBlock *s, int64_t offset, int64_t len,
 +                             bool append, uint8_t *status) {
 +    BlockDriverState *bs = blk_bs(s->blk);
 +    int index;
 +
 +    if (!virtio_has_feature(s->host_features, VIRTIO_BLK_F_ZONED)) {
 +        *status = VIRTIO_BLK_S_UNSUPP;
 +        return false;
 +    }
 +
 +    if (offset < 0 || len < 0 || len > (bs->total_sectors << BDRV_SECTOR_BITS)
 +        || offset > (bs->total_sectors << BDRV_SECTOR_BITS) - len) {
 +        *status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +        return false;
 +    }
 +
 +    if (append) {
 +        if (bs->bl.write_granularity) {
 +            if ((offset % bs->bl.write_granularity) != 0) {
 +                *status = VIRTIO_BLK_S_ZONE_UNALIGNED_WP;
 +                return false;
 +            }
 +        }
 +
 +        index = offset / bs->bl.zone_size;
 +        if (BDRV_ZT_IS_CONV(bs->wps->wp[index])) {
 +            *status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +            return false;
 +        }
 +
-         if (bkt->max && !bkt->avg) {
++        if (len / 512 > bs->bl.max_append_sectors) {
-             error_setg(errp, "bps_max/iops_max require corresponding"
++            if (bs->bl.max_append_sectors == 0) {
-                        " bps/iops values");
++                *status = VIRTIO_BLK_S_UNSUPP;
 +            } else {
 +                *status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +            }
 +            return false;
 +        }
 +    }
 +    return true;
 +}
 +
 +static void virtio_blk_zone_report_complete(void *opaque, int ret)
 +{
 +    ZoneCmdData *data = opaque;
 +    VirtIOBlockReq *req = data->req;
 +    VirtIOBlock *s = req->dev;
 +    VirtIODevice *vdev = VIRTIO_DEVICE(req->dev);
 +    struct iovec *in_iov = data->in_iov;
 +    unsigned in_num = data->in_num;
 +    int64_t zrp_size, n, j = 0;
 +    int64_t nz = data->zone_report_data.nr_zones;
 +    int8_t err_status = VIRTIO_BLK_S_OK;
 +
 +    if (ret) {
 +        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +        goto out;
 +    }
 +
 +    struct virtio_blk_zone_report zrp_hdr = (struct virtio_blk_zone_report) {
 +        .nr_zones = cpu_to_le64(nz),
 +    };
 +    zrp_size = sizeof(struct virtio_blk_zone_report)
 +               + sizeof(struct virtio_blk_zone_descriptor) * nz;
 +    n = iov_from_buf(in_iov, in_num, 0, &zrp_hdr, sizeof(zrp_hdr));
 +    if (n != sizeof(zrp_hdr)) {
 +        virtio_error(vdev, "Driver provided input buffer that is too small!");
 +        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +        goto out;
 +    }
 +
 +    for (size_t i = sizeof(zrp_hdr); i < zrp_size;
 +        i += sizeof(struct virtio_blk_zone_descriptor), ++j) {
 +        struct virtio_blk_zone_descriptor desc =
 +            (struct virtio_blk_zone_descriptor) {
 +                .z_start = cpu_to_le64(data->zone_report_data.zones[j].start
 +                    >> BDRV_SECTOR_BITS),
 +                .z_cap = cpu_to_le64(data->zone_report_data.zones[j].cap
 +                    >> BDRV_SECTOR_BITS),
 +                .z_wp = cpu_to_le64(data->zone_report_data.zones[j].wp
 +                    >> BDRV_SECTOR_BITS),
 +        };
 +
 +        switch (data->zone_report_data.zones[j].type) {
 +        case BLK_ZT_CONV:
 +            desc.z_type = VIRTIO_BLK_ZT_CONV;
 +            break;
 +        case BLK_ZT_SWR:
 +            desc.z_type = VIRTIO_BLK_ZT_SWR;
 +            break;
 +        case BLK_ZT_SWP:
 +            desc.z_type = VIRTIO_BLK_ZT_SWP;
 +            break;
 +        default:
 +            g_assert_not_reached();
 +        }
 +
 +        switch (data->zone_report_data.zones[j].state) {
 +        case BLK_ZS_RDONLY:
 +            desc.z_state = VIRTIO_BLK_ZS_RDONLY;
 +            break;
 +        case BLK_ZS_OFFLINE:
 +            desc.z_state = VIRTIO_BLK_ZS_OFFLINE;
 +            break;
 +        case BLK_ZS_EMPTY:
 +            desc.z_state = VIRTIO_BLK_ZS_EMPTY;
 +            break;
 +        case BLK_ZS_CLOSED:
 +            desc.z_state = VIRTIO_BLK_ZS_CLOSED;
 +            break;
 +        case BLK_ZS_FULL:
 +            desc.z_state = VIRTIO_BLK_ZS_FULL;
 +            break;
 +        case BLK_ZS_EOPEN:
 +            desc.z_state = VIRTIO_BLK_ZS_EOPEN;
 +            break;
 +        case BLK_ZS_IOPEN:
 +            desc.z_state = VIRTIO_BLK_ZS_IOPEN;
 +            break;
 +        case BLK_ZS_NOT_WP:
 +            desc.z_state = VIRTIO_BLK_ZS_NOT_WP;
 +            break;
 +        default:
 +            g_assert_not_reached();
 +        }
 +
 +        /* TODO: it takes O(n^2) time complexity. Optimizations required. */
 +        n = iov_from_buf(in_iov, in_num, i, &desc, sizeof(desc));
 +        if (n != sizeof(desc)) {
 +            virtio_error(vdev, "Driver provided input buffer "
 +                               "for descriptors that is too small!");
 +            err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +        }
 +    }
 +
 +out:
 +    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
 +    virtio_blk_req_complete(req, err_status);
 +    virtio_blk_free_request(req);
 +    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
 +    g_free(data->zone_report_data.zones);
 +    g_free(data);
 +}
 +
 +static void virtio_blk_handle_zone_report(VirtIOBlockReq *req,
 +                                         struct iovec *in_iov,
 +                                         unsigned in_num)
 +{
 +    VirtIOBlock *s = req->dev;
 +    VirtIODevice *vdev = VIRTIO_DEVICE(s);
 +    unsigned int nr_zones;
 +    ZoneCmdData *data;
 +    int64_t zone_size, offset;
 +    uint8_t err_status;
 +
 +    if (req->in_len < sizeof(struct virtio_blk_inhdr) +
 +            sizeof(struct virtio_blk_zone_report) +
 +            sizeof(struct virtio_blk_zone_descriptor)) {
 +        virtio_error(vdev, "in buffer too small for zone report");
 +        return;
 +    }
 +
 +    /* start byte offset of the zone report */
 +    offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
 +    if (!check_zoned_request(s, offset, 0, false, &err_status)) {
 +        goto out;
 +    }
 +    nr_zones = (req->in_len - sizeof(struct virtio_blk_inhdr) -
 +                sizeof(struct virtio_blk_zone_report)) /
 +               sizeof(struct virtio_blk_zone_descriptor);
 +
 +    zone_size = sizeof(BlockZoneDescriptor) * nr_zones;
 +    data = g_malloc(sizeof(ZoneCmdData));
 +    data->req = req;
 +    data->in_iov = in_iov;
 +    data->in_num = in_num;
 +    data->zone_report_data.nr_zones = nr_zones;
 +    data->zone_report_data.zones = g_malloc(zone_size),
 +
 +    blk_aio_zone_report(s->blk, offset, &data->zone_report_data.nr_zones,
 +                        data->zone_report_data.zones,
 +                        virtio_blk_zone_report_complete, data);
 +    return;
 +out:
 +    virtio_blk_req_complete(req, err_status);
 +    virtio_blk_free_request(req);
 +}
 +
 +static void virtio_blk_zone_mgmt_complete(void *opaque, int ret)
 +{
 +    VirtIOBlockReq *req = opaque;
 +    VirtIOBlock *s = req->dev;
 +    int8_t err_status = VIRTIO_BLK_S_OK;
 +
 +    if (ret) {
 +        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +    }
 +
 +    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
 +    virtio_blk_req_complete(req, err_status);
 +    virtio_blk_free_request(req);
 +    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
 +}
 +
 +static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, BlockZoneOp op)
 +{
 +    VirtIOBlock *s = req->dev;
 +    VirtIODevice *vdev = VIRTIO_DEVICE(s);
 +    BlockDriverState *bs = blk_bs(s->blk);
 +    int64_t offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
 +    uint64_t len;
 +    uint64_t capacity = bs->total_sectors << BDRV_SECTOR_BITS;
 +    uint8_t err_status = VIRTIO_BLK_S_OK;
 +
 +    uint32_t type = virtio_ldl_p(vdev, &req->out.type);
 +    if (type == VIRTIO_BLK_T_ZONE_RESET_ALL) {
 +        /* Entire drive capacity */
 +        offset = 0;
 +        len = capacity;
 +    } else {
 +        if (bs->bl.zone_size > capacity - offset) {
 +            /* The zoned device allows the last smaller zone. */
 +            len = capacity - bs->bl.zone_size * (bs->bl.nr_zones - 1);
 +        } else {
 +            len = bs->bl.zone_size;
 +        }
 +    }
 +
 +    if (!check_zoned_request(s, offset, len, false, &err_status)) {
 +        goto out;
 +    }
 +
 +    blk_aio_zone_mgmt(s->blk, op, offset, len,
 +                      virtio_blk_zone_mgmt_complete, req);
 +
 +    return 0;
 +out:
 +    virtio_blk_req_complete(req, err_status);
 +    virtio_blk_free_request(req);
 +    return err_status;
 +}
 +
 +static void virtio_blk_zone_append_complete(void *opaque, int ret)
 +{
 +    ZoneCmdData *data = opaque;
 +    VirtIOBlockReq *req = data->req;
 +    VirtIOBlock *s = req->dev;
 +    VirtIODevice *vdev = VIRTIO_DEVICE(req->dev);
 +    int64_t append_sector, n;
 +    uint8_t err_status = VIRTIO_BLK_S_OK;
 +
 +    if (ret) {
 +        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +        goto out;
 +    }
 +
 +    virtio_stq_p(vdev, &append_sector,
 +                 data->zone_append_data.offset >> BDRV_SECTOR_BITS);
 +    n = iov_from_buf(data->in_iov, data->in_num, 0, &append_sector,
 +                     sizeof(append_sector));
 +    if (n != sizeof(append_sector)) {
 +        virtio_error(vdev, "Driver provided input buffer less than size of "
 +                           "append_sector");
 +        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
 +        goto out;
 +    }
 +
 +out:
 +    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
 +    virtio_blk_req_complete(req, err_status);
 +    virtio_blk_free_request(req);
 +    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
 +    g_free(data);
 +}
 +
 +static int virtio_blk_handle_zone_append(VirtIOBlockReq *req,
 +                                         struct iovec *out_iov,
 +                                         struct iovec *in_iov,
 +                                         uint64_t out_num,
 +                                         unsigned in_num) {
 +    VirtIOBlock *s = req->dev;
 +    VirtIODevice *vdev = VIRTIO_DEVICE(s);
 +    uint8_t err_status = VIRTIO_BLK_S_OK;
 +
 +    int64_t offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
 +    int64_t len = iov_size(out_iov, out_num);
 +
 +    if (!check_zoned_request(s, offset, len, true, &err_status)) {
 +        goto out;
 +    }
 +
 +    ZoneCmdData *data = g_malloc(sizeof(ZoneCmdData));
 +    data->req = req;
 +    data->in_iov = in_iov;
 +    data->in_num = in_num;
 +    data->zone_append_data.offset = offset;
 +    qemu_iovec_init_external(&req->qiov, out_iov, out_num);
 +    blk_aio_zone_append(s->blk, &data->zone_append_data.offset, &req->qiov, 0,
 +                        virtio_blk_zone_append_complete, data);
 +    return 0;
 +
 +out:
 +    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
 +    virtio_blk_req_complete(req, err_status);
 +    virtio_blk_free_request(req);
 +    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
 +    return err_status;
 +}
 +
  static int virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
  {
      uint32_t type;
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
      case VIRTIO_BLK_T_FLUSH:
          virtio_blk_handle_flush(req, mrb);
          break;
 +    case VIRTIO_BLK_T_ZONE_REPORT:
 +        virtio_blk_handle_zone_report(req, in_iov, in_num);
 +        break;
 +    case VIRTIO_BLK_T_ZONE_OPEN:
 +        virtio_blk_handle_zone_mgmt(req, BLK_ZO_OPEN);
 +        break;
 +    case VIRTIO_BLK_T_ZONE_CLOSE:
 +        virtio_blk_handle_zone_mgmt(req, BLK_ZO_CLOSE);
 +        break;
 +    case VIRTIO_BLK_T_ZONE_FINISH:
 +        virtio_blk_handle_zone_mgmt(req, BLK_ZO_FINISH);
 +        break;
 +    case VIRTIO_BLK_T_ZONE_RESET:
 +        virtio_blk_handle_zone_mgmt(req, BLK_ZO_RESET);
 +        break;
 +    case VIRTIO_BLK_T_ZONE_RESET_ALL:
 +        virtio_blk_handle_zone_mgmt(req, BLK_ZO_RESET);
 +        break;
      case VIRTIO_BLK_T_SCSI_CMD:
          virtio_blk_handle_scsi(req);
          break;
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
          virtio_blk_free_request(req);
          break;
      }
 +    case VIRTIO_BLK_T_ZONE_APPEND & ~VIRTIO_BLK_T_OUT:
 +        /*
 +         * Passing out_iov/out_num and in_iov/in_num is not safe
 +         * to access req->elem.out_sg directly because it may be
 +         * modified by virtio_blk_handle_request().
 +         */
 +        virtio_blk_handle_zone_append(req, out_iov, in_iov, out_num, in_num);
 +        break;
      /*
       * VIRTIO_BLK_T_DISCARD and VIRTIO_BLK_T_WRITE_ZEROES are defined with
       * VIRTIO_BLK_T_OUT flag set. We masked this flag in the switch statement,
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
  {
      VirtIOBlock *s = VIRTIO_BLK(vdev);
      BlockConf *conf = &s->conf.conf;
 +    BlockDriverState *bs = blk_bs(s->blk);
      struct virtio_blk_config blkcfg;
      uint64_t capacity;
      int64_t length;
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
          blkcfg.write_zeroes_may_unmap = 1;
          virtio_stl_p(vdev, &blkcfg.max_write_zeroes_seg, 1);
      }
 +    if (bs->bl.zoned != BLK_Z_NONE) {
 +        switch (bs->bl.zoned) {
 +        case BLK_Z_HM:
 +            blkcfg.zoned.model = VIRTIO_BLK_Z_HM;
 +            break;
 +        case BLK_Z_HA:
 +            blkcfg.zoned.model = VIRTIO_BLK_Z_HA;
 +            break;
 +        default:
 +            g_assert_not_reached();
 +        }
 +
 +        virtio_stl_p(vdev, &blkcfg.zoned.zone_sectors,
 +                     bs->bl.zone_size / 512);
 +        virtio_stl_p(vdev, &blkcfg.zoned.max_active_zones,
 +                     bs->bl.max_active_zones);
 +        virtio_stl_p(vdev, &blkcfg.zoned.max_open_zones,
 +                     bs->bl.max_open_zones);
 +        virtio_stl_p(vdev, &blkcfg.zoned.write_granularity, blk_size);
 +        virtio_stl_p(vdev, &blkcfg.zoned.max_append_sectors,
 +                     bs->bl.max_append_sectors);
 +    } else {
 +        blkcfg.zoned.model = VIRTIO_BLK_Z_NONE;
 +    }
      memcpy(config, &blkcfg, s->config_size);
  }
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp)
      VirtIODevice *vdev = VIRTIO_DEVICE(dev);
      VirtIOBlock *s = VIRTIO_BLK(dev);
      VirtIOBlkConf *conf = &s->conf;
 +    BlockDriverState *bs = blk_bs(conf->conf.blk);
      Error *err = NULL;
      unsigned i;
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp)
          return;
      }
 +    if (bs->bl.zoned != BLK_Z_NONE) {
 +        virtio_add_feature(&s->host_features, VIRTIO_BLK_F_ZONED);
 +        if (bs->bl.zoned == BLK_Z_HM) {
 +            virtio_clear_feature(&s->host_features, VIRTIO_BLK_F_DISCARD);
 +        }
 +    }
 +
      if (virtio_has_feature(s->host_features, VIRTIO_BLK_F_DISCARD) &&
          (!conf->max_discard_sectors ||
           conf->max_discard_sectors > BDRV_REQUEST_MAX_SECTORS)) {
 diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
 index XXXXXXX..XXXXXXX 100644
 --- a/hw/virtio/virtio-qmp.c
 +++ b/hw/virtio/virtio-qmp.c
@@ -XXX,XX +XXX,XX @@ static const qmp_virtio_feature_map_t virtio_blk_feature_map[] = {
              "VIRTIO_BLK_F_DISCARD: Discard command supported"),
      FEATURE_ENTRY(VIRTIO_BLK_F_WRITE_ZEROES, \
              "VIRTIO_BLK_F_WRITE_ZEROES: Write zeroes command supported"),
 +    FEATURE_ENTRY(VIRTIO_BLK_F_ZONED, \
 +            "VIRTIO_BLK_F_ZONED: Zoned block devices"),
  #ifndef VIRTIO_BLK_NO_LEGACY
      FEATURE_ENTRY(VIRTIO_BLK_F_BARRIER, \
              "VIRTIO_BLK_F_BARRIER: Request barriers supported"),
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 05/15] throttle: Remove throttle_fix_bucket() / throttle_unfix_bucket()
+[PULL 17/20] block: add accounting for zone append operation
-From: Alberto Garcia <berto@igalia.com>
+From: Sam Li <faithilikerun@gmail.com>
-The throttling code can change internally the value of bkt->max if it
+Taking account of the new zone append write operation for zoned devices,
-hasn't been set by the user. The problem with this is that if we want
+BLOCK_ACCT_ZONE_APPEND enum is introduced as other I/O request type (read,
-to retrieve the original value we have to undo this change first. This
+write, flush).
 is ugly and unnecessary: this patch removes the throttle_fix_bucket()
 and throttle_unfix_bucket() functions completely and moves the logic
 to throttle_compute_wait().
-Signed-off-by: Alberto Garcia <berto@igalia.com>
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-Reviewed-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
+Message-id: 20230407082528.18841-4-faithilikerun@gmail.com
 Message-id: 5b0b9e1ac6eb208d709eddc7b09e7669a523bff3.1503580370.git.berto@igalia.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- util/throttle.c | 62 +++++++++++++++++++++------------------------------------
+ qapi/block-core.json       | 68 ++++++++++++++++++++++++++++++++------
-file changed, 23 insertions(+), 39 deletions(-)
+ qapi/block.json            |  4 +++
  include/block/accounting.h |  1 +
  block/qapi-sysemu.c        | 11 ++++++
  block/qapi.c               | 18 ++++++++++
  hw/block/virtio-blk.c      |  4 +++
 files changed, 95 insertions(+), 11 deletions(-)
-diff --git a/util/throttle.c b/util/throttle.c
+diff --git a/qapi/block-core.json b/qapi/block-core.json
 index XXXXXXX..XXXXXXX 100644
---- a/util/throttle.c
+--- a/qapi/block-core.json
-+++ b/util/throttle.c
++++ b/qapi/block-core.json
-@@ -XXX,XX +XXX,XX @@ static int64_t throttle_do_compute_wait(double limit, double extra)
+@@ -XXX,XX +XXX,XX @@
- int64_t throttle_compute_wait(LeakyBucket *bkt)
+ # @min_wr_latency_ns: Minimum latency of write operations in the
  #                     defined interval, in nanoseconds.
  #
 +# @min_zone_append_latency_ns: Minimum latency of zone append operations
 +#                              in the defined interval, in nanoseconds
 +#                              (since 8.1)
 +#
  # @min_flush_latency_ns: Minimum latency of flush operations in the
  #                        defined interval, in nanoseconds.
  #
@@ -XXX,XX +XXX,XX @@
  # @max_wr_latency_ns: Maximum latency of write operations in the
  #                     defined interval, in nanoseconds.
  #
 +# @max_zone_append_latency_ns: Maximum latency of zone append operations
 +#                              in the defined interval, in nanoseconds
 +#                              (since 8.1)
 +#
  # @max_flush_latency_ns: Maximum latency of flush operations in the
  #                        defined interval, in nanoseconds.
  #
@@ -XXX,XX +XXX,XX @@
  # @avg_wr_latency_ns: Average latency of write operations in the
  #                     defined interval, in nanoseconds.
  #
 +# @avg_zone_append_latency_ns: Average latency of zone append operations
 +#                              in the defined interval, in nanoseconds
 +#                              (since 8.1)
 +#
  # @avg_flush_latency_ns: Average latency of flush operations in the
  #                        defined interval, in nanoseconds.
  #
@@ -XXX,XX +XXX,XX @@
  # @avg_wr_queue_depth: Average number of pending write operations
  #                      in the defined interval.
  #
 +# @avg_zone_append_queue_depth: Average number of pending zone append
 +#                               operations in the defined interval
 +#                               (since 8.1).
 +#
  # Since: 2.5
  ##
  { 'struct': 'BlockDeviceTimedStats',
    'data': { 'interval_length': 'int', 'min_rd_latency_ns': 'int',
              'max_rd_latency_ns': 'int', 'avg_rd_latency_ns': 'int',
              'min_wr_latency_ns': 'int', 'max_wr_latency_ns': 'int',
 -            'avg_wr_latency_ns': 'int', 'min_flush_latency_ns': 'int',
 -            'max_flush_latency_ns': 'int', 'avg_flush_latency_ns': 'int',
 -            'avg_rd_queue_depth': 'number', 'avg_wr_queue_depth': 'number' } }
 +            'avg_wr_latency_ns': 'int', 'min_zone_append_latency_ns': 'int',
 +            'max_zone_append_latency_ns': 'int',
 +            'avg_zone_append_latency_ns': 'int',
 +            'min_flush_latency_ns': 'int', 'max_flush_latency_ns': 'int',
 +            'avg_flush_latency_ns': 'int', 'avg_rd_queue_depth': 'number',
 +            'avg_wr_queue_depth': 'number',
 +            'avg_zone_append_queue_depth': 'number'  } }
  ##
  # @BlockDeviceStats:
@@ -XXX,XX +XXX,XX @@
  #
  # @wr_bytes: The number of bytes written by the device.
  #
 +# @zone_append_bytes: The number of bytes appended by the zoned devices
 +#                     (since 8.1)
 +#
  # @unmap_bytes: The number of bytes unmapped by the device (Since 4.2)
  #
  # @rd_operations: The number of read operations performed by the device.
  #
  # @wr_operations: The number of write operations performed by the device.
  #
 +# @zone_append_operations: The number of zone append operations performed
 +#                          by the zoned devices (since 8.1)
 +#
  # @flush_operations: The number of cache flush operations performed by the
  #                    device (since 0.15)
  #
@@ -XXX,XX +XXX,XX @@
  #
  # @wr_total_time_ns: Total time spent on writes in nanoseconds (since 0.15).
  #
 +# @zone_append_total_time_ns: Total time spent on zone append writes
 +#                             in nanoseconds (since 8.1)
 +#
  # @flush_total_time_ns: Total time spent on cache flushes in nanoseconds
  #                       (since 0.15).
  #
@@ -XXX,XX +XXX,XX @@
  # @wr_merged: Number of write requests that have been merged into another
  #             request (Since 2.3).
  #
 +# @zone_append_merged: Number of zone append requests that have been merged
 +#                      into another request (since 8.1)
 +#
  # @unmap_merged: Number of unmap requests that have been merged into another
  #                request (Since 4.2)
  #
@@ -XXX,XX +XXX,XX @@
  # @failed_wr_operations: The number of failed write operations
  #                        performed by the device (Since 2.5)
  #
 +# @failed_zone_append_operations: The number of failed zone append write
 +#                                 operations performed by the zoned devices
 +#                                 (since 8.1)
 +#
  # @failed_flush_operations: The number of failed flush operations
  #                           performed by the device (Since 2.5)
  #
@@ -XXX,XX +XXX,XX @@
  # @invalid_wr_operations: The number of invalid write operations
  #                         performed by the device (Since 2.5)
  #
 +# @invalid_zone_append_operations: The number of invalid zone append operations
 +#                                  performed by the zoned device (since 8.1)
 +#
  # @invalid_flush_operations: The number of invalid flush operations
  #                            performed by the device (Since 2.5)
  #
@@ -XXX,XX +XXX,XX @@
  #
  # @wr_latency_histogram: @BlockLatencyHistogramInfo. (Since 4.0)
  #
 +# @zone_append_latency_histogram: @BlockLatencyHistogramInfo. (since 8.1)
 +#
  # @flush_latency_histogram: @BlockLatencyHistogramInfo. (Since 4.0)
  #
  # Since: 0.14
  ##
  { 'struct': 'BlockDeviceStats',
 -  'data': {'rd_bytes': 'int', 'wr_bytes': 'int', 'unmap_bytes' : 'int',
 -           'rd_operations': 'int', 'wr_operations': 'int',
 +  'data': {'rd_bytes': 'int', 'wr_bytes': 'int', 'zone_append_bytes': 'int',
 +           'unmap_bytes' : 'int', 'rd_operations': 'int',
 +           'wr_operations': 'int', 'zone_append_operations': 'int',
             'flush_operations': 'int', 'unmap_operations': 'int',
             'rd_total_time_ns': 'int', 'wr_total_time_ns': 'int',
 -           'flush_total_time_ns': 'int', 'unmap_total_time_ns': 'int',
 -           'wr_highest_offset': 'int',
 -           'rd_merged': 'int', 'wr_merged': 'int', 'unmap_merged': 'int',
 -           '*idle_time_ns': 'int',
 +           'zone_append_total_time_ns': 'int', 'flush_total_time_ns': 'int',
 +           'unmap_total_time_ns': 'int', 'wr_highest_offset': 'int',
 +           'rd_merged': 'int', 'wr_merged': 'int', 'zone_append_merged': 'int',
 +           'unmap_merged': 'int', '*idle_time_ns': 'int',
             'failed_rd_operations': 'int', 'failed_wr_operations': 'int',
 -           'failed_flush_operations': 'int', 'failed_unmap_operations': 'int',
 -           'invalid_rd_operations': 'int', 'invalid_wr_operations': 'int',
 +           'failed_zone_append_operations': 'int',
 +           'failed_flush_operations': 'int',
 +           'failed_unmap_operations': 'int', 'invalid_rd_operations': 'int',
 +           'invalid_wr_operations': 'int',
 +           'invalid_zone_append_operations': 'int',
             'invalid_flush_operations': 'int', 'invalid_unmap_operations': 'int',
             'account_invalid': 'bool', 'account_failed': 'bool',
             'timed_stats': ['BlockDeviceTimedStats'],
             '*rd_latency_histogram': 'BlockLatencyHistogramInfo',
             '*wr_latency_histogram': 'BlockLatencyHistogramInfo',
 +           '*zone_append_latency_histogram': 'BlockLatencyHistogramInfo',
             '*flush_latency_histogram': 'BlockLatencyHistogramInfo' } }
  ##
 diff --git a/qapi/block.json b/qapi/block.json
 index XXXXXXX..XXXXXXX 100644
 --- a/qapi/block.json
 +++ b/qapi/block.json
@@ -XXX,XX +XXX,XX @@
  # @boundaries-write: list of interval boundary values for write latency
  #                    histogram.
  #
 +# @boundaries-zap: list of interval boundary values for zone append write
 +#                  latency histogram.
 +#
  # @boundaries-flush: list of interval boundary values for flush latency
  #                    histogram.
  #
@@ -XXX,XX +XXX,XX @@
             '*boundaries': ['uint64'],
             '*boundaries-read': ['uint64'],
             '*boundaries-write': ['uint64'],
 +           '*boundaries-zap': ['uint64'],
             '*boundaries-flush': ['uint64'] },
    'allow-preconfig': true }
 diff --git a/include/block/accounting.h b/include/block/accounting.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/accounting.h
 +++ b/include/block/accounting.h
@@ -XXX,XX +XXX,XX @@ enum BlockAcctType {
      BLOCK_ACCT_READ,
      BLOCK_ACCT_WRITE,
      BLOCK_ACCT_FLUSH,
 +    BLOCK_ACCT_ZONE_APPEND,
      BLOCK_ACCT_UNMAP,
      BLOCK_MAX_IOTYPE,
  };
 diff --git a/block/qapi-sysemu.c b/block/qapi-sysemu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qapi-sysemu.c
 +++ b/block/qapi-sysemu.c
@@ -XXX,XX +XXX,XX @@ void qmp_block_latency_histogram_set(
      bool has_boundaries, uint64List *boundaries,
      bool has_boundaries_read, uint64List *boundaries_read,
      bool has_boundaries_write, uint64List *boundaries_write,
 +    bool has_boundaries_append, uint64List *boundaries_append,
      bool has_boundaries_flush, uint64List *boundaries_flush,
      Error **errp)
  {
-     double extra; /* the number of extra units blocking the io */
+@@ -XXX,XX +XXX,XX @@ void qmp_block_latency_histogram_set(
-+    double bucket_size;   /* I/O before throttling to bkt->avg */
+         }
 +    double burst_bucket_size; /* Before throttling to bkt->max */
      if (!bkt->avg) {
          return 0;
      }
--    /* If the bucket is full then we have to wait */
++    if (has_boundaries || has_boundaries_append) {
--    extra = bkt->level - bkt->max * bkt->burst_length;
++        ret = block_latency_histogram_set(
-+    if (!bkt->max) {
++                stats, BLOCK_ACCT_ZONE_APPEND,
-+        /* If bkt->max is 0 we still want to allow short bursts of I/O
++                has_boundaries_append ? boundaries_append : boundaries);
-+         * from the guest, otherwise every other request will be throttled
++        if (ret) {
-+         * and performance will suffer considerably. */
++            error_setg(errp, "Device '%s' set append write boundaries fail", id);
-+        bucket_size = bkt->avg / 10;
++            return;
-+        burst_bucket_size = 0;
++        }
 +    } else {
 +        /* If we have a burst limit then we have to wait until all I/O
 +         * at burst rate has finished before throttling to bkt->avg */
 +        bucket_size = bkt->max * bkt->burst_length;
 +        burst_bucket_size = bkt->max / 10;
 +    }
 +
-+    /* If the main bucket is full then we have to wait */
+     if (has_boundaries || has_boundaries_flush) {
-+    extra = bkt->level - bucket_size;
+         ret = block_latency_histogram_set(
-     if (extra > 0) {
+             stats, BLOCK_ACCT_FLUSH,
-         return throttle_do_compute_wait(bkt->avg, extra);
+diff --git a/block/qapi.c b/block/qapi.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qapi.c
 +++ b/block/qapi.c
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
      ds->rd_bytes = stats->nr_bytes[BLOCK_ACCT_READ];
      ds->wr_bytes = stats->nr_bytes[BLOCK_ACCT_WRITE];
 +    ds->zone_append_bytes = stats->nr_bytes[BLOCK_ACCT_ZONE_APPEND];
      ds->unmap_bytes = stats->nr_bytes[BLOCK_ACCT_UNMAP];
      ds->rd_operations = stats->nr_ops[BLOCK_ACCT_READ];
      ds->wr_operations = stats->nr_ops[BLOCK_ACCT_WRITE];
 +    ds->zone_append_operations = stats->nr_ops[BLOCK_ACCT_ZONE_APPEND];
      ds->unmap_operations = stats->nr_ops[BLOCK_ACCT_UNMAP];
      ds->failed_rd_operations = stats->failed_ops[BLOCK_ACCT_READ];
      ds->failed_wr_operations = stats->failed_ops[BLOCK_ACCT_WRITE];
 +    ds->failed_zone_append_operations =
 +        stats->failed_ops[BLOCK_ACCT_ZONE_APPEND];
      ds->failed_flush_operations = stats->failed_ops[BLOCK_ACCT_FLUSH];
      ds->failed_unmap_operations = stats->failed_ops[BLOCK_ACCT_UNMAP];
      ds->invalid_rd_operations = stats->invalid_ops[BLOCK_ACCT_READ];
      ds->invalid_wr_operations = stats->invalid_ops[BLOCK_ACCT_WRITE];
 +    ds->invalid_zone_append_operations =
 +        stats->invalid_ops[BLOCK_ACCT_ZONE_APPEND];
      ds->invalid_flush_operations =
          stats->invalid_ops[BLOCK_ACCT_FLUSH];
      ds->invalid_unmap_operations = stats->invalid_ops[BLOCK_ACCT_UNMAP];
      ds->rd_merged = stats->merged[BLOCK_ACCT_READ];
      ds->wr_merged = stats->merged[BLOCK_ACCT_WRITE];
 +    ds->zone_append_merged = stats->merged[BLOCK_ACCT_ZONE_APPEND];
      ds->unmap_merged = stats->merged[BLOCK_ACCT_UNMAP];
      ds->flush_operations = stats->nr_ops[BLOCK_ACCT_FLUSH];
      ds->wr_total_time_ns = stats->total_time_ns[BLOCK_ACCT_WRITE];
 +    ds->zone_append_total_time_ns =
 +        stats->total_time_ns[BLOCK_ACCT_ZONE_APPEND];
      ds->rd_total_time_ns = stats->total_time_ns[BLOCK_ACCT_READ];
      ds->flush_total_time_ns = stats->total_time_ns[BLOCK_ACCT_FLUSH];
      ds->unmap_total_time_ns = stats->total_time_ns[BLOCK_ACCT_UNMAP];
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
          TimedAverage *rd = &ts->latency[BLOCK_ACCT_READ];
          TimedAverage *wr = &ts->latency[BLOCK_ACCT_WRITE];
 +        TimedAverage *zap = &ts->latency[BLOCK_ACCT_ZONE_APPEND];
          TimedAverage *fl = &ts->latency[BLOCK_ACCT_FLUSH];
          dev_stats->interval_length = ts->interval_length;
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
          dev_stats->max_wr_latency_ns = timed_average_max(wr);
          dev_stats->avg_wr_latency_ns = timed_average_avg(wr);
 +        dev_stats->min_zone_append_latency_ns = timed_average_min(zap);
 +        dev_stats->max_zone_append_latency_ns = timed_average_max(zap);
 +        dev_stats->avg_zone_append_latency_ns = timed_average_avg(zap);
 +
          dev_stats->min_flush_latency_ns = timed_average_min(fl);
          dev_stats->max_flush_latency_ns = timed_average_max(fl);
          dev_stats->avg_flush_latency_ns = timed_average_avg(fl);
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
              block_acct_queue_depth(ts, BLOCK_ACCT_READ);
          dev_stats->avg_wr_queue_depth =
              block_acct_queue_depth(ts, BLOCK_ACCT_WRITE);
 +        dev_stats->avg_zone_append_queue_depth =
 +            block_acct_queue_depth(ts, BLOCK_ACCT_ZONE_APPEND);
          QAPI_LIST_PREPEND(ds->timed_stats, dev_stats);
      }
+@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
--    /* If the bucket is not full yet we have to make sure that we
+         = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_READ]);
--     * fulfill the goal of bkt->max units per second. */
+     ds->wr_latency_histogram
-+    /* If the main bucket is not full yet we still have to check the
+         = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_WRITE]);
-+     * burst bucket in order to enforce the burst limit */
++    ds->zone_append_latency_histogram
-     if (bkt->burst_length > 1) {
++        = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_ZONE_APPEND]);
--        /* We use 1/10 of the max value to smooth the throttling.
+     ds->flush_latency_histogram
--         * See throttle_fix_bucket() for more details. */
+         = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_FLUSH]);
 -        extra = bkt->burst_level - bkt->max / 10;
 +        extra = bkt->burst_level - burst_bucket_size;
          if (extra > 0) {
              return throttle_do_compute_wait(bkt->max, extra);
          }
@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
      return true;
  }
+diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
--/* fix bucket parameters */
+index XXXXXXX..XXXXXXX 100644
--static void throttle_fix_bucket(LeakyBucket *bkt)
+--- a/hw/block/virtio-blk.c
--{
++++ b/hw/block/virtio-blk.c
--    double min;
+@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_append(VirtIOBlockReq *req,
--
+     data->in_num = in_num;
--    /* zero bucket level */
+     data->zone_append_data.offset = offset;
--    bkt->level = bkt->burst_level = 0;
+     qemu_iovec_init_external(&req->qiov, out_iov, out_num);
--
++
--    /* If bkt->max is 0 we still want to allow short bursts of I/O
++    block_acct_start(blk_get_stats(s->blk), &req->acct, len,
--     * from the guest, otherwise every other request will be throttled
++                     BLOCK_ACCT_ZONE_APPEND);
--     * and performance will suffer considerably. */
++
--    min = bkt->avg / 10;
+     blk_aio_zone_append(s->blk, &data->zone_append_data.offset, &req->qiov, 0,
--    if (bkt->avg && !bkt->max) {
+                         virtio_blk_zone_append_complete, data);
--        bkt->max = min;
+     return 0;
 -    }
 -}
 -
 -/* undo internal bucket parameter changes (see throttle_fix_bucket()) */
 -static void throttle_unfix_bucket(LeakyBucket *bkt)
 -{
 -    if (bkt->max < bkt->avg) {
 -        bkt->max = 0;
 -    }
 -}
 -
  /* Used to configure the throttle
   *
   * @ts: the throttle state we are working on
@@ -XXX,XX +XXX,XX @@ void throttle_config(ThrottleState *ts,
      ts->cfg = *cfg;
 +    /* Zero bucket level */
      for (i = 0; i < BUCKETS_COUNT; i++) {
 -        throttle_fix_bucket(&ts->cfg.buckets[i]);
 +        ts->cfg.buckets[i].level = 0;
 +        ts->cfg.buckets[i].burst_level = 0;
      }
      ts->previous_leak = qemu_clock_get_ns(clock_type);
@@ -XXX,XX +XXX,XX @@ void throttle_config(ThrottleState *ts,
   */
  void throttle_get_config(ThrottleState *ts, ThrottleConfig *cfg)
  {
 -    int i;
 -
      *cfg = ts->cfg;
 -
 -    for (i = 0; i < BUCKETS_COUNT; i++) {
 -        throttle_unfix_bucket(&cfg->buckets[i]);
 -    }
  }
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 04/15] throttle: Make throttle_is_valid() a bit less verbose
+[PULL 18/20] virtio-blk: add some trace events for zoned emulation
-From: Alberto Garcia <berto@igalia.com>
+From: Sam Li <faithilikerun@gmail.com>
-Use a pointer to the bucket instead of repeating cfg->buckets[i] all
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-the time. This makes the code more concise and will help us expand the
+Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
-checks later and save a few line breaks.
+Message-id: 20230407082528.18841-5-faithilikerun@gmail.com
 Signed-off-by: Alberto Garcia <berto@igalia.com>
 Message-id: 763ffc40a26b17d54cf93f5a999e4656049fcf0c.1503580370.git.berto@igalia.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- util/throttle.c | 15 +++++++--------
+ hw/block/virtio-blk.c | 12 ++++++++++++
-file changed, 7 insertions(+), 8 deletions(-)
+ hw/block/trace-events |  7 +++++++
 files changed, 19 insertions(+)
-diff --git a/util/throttle.c b/util/throttle.c
+diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
 index XXXXXXX..XXXXXXX 100644
---- a/util/throttle.c
+--- a/hw/block/virtio-blk.c
-+++ b/util/throttle.c
++++ b/hw/block/virtio-blk.c
-@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
+@@ -XXX,XX +XXX,XX @@ static void virtio_blk_zone_report_complete(void *opaque, int ret)
      int64_t nz = data->zone_report_data.nr_zones;
      int8_t err_status = VIRTIO_BLK_S_OK;
 +    trace_virtio_blk_zone_report_complete(vdev, req, nz, ret);
      if (ret) {
          err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
          goto out;
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_handle_zone_report(VirtIOBlockReq *req,
      nr_zones = (req->in_len - sizeof(struct virtio_blk_inhdr) -
                  sizeof(struct virtio_blk_zone_report)) /
                 sizeof(struct virtio_blk_zone_descriptor);
 +    trace_virtio_blk_handle_zone_report(vdev, req,
 +                                        offset >> BDRV_SECTOR_BITS, nr_zones);
      zone_size = sizeof(BlockZoneDescriptor) * nr_zones;
      data = g_malloc(sizeof(ZoneCmdData));
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_zone_mgmt_complete(void *opaque, int ret)
  {
      VirtIOBlockReq *req = opaque;
      VirtIOBlock *s = req->dev;
 +    VirtIODevice *vdev = VIRTIO_DEVICE(s);
      int8_t err_status = VIRTIO_BLK_S_OK;
 +    trace_virtio_blk_zone_mgmt_complete(vdev, req,ret);
      if (ret) {
          err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, BlockZoneOp op)
          /* Entire drive capacity */
          offset = 0;
          len = capacity;
 +        trace_virtio_blk_handle_zone_reset_all(vdev, req, 0,
 +                                               bs->total_sectors);
      } else {
          if (bs->bl.zone_size > capacity - offset) {
              /* The zoned device allows the last smaller zone. */
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, BlockZoneOp op)
          } else {
              len = bs->bl.zone_size;
          }
 +        trace_virtio_blk_handle_zone_mgmt(vdev, req, op,
 +                                          offset >> BDRV_SECTOR_BITS,
 +                                          len >> BDRV_SECTOR_BITS);
      }
-     for (i = 0; i < BUCKETS_COUNT; i++) {
+     if (!check_zoned_request(s, offset, len, false, &err_status)) {
--        if (cfg->buckets[i].avg < 0 ||
+@@ -XXX,XX +XXX,XX @@ static void virtio_blk_zone_append_complete(void *opaque, int ret)
--            cfg->buckets[i].max < 0 ||
+         err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
--            cfg->buckets[i].avg > THROTTLE_VALUE_MAX ||
+         goto out;
--            cfg->buckets[i].max > THROTTLE_VALUE_MAX) {
+     }
-+        LeakyBucket *bkt = &cfg->buckets[i];
++    trace_virtio_blk_zone_append_complete(vdev, req, append_sector, ret);
-+        if (bkt->avg < 0 || bkt->max < 0 ||
-+            bkt->avg > THROTTLE_VALUE_MAX || bkt->max > THROTTLE_VALUE_MAX) {
+ out:
-             error_setg(errp, "bps/iops/max values must be within [0, %lld]",
+     aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
-                        THROTTLE_VALUE_MAX);
+@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_append(VirtIOBlockReq *req,
-             return false;
+     int64_t offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
-         }
+     int64_t len = iov_size(out_iov, out_num);
--        if (!cfg->buckets[i].burst_length) {
++    trace_virtio_blk_handle_zone_append(vdev, req, offset >> BDRV_SECTOR_BITS);
-+        if (!bkt->burst_length) {
+     if (!check_zoned_request(s, offset, len, true, &err_status)) {
-             error_setg(errp, "the burst length cannot be 0");
+         goto out;
-             return false;
+     }
-         }
+diff --git a/hw/block/trace-events b/hw/block/trace-events
+index XXXXXXX..XXXXXXX 100644
--        if (cfg->buckets[i].burst_length > 1 && !cfg->buckets[i].max) {
+--- a/hw/block/trace-events
-+        if (bkt->burst_length > 1 && !bkt->max) {
++++ b/hw/block/trace-events
-             error_setg(errp, "burst length set without burst rate");
+@@ -XXX,XX +XXX,XX @@ pflash_write_unknown(const char *name, uint8_t cmd) "%s: unknown command 0x%02x"
-             return false;
+ # virtio-blk.c
-         }
+ virtio_blk_req_complete(void *vdev, void *req, int status) "vdev %p req %p status %d"
+ virtio_blk_rw_complete(void *vdev, void *req, int ret) "vdev %p req %p ret %d"
--        if (cfg->buckets[i].max && !cfg->buckets[i].avg) {
++virtio_blk_zone_report_complete(void *vdev, void *req, unsigned int nr_zones, int ret) "vdev %p req %p nr_zones %u ret %d"
-+        if (bkt->max && !bkt->avg) {
++virtio_blk_zone_mgmt_complete(void *vdev, void *req, int ret) "vdev %p req %p ret %d"
-             error_setg(errp, "bps_max/iops_max require corresponding"
++virtio_blk_zone_append_complete(void *vdev, void *req, int64_t sector, int ret) "vdev %p req %p, append sector 0x%" PRIx64 " ret %d"
-                        " bps/iops values");
+ virtio_blk_handle_write(void *vdev, void *req, uint64_t sector, size_t nsectors) "vdev %p req %p sector %"PRIu64" nsectors %zu"
-             return false;
+ virtio_blk_handle_read(void *vdev, void *req, uint64_t sector, size_t nsectors) "vdev %p req %p sector %"PRIu64" nsectors %zu"
-         }
+ virtio_blk_submit_multireq(void *vdev, void *mrb, int start, int num_reqs, uint64_t offset, size_t size, bool is_write) "vdev %p mrb %p start %d num_reqs %d offset %"PRIu64" size %zu is_write %d"
++virtio_blk_handle_zone_report(void *vdev, void *req, int64_t sector, unsigned int nr_zones) "vdev %p req %p sector 0x%" PRIx64 " nr_zones %u"
--        if (cfg->buckets[i].max && cfg->buckets[i].max < cfg->buckets[i].avg) {
++virtio_blk_handle_zone_mgmt(void *vdev, void *req, uint8_t op, int64_t sector, int64_t len) "vdev %p req %p op 0x%x sector 0x%" PRIx64 " len 0x%" PRIx64 ""
-+        if (bkt->max && bkt->max < bkt->avg) {
++virtio_blk_handle_zone_reset_all(void *vdev, void *req, int64_t sector, int64_t len) "vdev %p req %p sector 0x%" PRIx64 " cap 0x%" PRIx64 ""
-             error_setg(errp, "bps_max/iops_max cannot be lower than bps/iops");
++virtio_blk_handle_zone_append(void *vdev, void *req, int64_t sector) "vdev %p req %p, append sector 0x%" PRIx64 ""
-             return false;
-         }
+ # hd-geometry.c
  hd_geometry_lchs_guess(void *blk, int cyls, int heads, int secs) "blk %p LCHS %d %d %d"
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 01/15] nvme: Fix get/set number of queues feature, again
+[PULL 19/20] docs/zoned-storage:add zoned emulation use case
-From: Dan Aloni <dan@kernelim.com>
+From: Sam Li <faithilikerun@gmail.com>
-The number of queues that should be return by the admin command should:
+Add the documentation about the example of using virtio-blk driver
 to pass the zoned block devices through to the guest.
-) Only mention the number of non-admin queues.
+Signed-off-by: Sam Li <faithilikerun@gmail.com>
-) It is zero-based, meaning that '0 == one non-admin queue',
+Message-id: 20230407082528.18841-6-faithilikerun@gmail.com
-     '1 == two non-admin queues', and so forth.
+[Fix Sphinx indentation error by turning command-lines into
+pre-formatted text.
-Because our `num_queues` means the number of queues _plus_ the admin
+--Stefan]
 queue, then the right calculation for the number returned from the admin
 command is `num_queues - 2`, combining the two requirements mentioned.
 The issue was discovered by reducing num_queues from 64 to 8 and running
 a Linux VM with an SMP parameter larger than that (e.g. 22). It tries to
 utilize all queues, and therefore fails with an invalid queue number
 when trying to queue I/Os on the last queue.
 Signed-off-by: Dan Aloni <dan@kernelim.com>
 CC: Alex Friedman <alex@e8storage.com>
 CC: Keith Busch <keith.busch@intel.com>
 CC: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: Keith Busch <keith.busch@intel.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- hw/block/nvme.c | 4 ++--
+ docs/devel/zoned-storage.rst | 25 ++++++++++++++++++++++---
-file changed, 2 insertions(+), 2 deletions(-)
+file changed, 22 insertions(+), 3 deletions(-)
-diff --git a/hw/block/nvme.c b/hw/block/nvme.c
+diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst
 index XXXXXXX..XXXXXXX 100644
---- a/hw/block/nvme.c
+--- a/docs/devel/zoned-storage.rst
-+++ b/hw/block/nvme.c
++++ b/docs/devel/zoned-storage.rst
-@@ -XXX,XX +XXX,XX @@ static uint16_t nvme_get_feature(NvmeCtrl *n, NvmeCmd *cmd, NvmeRequest *req)
+@@ -XXX,XX +XXX,XX @@ When the BlockBackend's BlockLimits model reports a zoned storage device, users
-         result = blk_enable_write_cache(n->conf.blk);
+ like the virtio-blk emulation or the qemu-io-cmds.c utility can use block layer
-         break;
+ APIs for zoned storage emulation or testing.
-     case NVME_NUMBER_OF_QUEUES:
--        result = cpu_to_le32((n->num_queues - 1) | ((n->num_queues - 1) << 16));
+-For example, to test zone_report on a null_blk device using qemu-io is:
-+        result = cpu_to_le32((n->num_queues - 2) | ((n->num_queues - 2) << 16));
+-$ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0
-         break;
+--c "zrp offset nr_zones"
-     default:
++For example, to test zone_report on a null_blk device using qemu-io is::
-         return NVME_INVALID_FIELD | NVME_DNR;
++
-@@ -XXX,XX +XXX,XX @@ static uint16_t nvme_set_feature(NvmeCtrl *n, NvmeCmd *cmd, NvmeRequest *req)
++  $ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0 -c "zrp offset nr_zones"
-         break;
++
-     case NVME_NUMBER_OF_QUEUES:
++To expose the host's zoned block device through virtio-blk, the command line
-         req->cqe.result =
++can be (includes the -device parameter)::
--            cpu_to_le32((n->num_queues - 1) | ((n->num_queues - 1) << 16));
++
-+            cpu_to_le32((n->num_queues - 2) | ((n->num_queues - 2) << 16));
++  -blockdev node-name=drive0,driver=host_device,filename=/dev/nullb0,cache.direct=on \
-         break;
++      -device virtio-blk-pci,drive=drive0
-     default:
++
-         return NVME_INVALID_FIELD | NVME_DNR;
++Or only use the -drive parameter::
 +
 +  -driver driver=host_device,file=/dev/nullb0,if=virtio,cache.direct=on
 +
 +Additionally, QEMU has several ways of supporting zoned storage, including:
 +(1) Using virtio-scsi: --device scsi-block allows for the passing through of
 +SCSI ZBC devices, enabling the attachment of ZBC or ZAC HDDs to QEMU.
 +(2) PCI device pass-through: While NVMe ZNS emulation is available for testing
 +purposes, it cannot yet pass through a zoned device from the host. To pass on
 +the NVMe ZNS device to the guest, use VFIO PCI pass the entire NVMe PCI adapter
 +through to the guest. Likewise, an HDD HBA can be passed on to QEMU all HDDs
 +attached to the HBA.
 --
-.13.5
+.39.2

-[Qemu-devel] [PULL for-2.10 02/15] throttle: Fix wrong variable name in the header documentation
+[PULL 20/20] tracing: install trace events file only if necessary
-From: Alberto Garcia <berto@igalia.com>
+From: Carlos Santos <casantos@redhat.com>
-The level of the burst bucket is stored in bkt.burst_level, not
+It is not useful when configuring with --enable-trace-backends=nop.
 bkt.burst_length.
-Signed-off-by: Alberto Garcia <berto@igalia.com>
+Signed-off-by: Carlos Santos <casantos@redhat.com>
 Reviewed-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
 Message-id: 49aab2711d02f285567f3b3b13a113847af33812.1503580370.git.berto@igalia.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-Id: <20230408010410.281263-1-casantos@redhat.com>
 ---
- include/qemu/throttle.h | 2 +-
+ trace/meson.build | 2 +-
 file changed, 1 insertion(+), 1 deletion(-)
-diff --git a/include/qemu/throttle.h b/include/qemu/throttle.h
+diff --git a/trace/meson.build b/trace/meson.build
 index XXXXXXX..XXXXXXX 100644
---- a/include/qemu/throttle.h
+--- a/trace/meson.build
-+++ b/include/qemu/throttle.h
++++ b/trace/meson.build
-@@ -XXX,XX +XXX,XX @@ typedef enum {
+@@ -XXX,XX +XXX,XX @@ trace_events_all = custom_target('trace-events-all',
-  * - The bkt.avg rate does not apply until the bucket is full,
+                                  input: trace_events_files,
-  *   allowing the user to do bursts until then. The I/O limit during
+                                  command: [ 'cat', '@INPUT@' ],
-  *   bursts is bkt.max. To enforce this limit we keep an additional
+                                  capture: true,
-- *   bucket in bkt.burst_length that leaks at a rate of bkt.max units
+-                                 install: true,
-+ *   bucket in bkt.burst_level that leaks at a rate of bkt.max units
++                                 install: get_option('trace_backends') != [ 'nop' ],
-  *   per second.
+                                  install_dir: qemu_datadir)
-  *
-  * - Because of all of the above, the user can perform I/O at a
+ if 'ust' in get_option('trace_backends')
 --
-.13.5
+.39.2

The following changes since commit 248b23735645f7cbb503d9be6f5bf825f2a603ab:

Update version for v2.10.0-rc4 release (2017-08-24 17:34:26 +0100)

are available in the git repository at:

git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 3e4c705212abfe8c9882a00beb2d1466a8a53cec:

qcow2: allocate cluster_cache/cluster_data on demand (2017-08-30 18:02:10 +0100)

----------------------------------------------------------------

Alberto Garcia (8):
  throttle: Fix wrong variable name in the header documentation
  throttle: Update the throttle_fix_bucket() documentation
  throttle: Make throttle_is_valid() a bit less verbose
  throttle: Remove throttle_fix_bucket() / throttle_unfix_bucket()
  throttle: Make LeakyBucket.avg and LeakyBucket.max integer types
  throttle: Make burst_length 64bit and add range checks
  throttle: Test the valid range of config values
  misc: Remove unused Error variables

Dan Aloni (1):
  nvme: Fix get/set number of queues feature, again

Eduardo Habkost (1):
  oslib-posix: Print errors before aborting on qemu_alloc_stack()

Fred Rolland (1):
  qemu-doc: Add UUID support in initiator name

Stefan Hajnoczi (4):
  scripts: add argparse module for Python 2.6 compatibility
  docker.py: Python 2.6 argparse compatibility
  tests: migration/guestperf Python 2.6 argparse compatibility
  qcow2: allocate cluster_cache/cluster_data on demand

-- 
2.13.5

From: Dan Aloni <dan@kernelim.com>

The number of queues that should be return by the admin command should:

1) Only mention the number of non-admin queues.
  2) It is zero-based, meaning that '0 == one non-admin queue',
     '1 == two non-admin queues', and so forth.

Because our `num_queues` means the number of queues _plus_ the admin
queue, then the right calculation for the number returned from the admin
command is `num_queues - 2`, combining the two requirements mentioned.

The issue was discovered by reducing num_queues from 64 to 8 and running
a Linux VM with an SMP parameter larger than that (e.g. 22). It tries to
utilize all queues, and therefore fails with an invalid queue number
when trying to queue I/Os on the last queue.

Signed-off-by: Dan Aloni <dan@kernelim.com>
CC: Alex Friedman <alex@e8storage.com>
CC: Keith Busch <keith.busch@intel.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/nvme.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -XXX,XX +XXX,XX @@ static uint16_t nvme_get_feature(NvmeCtrl *n, NvmeCmd *cmd, NvmeRequest *req)
         result = blk_enable_write_cache(n->conf.blk);
         break;
     case NVME_NUMBER_OF_QUEUES:
-        result = cpu_to_le32((n->num_queues - 1) | ((n->num_queues - 1) << 16));
+        result = cpu_to_le32((n->num_queues - 2) | ((n->num_queues - 2) << 16));
         break;
     default:
         return NVME_INVALID_FIELD | NVME_DNR;
@@ -XXX,XX +XXX,XX @@ static uint16_t nvme_set_feature(NvmeCtrl *n, NvmeCmd *cmd, NvmeRequest *req)
         break;
     case NVME_NUMBER_OF_QUEUES:
         req->cqe.result =
-            cpu_to_le32((n->num_queues - 1) | ((n->num_queues - 1) << 16));
+            cpu_to_le32((n->num_queues - 2) | ((n->num_queues - 2) << 16));
         break;
     default:
         return NVME_INVALID_FIELD | NVME_DNR;
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

The level of the burst bucket is stored in bkt.burst_level, not
bkt.burst_length.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
Message-id: 49aab2711d02f285567f3b3b13a113847af33812.1503580370.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/qemu/throttle.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/qemu/throttle.h b/include/qemu/throttle.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/throttle.h
+++ b/include/qemu/throttle.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
  * - The bkt.avg rate does not apply until the bucket is full,
  *   allowing the user to do bursts until then. The I/O limit during
  *   bursts is bkt.max. To enforce this limit we keep an additional
- *   bucket in bkt.burst_length that leaks at a rate of bkt.max units
+ *   bucket in bkt.burst_level that leaks at a rate of bkt.max units
  *   per second.
  *
  * - Because of all of the above, the user can perform I/O at a
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

The way the throttling algorithm works is that requests start being
throttled once the bucket level exceeds the burst limit. When we get
there the bucket leaks at the level set by the user (bkt->avg), and
that leak rate is what prevents guest I/O from exceeding the desired
limit.

If we don't allow bursts (i.e. bkt->max == 0) then we can start
throttling requests immediately. The problem with keeping the
threshold at 0 is that it only allows one request at a time, and as
soon as there's a bit of I/O from the guest every other request will
be throttled and performance will suffer considerably. That can even
make the guest unable to reach the throttle limit if that limit is
high enough, and that happens regardless of the block scheduler used
by the guest.

Increasing that threshold gives flexibility to the guest, allowing it
to perform short bursts of I/O before being throttled. Increasing the
threshold too much does not make a difference in the long run (because
it's the leak rate what defines the actual throughput) but it does
allow the guest to perform longer initial bursts and exceed the
throttle limit for a short while.

A burst value of bkt->avg / 10 allows the guest to perform 100ms'
worth of I/O at the target rate without being throttled.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 31aae6645f0d1fbf3860fb2b528b757236f0c0a7.1503580370.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/throttle.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/util/throttle.c b/util/throttle.c
index XXXXXXX..XXXXXXX 100644
--- a/util/throttle.c
+++ b/util/throttle.c
@@ -XXX,XX +XXX,XX @@ static void throttle_fix_bucket(LeakyBucket *bkt)
     /* zero bucket level */
     bkt->level = bkt->burst_level = 0;
 
-    /* The following is done to cope with the Linux CFQ block scheduler
-     * which regroup reads and writes by block of 100ms in the guest.
-     * When they are two process one making reads and one making writes cfq
-     * make a pattern looking like the following:
-     * WWWWWWWWWWWRRRRRRRRRRRRRRWWWWWWWWWWWWWwRRRRRRRRRRRRRRRRR
-     * Having a max burst value of 100ms of the average will help smooth the
-     * throttling
-     */
+    /* If bkt->max is 0 we still want to allow short bursts of I/O
+     * from the guest, otherwise every other request will be throttled
+     * and performance will suffer considerably. */
     min = bkt->avg / 10;
     if (bkt->avg && !bkt->max) {
         bkt->max = min;
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

Use a pointer to the bucket instead of repeating cfg->buckets[i] all
the time. This makes the code more concise and will help us expand the
checks later and save a few line breaks.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 763ffc40a26b17d54cf93f5a999e4656049fcf0c.1503580370.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/throttle.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/util/throttle.c b/util/throttle.c
index XXXXXXX..XXXXXXX 100644
--- a/util/throttle.c
+++ b/util/throttle.c
@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
     }
 
     for (i = 0; i < BUCKETS_COUNT; i++) {
-        if (cfg->buckets[i].avg < 0 ||
-            cfg->buckets[i].max < 0 ||
-            cfg->buckets[i].avg > THROTTLE_VALUE_MAX ||
-            cfg->buckets[i].max > THROTTLE_VALUE_MAX) {
+        LeakyBucket *bkt = &cfg->buckets[i];
+        if (bkt->avg < 0 || bkt->max < 0 ||
+            bkt->avg > THROTTLE_VALUE_MAX || bkt->max > THROTTLE_VALUE_MAX) {
             error_setg(errp, "bps/iops/max values must be within [0, %lld]",
                        THROTTLE_VALUE_MAX);
             return false;
         }
 
-        if (!cfg->buckets[i].burst_length) {
+        if (!bkt->burst_length) {
             error_setg(errp, "the burst length cannot be 0");
             return false;
         }
 
-        if (cfg->buckets[i].burst_length > 1 && !cfg->buckets[i].max) {
+        if (bkt->burst_length > 1 && !bkt->max) {
             error_setg(errp, "burst length set without burst rate");
             return false;
         }
 
-        if (cfg->buckets[i].max && !cfg->buckets[i].avg) {
+        if (bkt->max && !bkt->avg) {
             error_setg(errp, "bps_max/iops_max require corresponding"
                        " bps/iops values");
             return false;
         }
 
-        if (cfg->buckets[i].max && cfg->buckets[i].max < cfg->buckets[i].avg) {
+        if (bkt->max && bkt->max < bkt->avg) {
             error_setg(errp, "bps_max/iops_max cannot be lower than bps/iops");
             return false;
         }
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

The throttling code can change internally the value of bkt->max if it
hasn't been set by the user. The problem with this is that if we want
to retrieve the original value we have to undo this change first. This
is ugly and unnecessary: this patch removes the throttle_fix_bucket()
and throttle_unfix_bucket() functions completely and moves the logic
to throttle_compute_wait().

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
Message-id: 5b0b9e1ac6eb208d709eddc7b09e7669a523bff3.1503580370.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/throttle.c | 62 +++++++++++++++++++++------------------------------------
 1 file changed, 23 insertions(+), 39 deletions(-)

diff --git a/util/throttle.c b/util/throttle.c
index XXXXXXX..XXXXXXX 100644
--- a/util/throttle.c
+++ b/util/throttle.c
@@ -XXX,XX +XXX,XX @@ static int64_t throttle_do_compute_wait(double limit, double extra)
 int64_t throttle_compute_wait(LeakyBucket *bkt)
 {
     double extra; /* the number of extra units blocking the io */
+    double bucket_size;   /* I/O before throttling to bkt->avg */
+    double burst_bucket_size; /* Before throttling to bkt->max */
 
     if (!bkt->avg) {
         return 0;
     }
 
-    /* If the bucket is full then we have to wait */
-    extra = bkt->level - bkt->max * bkt->burst_length;
+    if (!bkt->max) {
+        /* If bkt->max is 0 we still want to allow short bursts of I/O
+         * from the guest, otherwise every other request will be throttled
+         * and performance will suffer considerably. */
+        bucket_size = bkt->avg / 10;
+        burst_bucket_size = 0;
+    } else {
+        /* If we have a burst limit then we have to wait until all I/O
+         * at burst rate has finished before throttling to bkt->avg */
+        bucket_size = bkt->max * bkt->burst_length;
+        burst_bucket_size = bkt->max / 10;
+    }
+
+    /* If the main bucket is full then we have to wait */
+    extra = bkt->level - bucket_size;
     if (extra > 0) {
         return throttle_do_compute_wait(bkt->avg, extra);
     }
 
-    /* If the bucket is not full yet we have to make sure that we
-     * fulfill the goal of bkt->max units per second. */
+    /* If the main bucket is not full yet we still have to check the
+     * burst bucket in order to enforce the burst limit */
     if (bkt->burst_length > 1) {
-        /* We use 1/10 of the max value to smooth the throttling.
-         * See throttle_fix_bucket() for more details. */
-        extra = bkt->burst_level - bkt->max / 10;
+        extra = bkt->burst_level - burst_bucket_size;
         if (extra > 0) {
             return throttle_do_compute_wait(bkt->max, extra);
         }
@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
     return true;
 }
 
-/* fix bucket parameters */
-static void throttle_fix_bucket(LeakyBucket *bkt)
-{
-    double min;
-
-    /* zero bucket level */
-    bkt->level = bkt->burst_level = 0;
-
-    /* If bkt->max is 0 we still want to allow short bursts of I/O
-     * from the guest, otherwise every other request will be throttled
-     * and performance will suffer considerably. */
-    min = bkt->avg / 10;
-    if (bkt->avg && !bkt->max) {
-        bkt->max = min;
-    }
-}
-
-/* undo internal bucket parameter changes (see throttle_fix_bucket()) */
-static void throttle_unfix_bucket(LeakyBucket *bkt)
-{
-    if (bkt->max < bkt->avg) {
-        bkt->max = 0;
-    }
-}
-
 /* Used to configure the throttle
  *
  * @ts: the throttle state we are working on
@@ -XXX,XX +XXX,XX @@ void throttle_config(ThrottleState *ts,
 
     ts->cfg = *cfg;
 
+    /* Zero bucket level */
     for (i = 0; i < BUCKETS_COUNT; i++) {
-        throttle_fix_bucket(&ts->cfg.buckets[i]);
+        ts->cfg.buckets[i].level = 0;
+        ts->cfg.buckets[i].burst_level = 0;
     }
 
     ts->previous_leak = qemu_clock_get_ns(clock_type);
@@ -XXX,XX +XXX,XX @@ void throttle_config(ThrottleState *ts,
  */
 void throttle_get_config(ThrottleState *ts, ThrottleConfig *cfg)
 {
-    int i;
-
     *cfg = ts->cfg;
-
-    for (i = 0; i < BUCKETS_COUNT; i++) {
-        throttle_unfix_bucket(&cfg->buckets[i]);
-    }
 }
 
 
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

Both the throttling limits set with the throttling.iops-* and
throttling.bps-* options and their QMP equivalents defined in the
BlockIOThrottle struct are integer values.

Those limits are also reported in the BlockDeviceInfo struct and they
are integers there as well.

Therefore there's no reason to store them internally as double and do
the conversion everytime we're setting or querying them, so this patch
uses uint64_t for those types. Let's also use an unsigned type because
we don't allow negative values anyway.

LeakyBucket.level and LeakyBucket.burst_level do however remain double
because their value changes depending on the fraction of time elapsed
since the previous I/O operation.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: f29b840422767b5be2c41c2dfdbbbf6c5f8fedf8.1503580370.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/qemu/throttle.h | 4 ++--
 tests/test-throttle.c   | 3 ++-
 util/throttle.c         | 7 +++----
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/qemu/throttle.h b/include/qemu/throttle.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/throttle.h
+++ b/include/qemu/throttle.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
  */
 
 typedef struct LeakyBucket {
-    double  avg;              /* average goal in units per second */
-    double  max;              /* leaky bucket max burst in units */
+    uint64_t avg;             /* average goal in units per second */
+    uint64_t max;             /* leaky bucket max burst in units */
     double  level;            /* bucket level in units */
     double  burst_level;      /* bucket level in units (for computing bursts) */
     unsigned burst_length;    /* max length of the burst period, in seconds */
diff --git a/tests/test-throttle.c b/tests/test-throttle.c
index XXXXXXX..XXXXXXX 100644
--- a/tests/test-throttle.c
+++ b/tests/test-throttle.c
@@ -XXX,XX +XXX,XX @@ static void test_enabled(void)
     for (i = 0; i < BUCKETS_COUNT; i++) {
         throttle_config_init(&cfg);
         set_cfg_value(false, i, 150);
+        g_assert(throttle_is_valid(&cfg, NULL));
         g_assert(throttle_enabled(&cfg));
     }
 
     for (i = 0; i < BUCKETS_COUNT; i++) {
         throttle_config_init(&cfg);
         set_cfg_value(false, i, -150);
-        g_assert(!throttle_enabled(&cfg));
+        g_assert(!throttle_is_valid(&cfg, NULL));
     }
 }
 
diff --git a/util/throttle.c b/util/throttle.c
index XXXXXXX..XXXXXXX 100644
--- a/util/throttle.c
+++ b/util/throttle.c
@@ -XXX,XX +XXX,XX @@ int64_t throttle_compute_wait(LeakyBucket *bkt)
         /* If bkt->max is 0 we still want to allow short bursts of I/O
          * from the guest, otherwise every other request will be throttled
          * and performance will suffer considerably. */
-        bucket_size = bkt->avg / 10;
+        bucket_size = (double) bkt->avg / 10;
         burst_bucket_size = 0;
     } else {
         /* If we have a burst limit then we have to wait until all I/O
          * at burst rate has finished before throttling to bkt->avg */
         bucket_size = bkt->max * bkt->burst_length;
-        burst_bucket_size = bkt->max / 10;
+        burst_bucket_size = (double) bkt->max / 10;
     }
 
     /* If the main bucket is full then we have to wait */
@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
 
     for (i = 0; i < BUCKETS_COUNT; i++) {
         LeakyBucket *bkt = &cfg->buckets[i];
-        if (bkt->avg < 0 || bkt->max < 0 ||
-            bkt->avg > THROTTLE_VALUE_MAX || bkt->max > THROTTLE_VALUE_MAX) {
+        if (bkt->avg > THROTTLE_VALUE_MAX || bkt->max > THROTTLE_VALUE_MAX) {
             error_setg(errp, "bps/iops/max values must be within [0, %lld]",
                        THROTTLE_VALUE_MAX);
             return false;
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

LeakyBucket.burst_length is defined as an unsigned integer but the
code never checks for overflows and it only makes sure that the value
is not 0.

In practice this means that the user can set something like
throttling.iops-total-max-length=4294967300 despite being larger than
UINT_MAX and the final value after casting to unsigned int will be 4.

This patch changes the data type to uint64_t. This does not increase
the storage size of LeakyBucket, and allows us to assign the value
directly from qemu_opt_get_number() or BlockIOThrottle and then do the
checks directly in throttle_is_valid().

The value of burst_length does not have a specific upper limit,
but since the bucket size is defined by max * burst_length we have
to prevent overflows. Instead of going for UINT64_MAX or something
similar this patch reuses THROTTLE_VALUE_MAX, which allows I/O bursts
of 1 GiB/s for 10 days in a row.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 1b2e3049803f71cafb2e1fa1be4fb47147a0d398.1503580370.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/qemu/throttle.h | 2 +-
 util/throttle.c         | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/include/qemu/throttle.h b/include/qemu/throttle.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/throttle.h
+++ b/include/qemu/throttle.h
@@ -XXX,XX +XXX,XX @@ typedef struct LeakyBucket {
     uint64_t max;             /* leaky bucket max burst in units */
     double  level;            /* bucket level in units */
     double  burst_level;      /* bucket level in units (for computing bursts) */
-    unsigned burst_length;    /* max length of the burst period, in seconds */
+    uint64_t burst_length;    /* max length of the burst period, in seconds */
 } LeakyBucket;
 
 /* The following structure is used to configure a ThrottleState
diff --git a/util/throttle.c b/util/throttle.c
index XXXXXXX..XXXXXXX 100644
--- a/util/throttle.c
+++ b/util/throttle.c
@@ -XXX,XX +XXX,XX @@ bool throttle_is_valid(ThrottleConfig *cfg, Error **errp)
             return false;
         }
 
+        if (bkt->max && bkt->burst_length > THROTTLE_VALUE_MAX / bkt->max) {
+            error_setg(errp, "burst length too high for this burst rate");
+            return false;
+        }
+
         if (bkt->max && !bkt->avg) {
             error_setg(errp, "bps_max/iops_max require corresponding"
                        " bps/iops values");
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: a57dd6274e1b6dc9c28769fec4c7ea543be5c5e3.1503580370.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 tests/test-throttle.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/tests/test-throttle.c b/tests/test-throttle.c
index XXXXXXX..XXXXXXX 100644
--- a/tests/test-throttle.c
+++ b/tests/test-throttle.c
@@ -XXX,XX +XXX,XX @@ static void test_is_valid(void)
     test_is_valid_for_value(1, true);
 }
 
+static void test_ranges(void)
+{
+    int i;
+
+    for (i = 0; i < BUCKETS_COUNT; i++) {
+        LeakyBucket *b = &cfg.buckets[i];
+        throttle_config_init(&cfg);
+
+        /* avg = 0 means throttling is disabled, but the config is valid */
+        b->avg = 0;
+        g_assert(throttle_is_valid(&cfg, NULL));
+        g_assert(!throttle_enabled(&cfg));
+
+        /* These are valid configurations (values <= THROTTLE_VALUE_MAX) */
+        b->avg = 1;
+        g_assert(throttle_is_valid(&cfg, NULL));
+
+        b->avg = THROTTLE_VALUE_MAX;
+        g_assert(throttle_is_valid(&cfg, NULL));
+
+        b->avg = THROTTLE_VALUE_MAX;
+        b->max = THROTTLE_VALUE_MAX;
+        g_assert(throttle_is_valid(&cfg, NULL));
+
+        /* Values over THROTTLE_VALUE_MAX are not allowed */
+        b->avg = THROTTLE_VALUE_MAX + 1;
+        g_assert(!throttle_is_valid(&cfg, NULL));
+
+        b->avg = THROTTLE_VALUE_MAX;
+        b->max = THROTTLE_VALUE_MAX + 1;
+        g_assert(!throttle_is_valid(&cfg, NULL));
+
+        /* burst_length must be between 1 and THROTTLE_VALUE_MAX */
+        b->avg = 1;
+        b->max = 1;
+        b->burst_length = 0;
+        g_assert(!throttle_is_valid(&cfg, NULL));
+
+        b->avg = 1;
+        b->max = 1;
+        b->burst_length = 1;
+        g_assert(throttle_is_valid(&cfg, NULL));
+
+        b->avg = 1;
+        b->max = 1;
+        b->burst_length = THROTTLE_VALUE_MAX;
+        g_assert(throttle_is_valid(&cfg, NULL));
+
+        b->avg = 1;
+        b->max = 1;
+        b->burst_length = THROTTLE_VALUE_MAX + 1;
+        g_assert(!throttle_is_valid(&cfg, NULL));
+
+        /* burst_length * max cannot exceed THROTTLE_VALUE_MAX */
+        b->avg = 1;
+        b->max = 2;
+        b->burst_length = THROTTLE_VALUE_MAX / 2;
+        g_assert(throttle_is_valid(&cfg, NULL));
+
+        b->avg = 1;
+        b->max = 3;
+        b->burst_length = THROTTLE_VALUE_MAX / 2;
+        g_assert(!throttle_is_valid(&cfg, NULL));
+
+        b->avg = 1;
+        b->max = THROTTLE_VALUE_MAX;
+        b->burst_length = 1;
+        g_assert(throttle_is_valid(&cfg, NULL));
+
+        b->avg = 1;
+        b->max = THROTTLE_VALUE_MAX;
+        b->burst_length = 2;
+        g_assert(!throttle_is_valid(&cfg, NULL));
+    }
+}
+
 static void test_max_is_missing_limit(void)
 {
     int i;
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
     g_test_add_func("/throttle/config/enabled",     test_enabled);
     g_test_add_func("/throttle/config/conflicting", test_conflicting_config);
     g_test_add_func("/throttle/config/is_valid",    test_is_valid);
+    g_test_add_func("/throttle/config/ranges",      test_ranges);
     g_test_add_func("/throttle/config/max",         test_max_is_missing_limit);
     g_test_add_func("/throttle/config/iops_size",
                     test_iops_size_is_missing_limit);
-- 
2.13.5

From: Eduardo Habkost <ehabkost@redhat.com>

If QEMU is running on a system that's out of memory and mmap()
fails, QEMU aborts with no error message at all, making it hard
to debug the reason for the failure.

Add perror() calls that will print error information before
aborting.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20170829212053.6003-1-ehabkost@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/oslib-posix.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -XXX,XX +XXX,XX @@ void *qemu_alloc_stack(size_t *sz)
     ptr = mmap(NULL, *sz, PROT_READ | PROT_WRITE,
                MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
     if (ptr == MAP_FAILED) {
+        perror("failed to allocate memory for stack");
         abort();
     }
 
@@ -XXX,XX +XXX,XX @@ void *qemu_alloc_stack(size_t *sz)
     guardpage = ptr;
 #endif
     if (mprotect(guardpage, pagesz, PROT_NONE) != 0) {
+        perror("failed to set up stack guard page");
         abort();
     }
 
-- 
2.13.5

From: Alberto Garcia <berto@igalia.com>

There's a few cases which we're passing an Error pointer to a function
only to discard it immediately afterwards without checking it. In
these cases we can simply remove the variable and pass NULL instead.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20170829120836.16091-1-berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/qcow.c  | 12 +++---------
 block/qcow2.c |  8 ++------
 dump.c        |  4 +---
 3 files changed, 6 insertions(+), 18 deletions(-)

diff --git a/block/qcow.c b/block/qcow.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -XXX,XX +XXX,XX @@ static uint64_t get_cluster_offset(BlockDriverState *bs,
                     start_sect = (offset & ~(s->cluster_size - 1)) >> 9;
                     for(i = 0; i < s->cluster_sectors; i++) {
                         if (i < n_start || i >= n_end) {
-                            Error *err = NULL;
                             memset(s->cluster_data, 0x00, 512);
                             if (qcrypto_block_encrypt(s->crypto, start_sect + i,
                                                       s->cluster_data,
                                                       BDRV_SECTOR_SIZE,
-                                                      &err) < 0) {
-                                error_free(err);
+                                                      NULL) < 0) {
                                 errno = EIO;
                                 return -1;
                             }
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_readv(BlockDriverState *bs, int64_t sector_num,
     QEMUIOVector hd_qiov;
     uint8_t *buf;
     void *orig_buf;
-    Error *err = NULL;
 
     if (qiov->niov > 1) {
         buf = orig_buf = qemu_try_blockalign(bs, qiov->size);
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_readv(BlockDriverState *bs, int64_t sector_num,
             if (bs->encrypted) {
                 assert(s->crypto);
                 if (qcrypto_block_decrypt(s->crypto, sector_num, buf,
-                                          n * BDRV_SECTOR_SIZE, &err) < 0) {
+                                          n * BDRV_SECTOR_SIZE, NULL) < 0) {
                     goto fail;
                 }
             }
@@ -XXX,XX +XXX,XX @@ done:
     return ret;
 
 fail:
-    error_free(err);
     ret = -EIO;
     goto done;
 }
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t sector_num,
             break;
         }
         if (bs->encrypted) {
-            Error *err = NULL;
             assert(s->crypto);
             if (qcrypto_block_encrypt(s->crypto, sector_num, buf,
-                                      n * BDRV_SECTOR_SIZE, &err) < 0) {
-                error_free(err);
+                                      n * BDRV_SECTOR_SIZE, NULL) < 0) {
                 ret = -EIO;
                 break;
             }
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset,
                 assert(s->crypto);
                 assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
                 assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-                Error *err = NULL;
                 if (qcrypto_block_decrypt(s->crypto,
                                           (s->crypt_physical_offset ?
                                            cluster_offset + offset_in_cluster :
                                            offset) >> BDRV_SECTOR_BITS,
                                           cluster_data,
                                           cur_bytes,
-                                          &err) < 0) {
-                    error_free(err);
+                                          NULL) < 0) {
                     ret = -EIO;
                     goto fail;
                 }
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset,
         qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes);
 
         if (bs->encrypted) {
-            Error *err = NULL;
             assert(s->crypto);
             if (!cluster_data) {
                 cluster_data = qemu_try_blockalign(bs->file->bs,
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset,
                                        cluster_offset + offset_in_cluster :
                                        offset) >> BDRV_SECTOR_BITS,
                                       cluster_data,
-                                      cur_bytes, &err) < 0) {
-                error_free(err);
+                                      cur_bytes, NULL) < 0) {
                 ret = -EIO;
                 goto fail;
             }
diff --git a/dump.c b/dump.c
index XXXXXXX..XXXXXXX 100644
--- a/dump.c
+++ b/dump.c
@@ -XXX,XX +XXX,XX @@ static void dump_process(DumpState *s, Error **errp)
 
 static void *dump_thread(void *data)
 {
-    Error *err = NULL;
     DumpState *s = (DumpState *)data;
-    dump_process(s, &err);
-    error_free(err);
+    dump_process(s, NULL);
     return NULL;
 }
 
-- 
2.13.5

The minimum Python version supported by QEMU is 2.6.  The argparse
standard library module was only added in Python 2.7.  Many scripts
would like to use argparse because it supports command-line
sub-commands.

This patch adds argparse.  See the top of argparse.py for details.

Suggested-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Message-id: 20170825155732.15665-2-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 COPYING.PYTHON      |  270 ++++++
 scripts/argparse.py | 2406 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 2676 insertions(+)
 create mode 100644 COPYING.PYTHON
 create mode 100644 scripts/argparse.py

diff --git a/COPYING.PYTHON b/COPYING.PYTHON
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/COPYING.PYTHON
@@ -XXX,XX +XXX,XX @@
+A. HISTORY OF THE SOFTWARE
+==========================
+
+Python was created in the early 1990s by Guido van Rossum at Stichting
+Mathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands
+as a successor of a language called ABC.  Guido remains Python's
+principal author, although it includes many contributions from others.
+
+In 1995, Guido continued his work on Python at the Corporation for
+National Research Initiatives (CNRI, see http://www.cnri.reston.va.us)
+in Reston, Virginia where he released several versions of the
+software.
+
+In May 2000, Guido and the Python core development team moved to
+BeOpen.com to form the BeOpen PythonLabs team.  In October of the same
+year, the PythonLabs team moved to Digital Creations (now Zope
+Corporation, see http://www.zope.com).  In 2001, the Python Software
+Foundation (PSF, see http://www.python.org/psf/) was formed, a
+non-profit organization created specifically to own Python-related
+Intellectual Property.  Zope Corporation is a sponsoring member of
+the PSF.
+
+All Python releases are Open Source (see http://www.opensource.org for
+the Open Source Definition).  Historically, most, but not all, Python
+releases have also been GPL-compatible; the table below summarizes
+the various releases.
+
+    Release         Derived     Year        Owner       GPL-
+                    from                                compatible? (1)
+
+    0.9.0 thru 1.2              1991-1995   CWI         yes
+    1.3 thru 1.5.2  1.2         1995-1999   CNRI        yes
+    1.6             1.5.2       2000        CNRI        no
+    2.0             1.6         2000        BeOpen.com  no
+    1.6.1           1.6         2001        CNRI        yes (2)
+    2.1             2.0+1.6.1   2001        PSF         no
+    2.0.1           2.0+1.6.1   2001        PSF         yes
+    2.1.1           2.1+2.0.1   2001        PSF         yes
+    2.2             2.1.1       2001        PSF         yes
+    2.1.2           2.1.1       2002        PSF         yes
+    2.1.3           2.1.2       2002        PSF         yes
+    2.2.1           2.2         2002        PSF         yes
+    2.2.2           2.2.1       2002        PSF         yes
+    2.2.3           2.2.2       2003        PSF         yes
+    2.3             2.2.2       2002-2003   PSF         yes
+    2.3.1           2.3         2002-2003   PSF         yes
+    2.3.2           2.3.1       2002-2003   PSF         yes
+    2.3.3           2.3.2       2002-2003   PSF         yes
+    2.3.4           2.3.3       2004        PSF         yes
+    2.3.5           2.3.4       2005        PSF         yes
+    2.4             2.3         2004        PSF         yes
+    2.4.1           2.4         2005        PSF         yes
+    2.4.2           2.4.1       2005        PSF         yes
+    2.4.3           2.4.2       2006        PSF         yes
+    2.5             2.4         2006        PSF         yes
+    2.7             2.6         2010        PSF         yes
+
+Footnotes:
+
+(1) GPL-compatible doesn't mean that we're distributing Python under
+    the GPL.  All Python licenses, unlike the GPL, let you distribute
+    a modified version without making your changes open source.  The
+    GPL-compatible licenses make it possible to combine Python with
+    other software that is released under the GPL; the others don't.
+
+(2) According to Richard Stallman, 1.6.1 is not GPL-compatible,
+    because its license has a choice of law clause.  According to
+    CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1
+    is "not incompatible" with the GPL.
+
+Thanks to the many outside volunteers who have worked under Guido's
+direction to make these releases possible.
+
+
+B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
+===============================================================
+
+PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
+--------------------------------------------
+
+1. This LICENSE AGREEMENT is between the Python Software Foundation
+("PSF"), and the Individual or Organization ("Licensee") accessing and
+otherwise using this software ("Python") in source or binary form and
+its associated documentation.
+
+2. Subject to the terms and conditions of this License Agreement, PSF
+hereby grants Licensee a nonexclusive, royalty-free, world-wide
+license to reproduce, analyze, test, perform and/or display publicly,
+prepare derivative works, distribute, and otherwise use Python
+alone or in any derivative version, provided, however, that PSF's
+License Agreement and PSF's notice of copyright, i.e., "Copyright (c)
+2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation; All Rights
+Reserved" are retained in Python alone or in any derivative version 
+prepared by Licensee.
+
+3. In the event Licensee prepares a derivative work that is based on
+or incorporates Python or any part thereof, and wants to make
+the derivative work available to others as provided herein, then
+Licensee hereby agrees to include in any such work a brief summary of
+the changes made to Python.
+
+4. PSF is making Python available to Licensee on an "AS IS"
+basis.  PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
+IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
+DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
+FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
+INFRINGE ANY THIRD PARTY RIGHTS.
+
+5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
+FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
+A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
+OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
+
+6. This License Agreement will automatically terminate upon a material
+breach of its terms and conditions.
+
+7. Nothing in this License Agreement shall be deemed to create any
+relationship of agency, partnership, or joint venture between PSF and
+Licensee.  This License Agreement does not grant permission to use PSF
+trademarks or trade name in a trademark sense to endorse or promote
+products or services of Licensee, or any third party.
+
+8. By copying, installing or otherwise using Python, Licensee
+agrees to be bound by the terms and conditions of this License
+Agreement.
+
+
+BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
+-------------------------------------------
+
+BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
+
+1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
+office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
+Individual or Organization ("Licensee") accessing and otherwise using
+this software in source or binary form and its associated
+documentation ("the Software").
+
+2. Subject to the terms and conditions of this BeOpen Python License
+Agreement, BeOpen hereby grants Licensee a non-exclusive,
+royalty-free, world-wide license to reproduce, analyze, test, perform
+and/or display publicly, prepare derivative works, distribute, and
+otherwise use the Software alone or in any derivative version,
+provided, however, that the BeOpen Python License is retained in the
+Software, alone or in any derivative version prepared by Licensee.
+
+3. BeOpen is making the Software available to Licensee on an "AS IS"
+basis.  BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
+IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
+DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
+FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
+INFRINGE ANY THIRD PARTY RIGHTS.
+
+4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
+SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
+AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
+DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
+
+5. This License Agreement will automatically terminate upon a material
+breach of its terms and conditions.
+
+6. This License Agreement shall be governed by and interpreted in all
+respects by the law of the State of California, excluding conflict of
+law provisions.  Nothing in this License Agreement shall be deemed to
+create any relationship of agency, partnership, or joint venture
+between BeOpen and Licensee.  This License Agreement does not grant
+permission to use BeOpen trademarks or trade names in a trademark
+sense to endorse or promote products or services of Licensee, or any
+third party.  As an exception, the "BeOpen Python" logos available at
+http://www.pythonlabs.com/logos.html may be used according to the
+permissions granted on that web page.
+
+7. By copying, installing or otherwise using the software, Licensee
+agrees to be bound by the terms and conditions of this License
+Agreement.
+
+
+CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
+---------------------------------------
+
+1. This LICENSE AGREEMENT is between the Corporation for National
+Research Initiatives, having an office at 1895 Preston White Drive,
+Reston, VA 20191 ("CNRI"), and the Individual or Organization
+("Licensee") accessing and otherwise using Python 1.6.1 software in
+source or binary form and its associated documentation.
+
+2. Subject to the terms and conditions of this License Agreement, CNRI
+hereby grants Licensee a nonexclusive, royalty-free, world-wide
+license to reproduce, analyze, test, perform and/or display publicly,
+prepare derivative works, distribute, and otherwise use Python 1.6.1
+alone or in any derivative version, provided, however, that CNRI's
+License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
+1995-2001 Corporation for National Research Initiatives; All Rights
+Reserved" are retained in Python 1.6.1 alone or in any derivative
+version prepared by Licensee.  Alternately, in lieu of CNRI's License
+Agreement, Licensee may substitute the following text (omitting the
+quotes): "Python 1.6.1 is made available subject to the terms and
+conditions in CNRI's License Agreement.  This Agreement together with
+Python 1.6.1 may be located on the Internet using the following
+unique, persistent identifier (known as a handle): 1895.22/1013.  This
+Agreement may also be obtained from a proxy server on the Internet
+using the following URL: http://hdl.handle.net/1895.22/1013".
+
+3. In the event Licensee prepares a derivative work that is based on
+or incorporates Python 1.6.1 or any part thereof, and wants to make
+the derivative work available to others as provided herein, then
+Licensee hereby agrees to include in any such work a brief summary of
+the changes made to Python 1.6.1.
+
+4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
+basis.  CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
+IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
+DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
+FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
+INFRINGE ANY THIRD PARTY RIGHTS.
+
+5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
+1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
+A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
+OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
+
+6. This License Agreement will automatically terminate upon a material
+breach of its terms and conditions.
+
+7. This License Agreement shall be governed by the federal
+intellectual property law of the United States, including without
+limitation the federal copyright law, and, to the extent such
+U.S. federal law does not apply, by the law of the Commonwealth of
+Virginia, excluding Virginia's conflict of law provisions.
+Notwithstanding the foregoing, with regard to derivative works based
+on Python 1.6.1 that incorporate non-separable material that was
+previously distributed under the GNU General Public License (GPL), the
+law of the Commonwealth of Virginia shall govern this License
+Agreement only as to issues arising under or with respect to
+Paragraphs 4, 5, and 7 of this License Agreement.  Nothing in this
+License Agreement shall be deemed to create any relationship of
+agency, partnership, or joint venture between CNRI and Licensee.  This
+License Agreement does not grant permission to use CNRI trademarks or
+trade name in a trademark sense to endorse or promote products or
+services of Licensee, or any third party.
+
+8. By clicking on the "ACCEPT" button where indicated, or by copying,
+installing or otherwise using Python 1.6.1, Licensee agrees to be
+bound by the terms and conditions of this License Agreement.
+
+        ACCEPT
+
+
+CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
+--------------------------------------------------
+
+Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
+The Netherlands.  All rights reserved.
+
+Permission to use, copy, modify, and distribute this software and its
+documentation for any purpose and without fee is hereby granted,
+provided that the above copyright notice appear in all copies and that
+both that copyright notice and this permission notice appear in
+supporting documentation, and that the name of Stichting Mathematisch
+Centrum or CWI not be used in advertising or publicity pertaining to
+distribution of the software without specific, written prior
+permission.
+
+STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
+THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
+FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
+FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
+OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
diff --git a/scripts/argparse.py b/scripts/argparse.py
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/scripts/argparse.py
@@ -XXX,XX +XXX,XX @@
+# This is a local copy of the standard library argparse module taken from PyPI.
+# It is licensed under the Python Software Foundation License.  This is a
+# fallback for Python 2.6 which does not include this module.  Python 2.7+ and
+# 3+ will never load this module because built-in modules are loaded before
+# anything in sys.path.
+#
+# If your script is not located in the same directory as this file, import it
+# like this:
+#
+#   import os
+#   import sys
+#   sys.path.append(os.path.join(os.path.dirname(__file__), ..., 'scripts'))
+#   import argparse
+
+# Author: Steven J. Bethard <steven.bethard@gmail.com>.
+# Maintainer: Thomas Waldmann <tw@waldmann-edv.de>
+
+"""Command-line parsing library
+
+This module is an optparse-inspired command-line parsing library that:
+
+    - handles both optional and positional arguments
+    - produces highly informative usage messages
+    - supports parsers that dispatch to sub-parsers
+
+The following is a simple usage example that sums integers from the
+command-line and writes the result to a file::
+
+    parser = argparse.ArgumentParser(
+        description='sum the integers at the command line')
+    parser.add_argument(
+        'integers', metavar='int', nargs='+', type=int,
+        help='an integer to be summed')
+    parser.add_argument(
+        '--log', default=sys.stdout, type=argparse.FileType('w'),
+        help='the file where the sum should be written')
+    args = parser.parse_args()
+    args.log.write('%s' % sum(args.integers))
+    args.log.close()
+
+The module contains the following public classes:
+
+    - ArgumentParser -- The main entry point for command-line parsing. As the
+        example above shows, the add_argument() method is used to populate
+        the parser with actions for optional and positional arguments. Then
+        the parse_args() method is invoked to convert the args at the
+        command-line into an object with attributes.
+
+    - ArgumentError -- The exception raised by ArgumentParser objects when
+        there are errors with the parser's actions. Errors raised while
+        parsing the command-line are caught by ArgumentParser and emitted
+        as command-line messages.
+
+    - FileType -- A factory for defining types of files to be created. As the
+        example above shows, instances of FileType are typically passed as
+        the type= argument of add_argument() calls.
+
+    - Action -- The base class for parser actions. Typically actions are
+        selected by passing strings like 'store_true' or 'append_const' to
+        the action= argument of add_argument(). However, for greater
+        customization of ArgumentParser actions, subclasses of Action may
+        be defined and passed as the action= argument.
+
+    - HelpFormatter, RawDescriptionHelpFormatter, RawTextHelpFormatter,
+        ArgumentDefaultsHelpFormatter -- Formatter classes which
+        may be passed as the formatter_class= argument to the
+        ArgumentParser constructor. HelpFormatter is the default,
+        RawDescriptionHelpFormatter and RawTextHelpFormatter tell the parser
+        not to change the formatting for help text, and
+        ArgumentDefaultsHelpFormatter adds information about argument defaults
+        to the help.
+
+All other classes in this module are considered implementation details.
+(Also note that HelpFormatter and RawDescriptionHelpFormatter are only
+considered public as object names -- the API of the formatter objects is
+still considered an implementation detail.)
+"""
+
+__version__ = '1.4.0'  # we use our own version number independant of the
+                       # one in stdlib and we release this on pypi.
+
+__external_lib__ = True  # to make sure the tests really test THIS lib,
+                         # not the builtin one in Python stdlib
+
+__all__ = [
+    'ArgumentParser',
+    'ArgumentError',
+    'ArgumentTypeError',
+    'FileType',
+    'HelpFormatter',
+    'ArgumentDefaultsHelpFormatter',
+    'RawDescriptionHelpFormatter',
+    'RawTextHelpFormatter',
+    'Namespace',
+    'Action',
+    'ONE_OR_MORE',
+    'OPTIONAL',
+    'PARSER',
+    'REMAINDER',
+    'SUPPRESS',
+    'ZERO_OR_MORE',
+]
+
+
+import copy as _copy
+import os as _os
+import re as _re
+import sys as _sys
+import textwrap as _textwrap
+
+from gettext import gettext as _
+
+try:
+    set
+except NameError:
+    # for python < 2.4 compatibility (sets module is there since 2.3):
+    from sets import Set as set
+
+try:
+    basestring
+except NameError:
+    basestring = str
+
+try:
+    sorted
+except NameError:
+    # for python < 2.4 compatibility:
+    def sorted(iterable, reverse=False):
+        result = list(iterable)
+        result.sort()
+        if reverse:
+            result.reverse()
+        return result
+
+
+def _callable(obj):
+    return hasattr(obj, '__call__') or hasattr(obj, '__bases__')
+
+
+SUPPRESS = '==SUPPRESS=='
+
+OPTIONAL = '?'
+ZERO_OR_MORE = '*'
+ONE_OR_MORE = '+'
+PARSER = 'A...'
+REMAINDER = '...'
+_UNRECOGNIZED_ARGS_ATTR = '_unrecognized_args'
+
+# =============================
+# Utility functions and classes
+# =============================
+
+class _AttributeHolder(object):
+    """Abstract base class that provides __repr__.
+
+    The __repr__ method returns a string in the format::
+        ClassName(attr=name, attr=name, ...)
+    The attributes are determined either by a class-level attribute,
+    '_kwarg_names', or by inspecting the instance __dict__.
+    """
+
+    def __repr__(self):
+        type_name = type(self).__name__
+        arg_strings = []
+        for arg in self._get_args():
+            arg_strings.append(repr(arg))
+        for name, value in self._get_kwargs():
+            arg_strings.append('%s=%r' % (name, value))
+        return '%s(%s)' % (type_name, ', '.join(arg_strings))
+
+    def _get_kwargs(self):
+        return sorted(self.__dict__.items())
+
+    def _get_args(self):
+        return []
+
+
+def _ensure_value(namespace, name, value):
+    if getattr(namespace, name, None) is None:
+        setattr(namespace, name, value)
+    return getattr(namespace, name)
+
+
+# ===============
+# Formatting Help
+# ===============
+
+class HelpFormatter(object):
+    """Formatter for generating usage messages and argument help strings.
+
+    Only the name of this class is considered a public API. All the methods
+    provided by the class are considered an implementation detail.
+    """
+
+    def __init__(self,
+                 prog,
+                 indent_increment=2,
+                 max_help_position=24,
+                 width=None):
+
+        # default setting for width
+        if width is None:
+            try:
+                width = int(_os.environ['COLUMNS'])
+            except (KeyError, ValueError):
+                width = 80
+            width -= 2
+
+        self._prog = prog
+        self._indent_increment = indent_increment
+        self._max_help_position = max_help_position
+        self._width = width
+
+        self._current_indent = 0
+        self._level = 0
+        self._action_max_length = 0
+
+        self._root_section = self._Section(self, None)
+        self._current_section = self._root_section
+
+        self._whitespace_matcher = _re.compile(r'\s+')
+        self._long_break_matcher = _re.compile(r'\n\n\n+')
+
+    # ===============================
+    # Section and indentation methods
+    # ===============================
+    def _indent(self):
+        self._current_indent += self._indent_increment
+        self._level += 1
+
+    def _dedent(self):
+        self._current_indent -= self._indent_increment
+        assert self._current_indent >= 0, 'Indent decreased below 0.'
+        self._level -= 1
+
+    class _Section(object):
+
+        def __init__(self, formatter, parent, heading=None):
+            self.formatter = formatter
+            self.parent = parent
+            self.heading = heading
+            self.items = []
+
+        def format_help(self):
+            # format the indented section
+            if self.parent is not None:
+                self.formatter._indent()
+            join = self.formatter._join_parts
+            for func, args in self.items:
+                func(*args)
+            item_help = join([func(*args) for func, args in self.items])
+            if self.parent is not None:
+                self.formatter._dedent()
+
+            # return nothing if the section was empty
+            if not item_help:
+                return ''
+
+            # add the heading if the section was non-empty
+            if self.heading is not SUPPRESS and self.heading is not None:
+                current_indent = self.formatter._current_indent
+                heading = '%*s%s:\n' % (current_indent, '', self.heading)
+            else:
+                heading = ''
+
+            # join the section-initial newline, the heading and the help
+            return join(['\n', heading, item_help, '\n'])
+
+    def _add_item(self, func, args):
+        self._current_section.items.append((func, args))
+
+    # ========================
+    # Message building methods
+    # ========================
+    def start_section(self, heading):
+        self._indent()
+        section = self._Section(self, self._current_section, heading)
+        self._add_item(section.format_help, [])
+        self._current_section = section
+
+    def end_section(self):
+        self._current_section = self._current_section.parent
+        self._dedent()
+
+    def add_text(self, text):
+        if text is not SUPPRESS and text is not None:
+            self._add_item(self._format_text, [text])
+
+    def add_usage(self, usage, actions, groups, prefix=None):
+        if usage is not SUPPRESS:
+            args = usage, actions, groups, prefix
+            self._add_item(self._format_usage, args)
+
+    def add_argument(self, action):
+        if action.help is not SUPPRESS:
+
+            # find all invocations
+            get_invocation = self._format_action_invocation
+            invocations = [get_invocation(action)]
+            for subaction in self._iter_indented_subactions(action):
+                invocations.append(get_invocation(subaction))
+
+            # update the maximum item length
+            invocation_length = max([len(s) for s in invocations])
+            action_length = invocation_length + self._current_indent
+            self._action_max_length = max(self._action_max_length,
+                                          action_length)
+
+            # add the item to the list
+            self._add_item(self._format_action, [action])
+
+    def add_arguments(self, actions):
+        for action in actions:
+            self.add_argument(action)
+
+    # =======================
+    # Help-formatting methods
+    # =======================
+    def format_help(self):
+        help = self._root_section.format_help()
+        if help:
+            help = self._long_break_matcher.sub('\n\n', help)
+            help = help.strip('\n') + '\n'
+        return help
+
+    def _join_parts(self, part_strings):
+        return ''.join([part
+                        for part in part_strings
+                        if part and part is not SUPPRESS])
+
+    def _format_usage(self, usage, actions, groups, prefix):
+        if prefix is None:
+            prefix = _('usage: ')
+
+        # if usage is specified, use that
+        if usage is not None:
+            usage = usage % dict(prog=self._prog)
+
+        # if no optionals or positionals are available, usage is just prog
+        elif usage is None and not actions:
+            usage = '%(prog)s' % dict(prog=self._prog)
+
+        # if optionals and positionals are available, calculate usage
+        elif usage is None:
+            prog = '%(prog)s' % dict(prog=self._prog)
+
+            # split optionals from positionals
+            optionals = []
+            positionals = []
+            for action in actions:
+                if action.option_strings:
+                    optionals.append(action)
+                else:
+                    positionals.append(action)
+
+            # build full usage string
+            format = self._format_actions_usage
+            action_usage = format(optionals + positionals, groups)
+            usage = ' '.join([s for s in [prog, action_usage] if s])
+
+            # wrap the usage parts if it's too long
+            text_width = self._width - self._current_indent
+            if len(prefix) + len(usage) > text_width:
+
+                # break usage into wrappable parts
+                part_regexp = r'$.*?$+|\[.*?\]+|\S+'
+                opt_usage = format(optionals, groups)
+                pos_usage = format(positionals, groups)
+                opt_parts = _re.findall(part_regexp, opt_usage)
+                pos_parts = _re.findall(part_regexp, pos_usage)
+                assert ' '.join(opt_parts) == opt_usage
+                assert ' '.join(pos_parts) == pos_usage
+
+                # helper for wrapping lines
+                def get_lines(parts, indent, prefix=None):
+                    lines = []
+                    line = []
+                    if prefix is not None:
+                        line_len = len(prefix) - 1
+                    else:
+                        line_len = len(indent) - 1
+                    for part in parts:
+                        if line_len + 1 + len(part) > text_width:
+                            lines.append(indent + ' '.join(line))
+                            line = []
+                            line_len = len(indent) - 1
+                        line.append(part)
+                        line_len += len(part) + 1
+                    if line:
+                        lines.append(indent + ' '.join(line))
+                    if prefix is not None:
+                        lines[0] = lines[0][len(indent):]
+                    return lines
+
+                # if prog is short, follow it with optionals or positionals
+                if len(prefix) + len(prog) <= 0.75 * text_width:
+                    indent = ' ' * (len(prefix) + len(prog) + 1)
+                    if opt_parts:
+                        lines = get_lines([prog] + opt_parts, indent, prefix)
+                        lines.extend(get_lines(pos_parts, indent))
+                    elif pos_parts:
+                        lines = get_lines([prog] + pos_parts, indent, prefix)
+                    else:
+                        lines = [prog]
+
+                # if prog is long, put it on its own line
+                else:
+                    indent = ' ' * len(prefix)
+                    parts = opt_parts + pos_parts
+                    lines = get_lines(parts, indent)
+                    if len(lines) > 1:
+                        lines = []
+                        lines.extend(get_lines(opt_parts, indent))
+                        lines.extend(get_lines(pos_parts, indent))
+                    lines = [prog] + lines
+
+                # join lines into usage
+                usage = '\n'.join(lines)
+
+        # prefix with 'usage:'
+        return '%s%s\n\n' % (prefix, usage)
+
+    def _format_actions_usage(self, actions, groups):
+        # find group indices and identify actions in groups
+        group_actions = set()
+        inserts = {}
+        for group in groups:
+            try:
+                start = actions.index(group._group_actions[0])
+            except ValueError:
+                continue
+            else:
+                end = start + len(group._group_actions)
+                if actions[start:end] == group._group_actions:
+                    for action in group._group_actions:
+                        group_actions.add(action)
+                    if not group.required:
+                        if start in inserts:
+                            inserts[start] += ' ['
+                        else:
+                            inserts[start] = '['
+                        inserts[end] = ']'
+                    else:
+                        if start in inserts:
+                            inserts[start] += ' ('
+                        else:
+                            inserts[start] = '('
+                        inserts[end] = ')'
+                    for i in range(start + 1, end):
+                        inserts[i] = '|'
+
+        # collect all actions format strings
+        parts = []
+        for i, action in enumerate(actions):
+
+            # suppressed arguments are marked with None
+            # remove | separators for suppressed arguments
+            if action.help is SUPPRESS:
+                parts.append(None)
+                if inserts.get(i) == '|':
+                    inserts.pop(i)
+                elif inserts.get(i + 1) == '|':
+                    inserts.pop(i + 1)
+
+            # produce all arg strings
+            elif not action.option_strings:
+                part = self._format_args(action, action.dest)
+
+                # if it's in a group, strip the outer []
+                if action in group_actions:
+                    if part[0] == '[' and part[-1] == ']':
+                        part = part[1:-1]
+
+                # add the action string to the list
+                parts.append(part)
+
+            # produce the first way to invoke the option in brackets
+            else:
+                option_string = action.option_strings[0]
+
+                # if the Optional doesn't take a value, format is:
+                #    -s or --long
+                if action.nargs == 0:
+                    part = '%s' % option_string
+
+                # if the Optional takes a value, format is:
+                #    -s ARGS or --long ARGS
+                else:
+                    default = action.dest.upper()
+                    args_string = self._format_args(action, default)
+                    part = '%s %s' % (option_string, args_string)
+
+                # make it look optional if it's not required or in a group
+                if not action.required and action not in group_actions:
+                    part = '[%s]' % part
+
+                # add the action string to the list
+                parts.append(part)
+
+        # insert things at the necessary indices
+        for i in sorted(inserts, reverse=True):
+            parts[i:i] = [inserts[i]]
+
+        # join all the action items with spaces
+        text = ' '.join([item for item in parts if item is not None])
+
+        # clean up separators for mutually exclusive groups
+        open = r'[\[(]'
+        close = r'[\])]'
+        text = _re.sub(r'(%s) ' % open, r'\1', text)
+        text = _re.sub(r' (%s)' % close, r'\1', text)
+        text = _re.sub(r'%s *%s' % (open, close), r'', text)
+        text = _re.sub(r'$([^|]*)$', r'\1', text)
+        text = text.strip()
+
+        # return the text
+        return text
+
+    def _format_text(self, text):
+        if '%(prog)' in text:
+            text = text % dict(prog=self._prog)
+        text_width = self._width - self._current_indent
+        indent = ' ' * self._current_indent
+        return self._fill_text(text, text_width, indent) + '\n\n'
+
+    def _format_action(self, action):
+        # determine the required width and the entry label
+        help_position = min(self._action_max_length + 2,
+                            self._max_help_position)
+        help_width = self._width - help_position
+        action_width = help_position - self._current_indent - 2
+        action_header = self._format_action_invocation(action)
+
+        # ho nelp; start on same line and add a final newline
+        if not action.help:
+            tup = self._current_indent, '', action_header
+            action_header = '%*s%s\n' % tup
+
+        # short action name; start on the same line and pad two spaces
+        elif len(action_header) <= action_width:
+            tup = self._current_indent, '', action_width, action_header
+            action_header = '%*s%-*s  ' % tup
+            indent_first = 0
+
+        # long action name; start on the next line
+        else:
+            tup = self._current_indent, '', action_header
+            action_header = '%*s%s\n' % tup
+            indent_first = help_position
+
+        # collect the pieces of the action help
+        parts = [action_header]
+
+        # if there was help for the action, add lines of help text
+        if action.help:
+            help_text = self._expand_help(action)
+            help_lines = self._split_lines(help_text, help_width)
+            parts.append('%*s%s\n' % (indent_first, '', help_lines[0]))
+            for line in help_lines[1:]:
+                parts.append('%*s%s\n' % (help_position, '', line))
+
+        # or add a newline if the description doesn't end with one
+        elif not action_header.endswith('\n'):
+            parts.append('\n')
+
+        # if there are any sub-actions, add their help as well
+        for subaction in self._iter_indented_subactions(action):
+            parts.append(self._format_action(subaction))
+
+        # return a single string
+        return self._join_parts(parts)
+
+    def _format_action_invocation(self, action):
+        if not action.option_strings:
+            metavar, = self._metavar_formatter(action, action.dest)(1)
+            return metavar
+
+        else:
+            parts = []
+
+            # if the Optional doesn't take a value, format is:
+            #    -s, --long
+            if action.nargs == 0:
+                parts.extend(action.option_strings)
+
+            # if the Optional takes a value, format is:
+            #    -s ARGS, --long ARGS
+            else:
+                default = action.dest.upper()
+                args_string = self._format_args(action, default)
+                for option_string in action.option_strings:
+                    parts.append('%s %s' % (option_string, args_string))
+
+            return ', '.join(parts)
+
+    def _metavar_formatter(self, action, default_metavar):
+        if action.metavar is not None:
+            result = action.metavar
+        elif action.choices is not None:
+            choice_strs = [str(choice) for choice in action.choices]
+            result = '{%s}' % ','.join(choice_strs)
+        else:
+            result = default_metavar
+
+        def format(tuple_size):
+            if isinstance(result, tuple):
+                return result
+            else:
+                return (result, ) * tuple_size
+        return format
+
+    def _format_args(self, action, default_metavar):
+        get_metavar = self._metavar_formatter(action, default_metavar)
+        if action.nargs is None:
+            result = '%s' % get_metavar(1)
+        elif action.nargs == OPTIONAL:
+            result = '[%s]' % get_metavar(1)
+        elif action.nargs == ZERO_OR_MORE:
+            result = '[%s [%s ...]]' % get_metavar(2)
+        elif action.nargs == ONE_OR_MORE:
+            result = '%s [%s ...]' % get_metavar(2)
+        elif action.nargs == REMAINDER:
+            result = '...'
+        elif action.nargs == PARSER:
+            result = '%s ...' % get_metavar(1)
+        else:
+            formats = ['%s' for _ in range(action.nargs)]
+            result = ' '.join(formats) % get_metavar(action.nargs)
+        return result
+
+    def _expand_help(self, action):
+        params = dict(vars(action), prog=self._prog)
+        for name in list(params):
+            if params[name] is SUPPRESS:
+                del params[name]
+        for name in list(params):
+            if hasattr(params[name], '__name__'):
+                params[name] = params[name].__name__
+        if params.get('choices') is not None:
+            choices_str = ', '.join([str(c) for c in params['choices']])
+            params['choices'] = choices_str
+        return self._get_help_string(action) % params
+
+    def _iter_indented_subactions(self, action):
+        try:
+            get_subactions = action._get_subactions
+        except AttributeError:
+            pass
+        else:
+            self._indent()
+            for subaction in get_subactions():
+                yield subaction
+            self._dedent()
+
+    def _split_lines(self, text, width):
+        text = self._whitespace_matcher.sub(' ', text).strip()
+        return _textwrap.wrap(text, width)
+
+    def _fill_text(self, text, width, indent):
+        text = self._whitespace_matcher.sub(' ', text).strip()
+        return _textwrap.fill(text, width, initial_indent=indent,
+                                           subsequent_indent=indent)
+
+    def _get_help_string(self, action):
+        return action.help
+
+
+class RawDescriptionHelpFormatter(HelpFormatter):
+    """Help message formatter which retains any formatting in descriptions.
+
+    Only the name of this class is considered a public API. All the methods
+    provided by the class are considered an implementation detail.
+    """
+
+    def _fill_text(self, text, width, indent):
+        return ''.join([indent + line for line in text.splitlines(True)])
+
+
+class RawTextHelpFormatter(RawDescriptionHelpFormatter):
+    """Help message formatter which retains formatting of all help text.
+
+    Only the name of this class is considered a public API. All the methods
+    provided by the class are considered an implementation detail.
+    """
+
+    def _split_lines(self, text, width):
+        return text.splitlines()
+
+
+class ArgumentDefaultsHelpFormatter(HelpFormatter):
+    """Help message formatter which adds default values to argument help.
+
+    Only the name of this class is considered a public API. All the methods
+    provided by the class are considered an implementation detail.
+    """
+
+    def _get_help_string(self, action):
+        help = action.help
+        if '%(default)' not in action.help:
+            if action.default is not SUPPRESS:
+                defaulting_nargs = [OPTIONAL, ZERO_OR_MORE]
+                if action.option_strings or action.nargs in defaulting_nargs:
+                    help += ' (default: %(default)s)'
+        return help
+
+
+# =====================
+# Options and Arguments
+# =====================
+
+def _get_action_name(argument):
+    if argument is None:
+        return None
+    elif argument.option_strings:
+        return  '/'.join(argument.option_strings)
+    elif argument.metavar not in (None, SUPPRESS):
+        return argument.metavar
+    elif argument.dest not in (None, SUPPRESS):
+        return argument.dest
+    else:
+        return None
+
+
+class ArgumentError(Exception):
+    """An error from creating or using an argument (optional or positional).
+
+    The string value of this exception is the message, augmented with
+    information about the argument that caused it.
+    """
+
+    def __init__(self, argument, message):
+        self.argument_name = _get_action_name(argument)
+        self.message = message
+
+    def __str__(self):
+        if self.argument_name is None:
+            format = '%(message)s'
+        else:
+            format = 'argument %(argument_name)s: %(message)s'
+        return format % dict(message=self.message,
+                             argument_name=self.argument_name)
+
+
+class ArgumentTypeError(Exception):
+    """An error from trying to convert a command line string to a type."""
+    pass
+
+
+# ==============
+# Action classes
+# ==============
+
+class Action(_AttributeHolder):
+    """Information about how to convert command line strings to Python objects.
+
+    Action objects are used by an ArgumentParser to represent the information
+    needed to parse a single argument from one or more strings from the
+    command line. The keyword arguments to the Action constructor are also
+    all attributes of Action instances.
+
+    Keyword Arguments:
+
+        - option_strings -- A list of command-line option strings which
+            should be associated with this action.
+
+        - dest -- The name of the attribute to hold the created object(s)
+
+        - nargs -- The number of command-line arguments that should be
+            consumed. By default, one argument will be consumed and a single
+            value will be produced.  Other values include:
+                - N (an integer) consumes N arguments (and produces a list)
+                - '?' consumes zero or one arguments
+                - '*' consumes zero or more arguments (and produces a list)
+                - '+' consumes one or more arguments (and produces a list)
+            Note that the difference between the default and nargs=1 is that
+            with the default, a single value will be produced, while with
+            nargs=1, a list containing a single value will be produced.
+
+        - const -- The value to be produced if the option is specified and the
+            option uses an action that takes no values.
+
+        - default -- The value to be produced if the option is not specified.
+
+        - type -- The type which the command-line arguments should be converted
+            to, should be one of 'string', 'int', 'float', 'complex' or a
+            callable object that accepts a single string argument. If None,
+            'string' is assumed.
+
+        - choices -- A container of values that should be allowed. If not None,
+            after a command-line argument has been converted to the appropriate
+            type, an exception will be raised if it is not a member of this
+            collection.
+
+        - required -- True if the action must always be specified at the
+            command line. This is only meaningful for optional command-line
+            arguments.
+
+        - help -- The help string describing the argument.
+
+        - metavar -- The name to be used for the option's argument with the
+            help string. If None, the 'dest' value will be used as the name.
+    """
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 nargs=None,
+                 const=None,
+                 default=None,
+                 type=None,
+                 choices=None,
+                 required=False,
+                 help=None,
+                 metavar=None):
+        self.option_strings = option_strings
+        self.dest = dest
+        self.nargs = nargs
+        self.const = const
+        self.default = default
+        self.type = type
+        self.choices = choices
+        self.required = required
+        self.help = help
+        self.metavar = metavar
+
+    def _get_kwargs(self):
+        names = [
+            'option_strings',
+            'dest',
+            'nargs',
+            'const',
+            'default',
+            'type',
+            'choices',
+            'help',
+            'metavar',
+        ]
+        return [(name, getattr(self, name)) for name in names]
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        raise NotImplementedError(_('.__call__() not defined'))
+
+
+class _StoreAction(Action):
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 nargs=None,
+                 const=None,
+                 default=None,
+                 type=None,
+                 choices=None,
+                 required=False,
+                 help=None,
+                 metavar=None):
+        if nargs == 0:
+            raise ValueError('nargs for store actions must be > 0; if you '
+                             'have nothing to store, actions such as store '
+                             'true or store const may be more appropriate')
+        if const is not None and nargs != OPTIONAL:
+            raise ValueError('nargs must be %r to supply const' % OPTIONAL)
+        super(_StoreAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            nargs=nargs,
+            const=const,
+            default=default,
+            type=type,
+            choices=choices,
+            required=required,
+            help=help,
+            metavar=metavar)
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        setattr(namespace, self.dest, values)
+
+
+class _StoreConstAction(Action):
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 const,
+                 default=None,
+                 required=False,
+                 help=None,
+                 metavar=None):
+        super(_StoreConstAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            nargs=0,
+            const=const,
+            default=default,
+            required=required,
+            help=help)
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        setattr(namespace, self.dest, self.const)
+
+
+class _StoreTrueAction(_StoreConstAction):
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 default=False,
+                 required=False,
+                 help=None):
+        super(_StoreTrueAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            const=True,
+            default=default,
+            required=required,
+            help=help)
+
+
+class _StoreFalseAction(_StoreConstAction):
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 default=True,
+                 required=False,
+                 help=None):
+        super(_StoreFalseAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            const=False,
+            default=default,
+            required=required,
+            help=help)
+
+
+class _AppendAction(Action):
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 nargs=None,
+                 const=None,
+                 default=None,
+                 type=None,
+                 choices=None,
+                 required=False,
+                 help=None,
+                 metavar=None):
+        if nargs == 0:
+            raise ValueError('nargs for append actions must be > 0; if arg '
+                             'strings are not supplying the value to append, '
+                             'the append const action may be more appropriate')
+        if const is not None and nargs != OPTIONAL:
+            raise ValueError('nargs must be %r to supply const' % OPTIONAL)
+        super(_AppendAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            nargs=nargs,
+            const=const,
+            default=default,
+            type=type,
+            choices=choices,
+            required=required,
+            help=help,
+            metavar=metavar)
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        items = _copy.copy(_ensure_value(namespace, self.dest, []))
+        items.append(values)
+        setattr(namespace, self.dest, items)
+
+
+class _AppendConstAction(Action):
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 const,
+                 default=None,
+                 required=False,
+                 help=None,
+                 metavar=None):
+        super(_AppendConstAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            nargs=0,
+            const=const,
+            default=default,
+            required=required,
+            help=help,
+            metavar=metavar)
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        items = _copy.copy(_ensure_value(namespace, self.dest, []))
+        items.append(self.const)
+        setattr(namespace, self.dest, items)
+
+
+class _CountAction(Action):
+
+    def __init__(self,
+                 option_strings,
+                 dest,
+                 default=None,
+                 required=False,
+                 help=None):
+        super(_CountAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            nargs=0,
+            default=default,
+            required=required,
+            help=help)
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        new_count = _ensure_value(namespace, self.dest, 0) + 1
+        setattr(namespace, self.dest, new_count)
+
+
+class _HelpAction(Action):
+
+    def __init__(self,
+                 option_strings,
+                 dest=SUPPRESS,
+                 default=SUPPRESS,
+                 help=None):
+        super(_HelpAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            default=default,
+            nargs=0,
+            help=help)
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        parser.print_help()
+        parser.exit()
+
+
+class _VersionAction(Action):
+
+    def __init__(self,
+                 option_strings,
+                 version=None,
+                 dest=SUPPRESS,
+                 default=SUPPRESS,
+                 help="show program's version number and exit"):
+        super(_VersionAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            default=default,
+            nargs=0,
+            help=help)
+        self.version = version
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        version = self.version
+        if version is None:
+            version = parser.version
+        formatter = parser._get_formatter()
+        formatter.add_text(version)
+        parser.exit(message=formatter.format_help())
+
+
+class _SubParsersAction(Action):
+
+    class _ChoicesPseudoAction(Action):
+
+        def __init__(self, name, aliases, help):
+            metavar = dest = name
+            if aliases:
+                metavar += ' (%s)' % ', '.join(aliases)
+            sup = super(_SubParsersAction._ChoicesPseudoAction, self)
+            sup.__init__(option_strings=[], dest=dest, help=help,
+                        metavar=metavar)
+
+    def __init__(self,
+                 option_strings,
+                 prog,
+                 parser_class,
+                 dest=SUPPRESS,
+                 help=None,
+                 metavar=None):
+
+        self._prog_prefix = prog
+        self._parser_class = parser_class
+        self._name_parser_map = {}
+        self._choices_actions = []
+
+        super(_SubParsersAction, self).__init__(
+            option_strings=option_strings,
+            dest=dest,
+            nargs=PARSER,
+            choices=self._name_parser_map,
+            help=help,
+            metavar=metavar)
+
+    def add_parser(self, name, **kwargs):
+        # set prog from the existing prefix
+        if kwargs.get('prog') is None:
+            kwargs['prog'] = '%s %s' % (self._prog_prefix, name)
+
+        aliases = kwargs.pop('aliases', ())
+
+        # create a pseudo-action to hold the choice help
+        if 'help' in kwargs:
+            help = kwargs.pop('help')
+            choice_action = self._ChoicesPseudoAction(name, aliases, help)
+            self._choices_actions.append(choice_action)
+
+        # create the parser and add it to the map
+        parser = self._parser_class(**kwargs)
+        self._name_parser_map[name] = parser
+
+        # make parser available under aliases also
+        for alias in aliases:
+            self._name_parser_map[alias] = parser
+
+        return parser
+
+    def _get_subactions(self):
+        return self._choices_actions
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        parser_name = values[0]
+        arg_strings = values[1:]
+
+        # set the parser name if requested
+        if self.dest is not SUPPRESS:
+            setattr(namespace, self.dest, parser_name)
+
+        # select the parser
+        try:
+            parser = self._name_parser_map[parser_name]
+        except KeyError:
+            tup = parser_name, ', '.join(self._name_parser_map)
+            msg = _('unknown parser %r (choices: %s)' % tup)
+            raise ArgumentError(self, msg)
+
+        # parse all the remaining options into the namespace
+        # store any unrecognized options on the object, so that the top
+        # level parser can decide what to do with them
+        namespace, arg_strings = parser.parse_known_args(arg_strings, namespace)
+        if arg_strings:
+            vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
+            getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)
+
+
+# ==============
+# Type classes
+# ==============
+
+class FileType(object):
+    """Factory for creating file object types
+
+    Instances of FileType are typically passed as type= arguments to the
+    ArgumentParser add_argument() method.
+
+    Keyword Arguments:
+        - mode -- A string indicating how the file is to be opened. Accepts the
+            same values as the builtin open() function.
+        - bufsize -- The file's desired buffer size. Accepts the same values as
+            the builtin open() function.
+    """
+
+    def __init__(self, mode='r', bufsize=None):
+        self._mode = mode
+        self._bufsize = bufsize
+
+    def __call__(self, string):
+        # the special argument "-" means sys.std{in,out}
+        if string == '-':
+            if 'r' in self._mode:
+                return _sys.stdin
+            elif 'w' in self._mode:
+                return _sys.stdout
+            else:
+                msg = _('argument "-" with mode %r' % self._mode)
+                raise ValueError(msg)
+
+        try:
+            # all other arguments are used as file names
+            if self._bufsize:
+                return open(string, self._mode, self._bufsize)
+            else:
+                return open(string, self._mode)
+        except IOError:
+            err = _sys.exc_info()[1]
+            message = _("can't open '%s': %s")
+            raise ArgumentTypeError(message % (string, err))
+
+    def __repr__(self):
+        args = [self._mode, self._bufsize]
+        args_str = ', '.join([repr(arg) for arg in args if arg is not None])
+        return '%s(%s)' % (type(self).__name__, args_str)
+
+# ===========================
+# Optional and Positional Parsing
+# ===========================
+
+class Namespace(_AttributeHolder):
+    """Simple object for storing attributes.
+
+    Implements equality by attribute names and values, and provides a simple
+    string representation.
+    """
+
+    def __init__(self, **kwargs):
+        for name in kwargs:
+            setattr(self, name, kwargs[name])
+
+    __hash__ = None
+
+    def __eq__(self, other):
+        return vars(self) == vars(other)
+
+    def __ne__(self, other):
+        return not (self == other)
+
+    def __contains__(self, key):
+        return key in self.__dict__
+
+
+class _ActionsContainer(object):
+
+    def __init__(self,
+                 description,
+                 prefix_chars,
+                 argument_default,
+                 conflict_handler):
+        super(_ActionsContainer, self).__init__()
+
+        self.description = description
+        self.argument_default = argument_default
+        self.prefix_chars = prefix_chars
+        self.conflict_handler = conflict_handler
+
+        # set up registries
+        self._registries = {}
+
+        # register actions
+        self.register('action', None, _StoreAction)
+        self.register('action', 'store', _StoreAction)
+        self.register('action', 'store_const', _StoreConstAction)
+        self.register('action', 'store_true', _StoreTrueAction)
+        self.register('action', 'store_false', _StoreFalseAction)
+        self.register('action', 'append', _AppendAction)
+        self.register('action', 'append_const', _AppendConstAction)
+        self.register('action', 'count', _CountAction)
+        self.register('action', 'help', _HelpAction)
+        self.register('action', 'version', _VersionAction)
+        self.register('action', 'parsers', _SubParsersAction)
+
+        # raise an exception if the conflict handler is invalid
+        self._get_handler()
+
+        # action storage
+        self._actions = []
+        self._option_string_actions = {}
+
+        # groups
+        self._action_groups = []
+        self._mutually_exclusive_groups = []
+
+        # defaults storage
+        self._defaults = {}
+
+        # determines whether an "option" looks like a negative number
+        self._negative_number_matcher = _re.compile(r'^-\d+$|^-\d*\.\d+$')
+
+        # whether or not there are any optionals that look like negative
+        # numbers -- uses a list so it can be shared and edited
+        self._has_negative_number_optionals = []
+
+    # ====================
+    # Registration methods
+    # ====================
+    def register(self, registry_name, value, object):
+        registry = self._registries.setdefault(registry_name, {})
+        registry[value] = object
+
+    def _registry_get(self, registry_name, value, default=None):
+        return self._registries[registry_name].get(value, default)
+
+    # ==================================
+    # Namespace default accessor methods
+    # ==================================
+    def set_defaults(self, **kwargs):
+        self._defaults.update(kwargs)
+
+        # if these defaults match any existing arguments, replace
+        # the previous default on the object with the new one
+        for action in self._actions:
+            if action.dest in kwargs:
+                action.default = kwargs[action.dest]
+
+    def get_default(self, dest):
+        for action in self._actions:
+            if action.dest == dest and action.default is not None:
+                return action.default
+        return self._defaults.get(dest, None)
+
+
+    # =======================
+    # Adding argument actions
+    # =======================
+    def add_argument(self, *args, **kwargs):
+        """
+        add_argument(dest, ..., name=value, ...)
+        add_argument(option_string, option_string, ..., name=value, ...)
+        """
+
+        # if no positional args are supplied or only one is supplied and
+        # it doesn't look like an option string, parse a positional
+        # argument
+        chars = self.prefix_chars
+        if not args or len(args) == 1 and args[0][0] not in chars:
+            if args and 'dest' in kwargs:
+                raise ValueError('dest supplied twice for positional argument')
+            kwargs = self._get_positional_kwargs(*args, **kwargs)
+
+        # otherwise, we're adding an optional argument
+        else:
+            kwargs = self._get_optional_kwargs(*args, **kwargs)
+
+        # if no default was supplied, use the parser-level default
+        if 'default' not in kwargs:
+            dest = kwargs['dest']
+            if dest in self._defaults:
+                kwargs['default'] = self._defaults[dest]
+            elif self.argument_default is not None:
+                kwargs['default'] = self.argument_default
+
+        # create the action object, and add it to the parser
+        action_class = self._pop_action_class(kwargs)
+        if not _callable(action_class):
+            raise ValueError('unknown action "%s"' % action_class)
+        action = action_class(**kwargs)
+
+        # raise an error if the action type is not callable
+        type_func = self._registry_get('type', action.type, action.type)
+        if not _callable(type_func):
+            raise ValueError('%r is not callable' % type_func)
+
+        return self._add_action(action)
+
+    def add_argument_group(self, *args, **kwargs):
+        group = _ArgumentGroup(self, *args, **kwargs)
+        self._action_groups.append(group)
+        return group
+
+    def add_mutually_exclusive_group(self, **kwargs):
+        group = _MutuallyExclusiveGroup(self, **kwargs)
+        self._mutually_exclusive_groups.append(group)
+        return group
+
+    def _add_action(self, action):
+        # resolve any conflicts
+        self._check_conflict(action)
+
+        # add to actions list
+        self._actions.append(action)
+        action.container = self
+
+        # index the action by any option strings it has
+        for option_string in action.option_strings:
+            self._option_string_actions[option_string] = action
+
+        # set the flag if any option strings look like negative numbers
+        for option_string in action.option_strings:
+            if self._negative_number_matcher.match(option_string):
+                if not self._has_negative_number_optionals:
+                    self._has_negative_number_optionals.append(True)
+
+        # return the created action
+        return action
+
+    def _remove_action(self, action):
+        self._actions.remove(action)
+
+    def _add_container_actions(self, container):
+        # collect groups by titles
+        title_group_map = {}
+        for group in self._action_groups:
+            if group.title in title_group_map:
+                msg = _('cannot merge actions - two groups are named %r')
+                raise ValueError(msg % (group.title))
+            title_group_map[group.title] = group
+
+        # map each action to its group
+        group_map = {}
+        for group in container._action_groups:
+
+            # if a group with the title exists, use that, otherwise
+            # create a new group matching the container's group
+            if group.title not in title_group_map:
+                title_group_map[group.title] = self.add_argument_group(
+                    title=group.title,
+                    description=group.description,
+                    conflict_handler=group.conflict_handler)
+
+            # map the actions to their new group
+            for action in group._group_actions:
+                group_map[action] = title_group_map[group.title]
+
+        # add container's mutually exclusive groups
+        # NOTE: if add_mutually_exclusive_group ever gains title= and
+        # description= then this code will need to be expanded as above
+        for group in container._mutually_exclusive_groups:
+            mutex_group = self.add_mutually_exclusive_group(
+                required=group.required)
+
+            # map the actions to their new mutex group
+            for action in group._group_actions:
+                group_map[action] = mutex_group
+
+        # add all actions to this container or their group
+        for action in container._actions:
+            group_map.get(action, self)._add_action(action)
+
+    def _get_positional_kwargs(self, dest, **kwargs):
+        # make sure required is not specified
+        if 'required' in kwargs:
+            msg = _("'required' is an invalid argument for positionals")
+            raise TypeError(msg)
+
+        # mark positional arguments as required if at least one is
+        # always required
+        if kwargs.get('nargs') not in [OPTIONAL, ZERO_OR_MORE]:
+            kwargs['required'] = True
+        if kwargs.get('nargs') == ZERO_OR_MORE and 'default' not in kwargs:
+            kwargs['required'] = True
+
+        # return the keyword arguments with no option strings
+        return dict(kwargs, dest=dest, option_strings=[])
+
+    def _get_optional_kwargs(self, *args, **kwargs):
+        # determine short and long option strings
+        option_strings = []
+        long_option_strings = []
+        for option_string in args:
+            # error on strings that don't start with an appropriate prefix
+            if not option_string[0] in self.prefix_chars:
+                msg = _('invalid option string %r: '
+                        'must start with a character %r')
+                tup = option_string, self.prefix_chars
+                raise ValueError(msg % tup)
+
+            # strings starting with two prefix characters are long options
+            option_strings.append(option_string)
+            if option_string[0] in self.prefix_chars:
+                if len(option_string) > 1:
+                    if option_string[1] in self.prefix_chars:
+                        long_option_strings.append(option_string)
+
+        # infer destination, '--foo-bar' -> 'foo_bar' and '-x' -> 'x'
+        dest = kwargs.pop('dest', None)
+        if dest is None:
+            if long_option_strings:
+                dest_option_string = long_option_strings[0]
+            else:
+                dest_option_string = option_strings[0]
+            dest = dest_option_string.lstrip(self.prefix_chars)
+            if not dest:
+                msg = _('dest= is required for options like %r')
+                raise ValueError(msg % option_string)
+            dest = dest.replace('-', '_')
+
+        # return the updated keyword arguments
+        return dict(kwargs, dest=dest, option_strings=option_strings)
+
+    def _pop_action_class(self, kwargs, default=None):
+        action = kwargs.pop('action', default)
+        return self._registry_get('action', action, action)
+
+    def _get_handler(self):
+        # determine function from conflict handler string
+        handler_func_name = '_handle_conflict_%s' % self.conflict_handler
+        try:
+            return getattr(self, handler_func_name)
+        except AttributeError:
+            msg = _('invalid conflict_resolution value: %r')
+            raise ValueError(msg % self.conflict_handler)
+
+    def _check_conflict(self, action):
+
+        # find all options that conflict with this option
+        confl_optionals = []
+        for option_string in action.option_strings:
+            if option_string in self._option_string_actions:
+                confl_optional = self._option_string_actions[option_string]
+                confl_optionals.append((option_string, confl_optional))
+
+        # resolve any conflicts
+        if confl_optionals:
+            conflict_handler = self._get_handler()
+            conflict_handler(action, confl_optionals)
+
+    def _handle_conflict_error(self, action, conflicting_actions):
+        message = _('conflicting option string(s): %s')
+        conflict_string = ', '.join([option_string
+                                     for option_string, action
+                                     in conflicting_actions])
+        raise ArgumentError(action, message % conflict_string)
+
+    def _handle_conflict_resolve(self, action, conflicting_actions):
+
+        # remove all conflicting options
+        for option_string, action in conflicting_actions:
+
+            # remove the conflicting option
+            action.option_strings.remove(option_string)
+            self._option_string_actions.pop(option_string, None)
+
+            # if the option now has no option string, remove it from the
+            # container holding it
+            if not action.option_strings:
+                action.container._remove_action(action)
+
+
+class _ArgumentGroup(_ActionsContainer):
+
+    def __init__(self, container, title=None, description=None, **kwargs):
+        # add any missing keyword arguments by checking the container
+        update = kwargs.setdefault
+        update('conflict_handler', container.conflict_handler)
+        update('prefix_chars', container.prefix_chars)
+        update('argument_default', container.argument_default)
+        super_init = super(_ArgumentGroup, self).__init__
+        super_init(description=description, **kwargs)
+
+        # group attributes
+        self.title = title
+        self._group_actions = []
+
+        # share most attributes with the container
+        self._registries = container._registries
+        self._actions = container._actions
+        self._option_string_actions = container._option_string_actions
+        self._defaults = container._defaults
+        self._has_negative_number_optionals = \
+            container._has_negative_number_optionals
+
+    def _add_action(self, action):
+        action = super(_ArgumentGroup, self)._add_action(action)
+        self._group_actions.append(action)
+        return action
+
+    def _remove_action(self, action):
+        super(_ArgumentGroup, self)._remove_action(action)
+        self._group_actions.remove(action)
+
+
+class _MutuallyExclusiveGroup(_ArgumentGroup):
+
+    def __init__(self, container, required=False):
+        super(_MutuallyExclusiveGroup, self).__init__(container)
+        self.required = required
+        self._container = container
+
+    def _add_action(self, action):
+        if action.required:
+            msg = _('mutually exclusive arguments must be optional')
+            raise ValueError(msg)
+        action = self._container._add_action(action)
+        self._group_actions.append(action)
+        return action
+
+    def _remove_action(self, action):
+        self._container._remove_action(action)
+        self._group_actions.remove(action)
+
+
+class ArgumentParser(_AttributeHolder, _ActionsContainer):
+    """Object for parsing command line strings into Python objects.
+
+    Keyword Arguments:
+        - prog -- The name of the program (default: sys.argv[0])
+        - usage -- A usage message (default: auto-generated from arguments)
+        - description -- A description of what the program does
+        - epilog -- Text following the argument descriptions
+        - parents -- Parsers whose arguments should be copied into this one
+        - formatter_class -- HelpFormatter class for printing help messages
+        - prefix_chars -- Characters that prefix optional arguments
+        - fromfile_prefix_chars -- Characters that prefix files containing
+            additional arguments
+        - argument_default -- The default value for all arguments
+        - conflict_handler -- String indicating how to handle conflicts
+        - add_help -- Add a -h/-help option
+    """
+
+    def __init__(self,
+                 prog=None,
+                 usage=None,
+                 description=None,
+                 epilog=None,
+                 version=None,
+                 parents=[],
+                 formatter_class=HelpFormatter,
+                 prefix_chars='-',
+                 fromfile_prefix_chars=None,
+                 argument_default=None,
+                 conflict_handler='error',
+                 add_help=True):
+
+        if version is not None:
+            import warnings
+            warnings.warn(
+                """The "version" argument to ArgumentParser is deprecated. """
+                """Please use """
+                """"add_argument(..., action='version', version="N", ...)" """
+                """instead""", DeprecationWarning)
+
+        superinit = super(ArgumentParser, self).__init__
+        superinit(description=description,
+                  prefix_chars=prefix_chars,
+                  argument_default=argument_default,
+                  conflict_handler=conflict_handler)
+
+        # default setting for prog
+        if prog is None:
+            prog = _os.path.basename(_sys.argv[0])
+
+        self.prog = prog
+        self.usage = usage
+        self.epilog = epilog
+        self.version = version
+        self.formatter_class = formatter_class
+        self.fromfile_prefix_chars = fromfile_prefix_chars
+        self.add_help = add_help
+
+        add_group = self.add_argument_group
+        self._positionals = add_group(_('positional arguments'))
+        self._optionals = add_group(_('optional arguments'))
+        self._subparsers = None
+
+        # register types
+        def identity(string):
+            return string
+        self.register('type', None, identity)
+
+        # add help and version arguments if necessary
+        # (using explicit default to override global argument_default)
+        if '-' in prefix_chars:
+            default_prefix = '-'
+        else:
+            default_prefix = prefix_chars[0]
+        if self.add_help:
+            self.add_argument(
+                default_prefix+'h', default_prefix*2+'help',
+                action='help', default=SUPPRESS,
+                help=_('show this help message and exit'))
+        if self.version:
+            self.add_argument(
+                default_prefix+'v', default_prefix*2+'version',
+                action='version', default=SUPPRESS,
+                version=self.version,
+                help=_("show program's version number and exit"))
+
+        # add parent arguments and defaults
+        for parent in parents:
+            self._add_container_actions(parent)
+            try:
+                defaults = parent._defaults
+            except AttributeError:
+                pass
+            else:
+                self._defaults.update(defaults)
+
+    # =======================
+    # Pretty __repr__ methods
+    # =======================
+    def _get_kwargs(self):
+        names = [
+            'prog',
+            'usage',
+            'description',
+            'version',
+            'formatter_class',
+            'conflict_handler',
+            'add_help',
+        ]
+        return [(name, getattr(self, name)) for name in names]
+
+    # ==================================
+    # Optional/Positional adding methods
+    # ==================================
+    def add_subparsers(self, **kwargs):
+        if self._subparsers is not None:
+            self.error(_('cannot have multiple subparser arguments'))
+
+        # add the parser class to the arguments if it's not present
+        kwargs.setdefault('parser_class', type(self))
+
+        if 'title' in kwargs or 'description' in kwargs:
+            title = _(kwargs.pop('title', 'subcommands'))
+            description = _(kwargs.pop('description', None))
+            self._subparsers = self.add_argument_group(title, description)
+        else:
+            self._subparsers = self._positionals
+
+        # prog defaults to the usage message of this parser, skipping
+        # optional arguments and with no "usage:" prefix
+        if kwargs.get('prog') is None:
+            formatter = self._get_formatter()
+            positionals = self._get_positional_actions()
+            groups = self._mutually_exclusive_groups
+            formatter.add_usage(self.usage, positionals, groups, '')
+            kwargs['prog'] = formatter.format_help().strip()
+
+        # create the parsers action and add it to the positionals list
+        parsers_class = self._pop_action_class(kwargs, 'parsers')
+        action = parsers_class(option_strings=[], **kwargs)
+        self._subparsers._add_action(action)
+
+        # return the created parsers action
+        return action
+
+    def _add_action(self, action):
+        if action.option_strings:
+            self._optionals._add_action(action)
+        else:
+            self._positionals._add_action(action)
+        return action
+
+    def _get_optional_actions(self):
+        return [action
+                for action in self._actions
+                if action.option_strings]
+
+    def _get_positional_actions(self):
+        return [action
+                for action in self._actions
+                if not action.option_strings]
+
+    # =====================================
+    # Command line argument parsing methods
+    # =====================================
+    def parse_args(self, args=None, namespace=None):
+        args, argv = self.parse_known_args(args, namespace)
+        if argv:
+            msg = _('unrecognized arguments: %s')
+            self.error(msg % ' '.join(argv))
+        return args
+
+    def parse_known_args(self, args=None, namespace=None):
+        # args default to the system args
+        if args is None:
+            args = _sys.argv[1:]
+
+        # default Namespace built from parser defaults
+        if namespace is None:
+            namespace = Namespace()
+
+        # add any action defaults that aren't present
+        for action in self._actions:
+            if action.dest is not SUPPRESS:
+                if not hasattr(namespace, action.dest):
+                    if action.default is not SUPPRESS:
+                        setattr(namespace, action.dest, action.default)
+
+        # add any parser defaults that aren't present
+        for dest in self._defaults:
+            if not hasattr(namespace, dest):
+                setattr(namespace, dest, self._defaults[dest])
+
+        # parse the arguments and exit if there are any errors
+        try:
+            namespace, args = self._parse_known_args(args, namespace)
+            if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR):
+                args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))
+                delattr(namespace, _UNRECOGNIZED_ARGS_ATTR)
+            return namespace, args
+        except ArgumentError:
+            err = _sys.exc_info()[1]
+            self.error(str(err))
+
+    def _parse_known_args(self, arg_strings, namespace):
+        # replace arg strings that are file references
+        if self.fromfile_prefix_chars is not None:
+            arg_strings = self._read_args_from_files(arg_strings)
+
+        # map all mutually exclusive arguments to the other arguments
+        # they can't occur with
+        action_conflicts = {}
+        for mutex_group in self._mutually_exclusive_groups:
+            group_actions = mutex_group._group_actions
+            for i, mutex_action in enumerate(mutex_group._group_actions):
+                conflicts = action_conflicts.setdefault(mutex_action, [])
+                conflicts.extend(group_actions[:i])
+                conflicts.extend(group_actions[i + 1:])
+
+        # find all option indices, and determine the arg_string_pattern
+        # which has an 'O' if there is an option at an index,
+        # an 'A' if there is an argument, or a '-' if there is a '--'
+        option_string_indices = {}
+        arg_string_pattern_parts = []
+        arg_strings_iter = iter(arg_strings)
+        for i, arg_string in enumerate(arg_strings_iter):
+
+            # all args after -- are non-options
+            if arg_string == '--':
+                arg_string_pattern_parts.append('-')
+                for arg_string in arg_strings_iter:
+                    arg_string_pattern_parts.append('A')
+
+            # otherwise, add the arg to the arg strings
+            # and note the index if it was an option
+            else:
+                option_tuple = self._parse_optional(arg_string)
+                if option_tuple is None:
+                    pattern = 'A'
+                else:
+                    option_string_indices[i] = option_tuple
+                    pattern = 'O'
+                arg_string_pattern_parts.append(pattern)
+
+        # join the pieces together to form the pattern
+        arg_strings_pattern = ''.join(arg_string_pattern_parts)
+
+        # converts arg strings to the appropriate and then takes the action
+        seen_actions = set()
+        seen_non_default_actions = set()
+
+        def take_action(action, argument_strings, option_string=None):
+            seen_actions.add(action)
+            argument_values = self._get_values(action, argument_strings)
+
+            # error if this argument is not allowed with other previously
+            # seen arguments, assuming that actions that use the default
+            # value don't really count as "present"
+            if argument_values is not action.default:
+                seen_non_default_actions.add(action)
+                for conflict_action in action_conflicts.get(action, []):
+                    if conflict_action in seen_non_default_actions:
+                        msg = _('not allowed with argument %s')
+                        action_name = _get_action_name(conflict_action)
+                        raise ArgumentError(action, msg % action_name)
+
+            # take the action if we didn't receive a SUPPRESS value
+            # (e.g. from a default)
+            if argument_values is not SUPPRESS:
+                action(self, namespace, argument_values, option_string)
+
+        # function to convert arg_strings into an optional action
+        def consume_optional(start_index):
+
+            # get the optional identified at this index
+            option_tuple = option_string_indices[start_index]
+            action, option_string, explicit_arg = option_tuple
+
+            # identify additional optionals in the same arg string
+            # (e.g. -xyz is the same as -x -y -z if no args are required)
+            match_argument = self._match_argument
+            action_tuples = []
+            while True:
+
+                # if we found no optional action, skip it
+                if action is None:
+                    extras.append(arg_strings[start_index])
+                    return start_index + 1
+
+                # if there is an explicit argument, try to match the
+                # optional's string arguments to only this
+                if explicit_arg is not None:
+                    arg_count = match_argument(action, 'A')
+
+                    # if the action is a single-dash option and takes no
+                    # arguments, try to parse more single-dash options out
+                    # of the tail of the option string
+                    chars = self.prefix_chars
+                    if arg_count == 0 and option_string[1] not in chars:
+                        action_tuples.append((action, [], option_string))
+                        char = option_string[0]
+                        option_string = char + explicit_arg[0]
+                        new_explicit_arg = explicit_arg[1:] or None
+                        optionals_map = self._option_string_actions
+                        if option_string in optionals_map:
+                            action = optionals_map[option_string]
+                            explicit_arg = new_explicit_arg
+                        else:
+                            msg = _('ignored explicit argument %r')
+                            raise ArgumentError(action, msg % explicit_arg)
+
+                    # if the action expect exactly one argument, we've
+                    # successfully matched the option; exit the loop
+                    elif arg_count == 1:
+                        stop = start_index + 1
+                        args = [explicit_arg]
+                        action_tuples.append((action, args, option_string))
+                        break
+
+                    # error if a double-dash option did not use the
+                    # explicit argument
+                    else:
+                        msg = _('ignored explicit argument %r')
+                        raise ArgumentError(action, msg % explicit_arg)
+
+                # if there is no explicit argument, try to match the
+                # optional's string arguments with the following strings
+                # if successful, exit the loop
+                else:
+                    start = start_index + 1
+                    selected_patterns = arg_strings_pattern[start:]
+                    arg_count = match_argument(action, selected_patterns)
+                    stop = start + arg_count
+                    args = arg_strings[start:stop]
+                    action_tuples.append((action, args, option_string))
+                    break
+
+            # add the Optional to the list and return the index at which
+            # the Optional's string args stopped
+            assert action_tuples
+            for action, args, option_string in action_tuples:
+                take_action(action, args, option_string)
+            return stop
+
+        # the list of Positionals left to be parsed; this is modified
+        # by consume_positionals()
+        positionals = self._get_positional_actions()
+
+        # function to convert arg_strings into positional actions
+        def consume_positionals(start_index):
+            # match as many Positionals as possible
+            match_partial = self._match_arguments_partial
+            selected_pattern = arg_strings_pattern[start_index:]
+            arg_counts = match_partial(positionals, selected_pattern)
+
+            # slice off the appropriate arg strings for each Positional
+            # and add the Positional and its args to the list
+            for action, arg_count in zip(positionals, arg_counts):
+                args = arg_strings[start_index: start_index + arg_count]
+                start_index += arg_count
+                take_action(action, args)
+
+            # slice off the Positionals that we just parsed and return the
+            # index at which the Positionals' string args stopped
+            positionals[:] = positionals[len(arg_counts):]
+            return start_index
+
+        # consume Positionals and Optionals alternately, until we have
+        # passed the last option string
+        extras = []
+        start_index = 0
+        if option_string_indices:
+            max_option_string_index = max(option_string_indices)
+        else:
+            max_option_string_index = -1
+        while start_index <= max_option_string_index:
+
+            # consume any Positionals preceding the next option
+            next_option_string_index = min([
+                index
+                for index in option_string_indices
+                if index >= start_index])
+            if start_index != next_option_string_index:
+                positionals_end_index = consume_positionals(start_index)
+
+                # only try to parse the next optional if we didn't consume
+                # the option string during the positionals parsing
+                if positionals_end_index > start_index:
+                    start_index = positionals_end_index
+                    continue
+                else:
+                    start_index = positionals_end_index
+
+            # if we consumed all the positionals we could and we're not
+            # at the index of an option string, there were extra arguments
+            if start_index not in option_string_indices:
+                strings = arg_strings[start_index:next_option_string_index]
+                extras.extend(strings)
+                start_index = next_option_string_index
+
+            # consume the next optional and any arguments for it
+            start_index = consume_optional(start_index)
+
+        # consume any positionals following the last Optional
+        stop_index = consume_positionals(start_index)
+
+        # if we didn't consume all the argument strings, there were extras
+        extras.extend(arg_strings[stop_index:])
+
+        # if we didn't use all the Positional objects, there were too few
+        # arg strings supplied.
+        if positionals:
+            self.error(_('too few arguments'))
+
+        # make sure all required actions were present, and convert defaults.
+        for action in self._actions:
+            if action not in seen_actions:
+                if action.required:
+                    name = _get_action_name(action)
+                    self.error(_('argument %s is required') % name)
+                else:
+                    # Convert action default now instead of doing it before
+                    # parsing arguments to avoid calling convert functions
+                    # twice (which may fail) if the argument was given, but
+                    # only if it was defined already in the namespace
+                    if (action.default is not None and
+                            isinstance(action.default, basestring) and
+                            hasattr(namespace, action.dest) and
+                            action.default is getattr(namespace, action.dest)):
+                        setattr(namespace, action.dest,
+                                self._get_value(action, action.default))
+
+        # make sure all required groups had one option present
+        for group in self._mutually_exclusive_groups:
+            if group.required:
+                for action in group._group_actions:
+                    if action in seen_non_default_actions:
+                        break
+
+                # if no actions were used, report the error
+                else:
+                    names = [_get_action_name(action)
+                             for action in group._group_actions
+                             if action.help is not SUPPRESS]
+                    msg = _('one of the arguments %s is required')
+                    self.error(msg % ' '.join(names))
+
+        # return the updated namespace and the extra arguments
+        return namespace, extras
+
+    def _read_args_from_files(self, arg_strings):
+        # expand arguments referencing files
+        new_arg_strings = []
+        for arg_string in arg_strings:
+
+            # for regular arguments, just add them back into the list
+            if arg_string[0] not in self.fromfile_prefix_chars:
+                new_arg_strings.append(arg_string)
+
+            # replace arguments referencing files with the file content
+            else:
+                try:
+                    args_file = open(arg_string[1:])
+                    try:
+                        arg_strings = []
+                        for arg_line in args_file.read().splitlines():
+                            for arg in self.convert_arg_line_to_args(arg_line):
+                                arg_strings.append(arg)
+                        arg_strings = self._read_args_from_files(arg_strings)
+                        new_arg_strings.extend(arg_strings)
+                    finally:
+                        args_file.close()
+                except IOError:
+                    err = _sys.exc_info()[1]
+                    self.error(str(err))
+
+        # return the modified argument list
+        return new_arg_strings
+
+    def convert_arg_line_to_args(self, arg_line):
+        return [arg_line]
+
+    def _match_argument(self, action, arg_strings_pattern):
+        # match the pattern for this action to the arg strings
+        nargs_pattern = self._get_nargs_pattern(action)
+        match = _re.match(nargs_pattern, arg_strings_pattern)
+
+        # raise an exception if we weren't able to find a match
+        if match is None:
+            nargs_errors = {
+                None: _('expected one argument'),
+                OPTIONAL: _('expected at most one argument'),
+                ONE_OR_MORE: _('expected at least one argument'),
+            }
+            default = _('expected %s argument(s)') % action.nargs
+            msg = nargs_errors.get(action.nargs, default)
+            raise ArgumentError(action, msg)
+
+        # return the number of arguments matched
+        return len(match.group(1))
+
+    def _match_arguments_partial(self, actions, arg_strings_pattern):
+        # progressively shorten the actions list by slicing off the
+        # final actions until we find a match
+        result = []
+        for i in range(len(actions), 0, -1):
+            actions_slice = actions[:i]
+            pattern = ''.join([self._get_nargs_pattern(action)
+                               for action in actions_slice])
+            match = _re.match(pattern, arg_strings_pattern)
+            if match is not None:
+                result.extend([len(string) for string in match.groups()])
+                break
+
+        # return the list of arg string counts
+        return result
+
+    def _parse_optional(self, arg_string):
+        # if it's an empty string, it was meant to be a positional
+        if not arg_string:
+            return None
+
+        # if it doesn't start with a prefix, it was meant to be positional
+        if not arg_string[0] in self.prefix_chars:
+            return None
+
+        # if the option string is present in the parser, return the action
+        if arg_string in self._option_string_actions:
+            action = self._option_string_actions[arg_string]
+            return action, arg_string, None
+
+        # if it's just a single character, it was meant to be positional
+        if len(arg_string) == 1:
+            return None
+
+        # if the option string before the "=" is present, return the action
+        if '=' in arg_string:
+            option_string, explicit_arg = arg_string.split('=', 1)
+            if option_string in self._option_string_actions:
+                action = self._option_string_actions[option_string]
+                return action, option_string, explicit_arg
+
+        # search through all possible prefixes of the option string
+        # and all actions in the parser for possible interpretations
+        option_tuples = self._get_option_tuples(arg_string)
+
+        # if multiple actions match, the option string was ambiguous
+        if len(option_tuples) > 1:
+            options = ', '.join([option_string
+                for action, option_string, explicit_arg in option_tuples])
+            tup = arg_string, options
+            self.error(_('ambiguous option: %s could match %s') % tup)
+
+        # if exactly one action matched, this segmentation is good,
+        # so return the parsed action
+        elif len(option_tuples) == 1:
+            option_tuple, = option_tuples
+            return option_tuple
+
+        # if it was not found as an option, but it looks like a negative
+        # number, it was meant to be positional
+        # unless there are negative-number-like options
+        if self._negative_number_matcher.match(arg_string):
+            if not self._has_negative_number_optionals:
+                return None
+
+        # if it contains a space, it was meant to be a positional
+        if ' ' in arg_string:
+            return None
+
+        # it was meant to be an optional but there is no such option
+        # in this parser (though it might be a valid option in a subparser)
+        return None, arg_string, None
+
+    def _get_option_tuples(self, option_string):
+        result = []
+
+        # option strings starting with two prefix characters are only
+        # split at the '='
+        chars = self.prefix_chars
+        if option_string[0] in chars and option_string[1] in chars:
+            if '=' in option_string:
+                option_prefix, explicit_arg = option_string.split('=', 1)
+            else:
+                option_prefix = option_string
+                explicit_arg = None
+            for option_string in self._option_string_actions:
+                if option_string.startswith(option_prefix):
+                    action = self._option_string_actions[option_string]
+                    tup = action, option_string, explicit_arg
+                    result.append(tup)
+
+        # single character options can be concatenated with their arguments
+        # but multiple character options always have to have their argument
+        # separate
+        elif option_string[0] in chars and option_string[1] not in chars:
+            option_prefix = option_string
+            explicit_arg = None
+            short_option_prefix = option_string[:2]
+            short_explicit_arg = option_string[2:]
+
+            for option_string in self._option_string_actions:
+                if option_string == short_option_prefix:
+                    action = self._option_string_actions[option_string]
+                    tup = action, option_string, short_explicit_arg
+                    result.append(tup)
+                elif option_string.startswith(option_prefix):
+                    action = self._option_string_actions[option_string]
+                    tup = action, option_string, explicit_arg
+                    result.append(tup)
+
+        # shouldn't ever get here
+        else:
+            self.error(_('unexpected option string: %s') % option_string)
+
+        # return the collected option tuples
+        return result
+
+    def _get_nargs_pattern(self, action):
+        # in all examples below, we have to allow for '--' args
+        # which are represented as '-' in the pattern
+        nargs = action.nargs
+
+        # the default (None) is assumed to be a single argument
+        if nargs is None:
+            nargs_pattern = '(-*A-*)'
+
+        # allow zero or one arguments
+        elif nargs == OPTIONAL:
+            nargs_pattern = '(-*A?-*)'
+
+        # allow zero or more arguments
+        elif nargs == ZERO_OR_MORE:
+            nargs_pattern = '(-*[A-]*)'
+
+        # allow one or more arguments
+        elif nargs == ONE_OR_MORE:
+            nargs_pattern = '(-*A[A-]*)'
+
+        # allow any number of options or arguments
+        elif nargs == REMAINDER:
+            nargs_pattern = '([-AO]*)'
+
+        # allow one argument followed by any number of options or arguments
+        elif nargs == PARSER:
+            nargs_pattern = '(-*A[-AO]*)'
+
+        # all others should be integers
+        else:
+            nargs_pattern = '(-*%s-*)' % '-*'.join('A' * nargs)
+
+        # if this is an optional action, -- is not allowed
+        if action.option_strings:
+            nargs_pattern = nargs_pattern.replace('-*', '')
+            nargs_pattern = nargs_pattern.replace('-', '')
+
+        # return the pattern
+        return nargs_pattern
+
+    # ========================
+    # Value conversion methods
+    # ========================
+    def _get_values(self, action, arg_strings):
+        # for everything but PARSER args, strip out '--'
+        if action.nargs not in [PARSER, REMAINDER]:
+            arg_strings = [s for s in arg_strings if s != '--']
+
+        # optional argument produces a default when not present
+        if not arg_strings and action.nargs == OPTIONAL:
+            if action.option_strings:
+                value = action.const
+            else:
+                value = action.default
+            if isinstance(value, basestring):
+                value = self._get_value(action, value)
+                self._check_value(action, value)
+
+        # when nargs='*' on a positional, if there were no command-line
+        # args, use the default if it is anything other than None
+        elif (not arg_strings and action.nargs == ZERO_OR_MORE and
+              not action.option_strings):
+            if action.default is not None:
+                value = action.default
+            else:
+                value = arg_strings
+            self._check_value(action, value)
+
+        # single argument or optional argument produces a single value
+        elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:
+            arg_string, = arg_strings
+            value = self._get_value(action, arg_string)
+            self._check_value(action, value)
+
+        # REMAINDER arguments convert all values, checking none
+        elif action.nargs == REMAINDER:
+            value = [self._get_value(action, v) for v in arg_strings]
+
+        # PARSER arguments convert all values, but check only the first
+        elif action.nargs == PARSER:
+            value = [self._get_value(action, v) for v in arg_strings]
+            self._check_value(action, value[0])
+
+        # all other types of nargs produce a list
+        else:
+            value = [self._get_value(action, v) for v in arg_strings]
+            for v in value:
+                self._check_value(action, v)
+
+        # return the converted value
+        return value
+
+    def _get_value(self, action, arg_string):
+        type_func = self._registry_get('type', action.type, action.type)
+        if not _callable(type_func):
+            msg = _('%r is not callable')
+            raise ArgumentError(action, msg % type_func)
+
+        # convert the value to the appropriate type
+        try:
+            result = type_func(arg_string)
+
+        # ArgumentTypeErrors indicate errors
+        except ArgumentTypeError:
+            name = getattr(action.type, '__name__', repr(action.type))
+            msg = str(_sys.exc_info()[1])
+            raise ArgumentError(action, msg)
+
+        # TypeErrors or ValueErrors also indicate errors
+        except (TypeError, ValueError):
+            name = getattr(action.type, '__name__', repr(action.type))
+            msg = _('invalid %s value: %r')
+            raise ArgumentError(action, msg % (name, arg_string))
+
+        # return the converted value
+        return result
+
+    def _check_value(self, action, value):
+        # converted value must be one of the choices (if specified)
+        if action.choices is not None and value not in action.choices:
+            tup = value, ', '.join(map(repr, action.choices))
+            msg = _('invalid choice: %r (choose from %s)') % tup
+            raise ArgumentError(action, msg)
+
+    # =======================
+    # Help-formatting methods
+    # =======================
+    def format_usage(self):
+        formatter = self._get_formatter()
+        formatter.add_usage(self.usage, self._actions,
+                            self._mutually_exclusive_groups)
+        return formatter.format_help()
+
+    def format_help(self):
+        formatter = self._get_formatter()
+
+        # usage
+        formatter.add_usage(self.usage, self._actions,
+                            self._mutually_exclusive_groups)
+
+        # description
+        formatter.add_text(self.description)
+
+        # positionals, optionals and user-defined groups
+        for action_group in self._action_groups:
+            formatter.start_section(action_group.title)
+            formatter.add_text(action_group.description)
+            formatter.add_arguments(action_group._group_actions)
+            formatter.end_section()
+
+        # epilog
+        formatter.add_text(self.epilog)
+
+        # determine help from format above
+        return formatter.format_help()
+
+    def format_version(self):
+        import warnings
+        warnings.warn(
+            'The format_version method is deprecated -- the "version" '
+            'argument to ArgumentParser is no longer supported.',
+            DeprecationWarning)
+        formatter = self._get_formatter()
+        formatter.add_text(self.version)
+        return formatter.format_help()
+
+    def _get_formatter(self):
+        return self.formatter_class(prog=self.prog)
+
+    # =====================
+    # Help-printing methods
+    # =====================
+    def print_usage(self, file=None):
+        if file is None:
+            file = _sys.stdout
+        self._print_message(self.format_usage(), file)
+
+    def print_help(self, file=None):
+        if file is None:
+            file = _sys.stdout
+        self._print_message(self.format_help(), file)
+
+    def print_version(self, file=None):
+        import warnings
+        warnings.warn(
+            'The print_version method is deprecated -- the "version" '
+            'argument to ArgumentParser is no longer supported.',
+            DeprecationWarning)
+        self._print_message(self.format_version(), file)
+
+    def _print_message(self, message, file=None):
+        if message:
+            if file is None:
+                file = _sys.stderr
+            file.write(message)
+
+    # ===============
+    # Exiting methods
+    # ===============
+    def exit(self, status=0, message=None):
+        if message:
+            self._print_message(message, _sys.stderr)
+        _sys.exit(status)
+
+    def error(self, message):
+        """error(message: string)
+
+        Prints a usage message incorporating the message to stderr and
+        exits.
+
+        If you override this in a subclass, it should not return -- it
+        should either exit or raise an exception.
+        """
+        self.print_usage(_sys.stderr)
+        self.exit(2, _('%s: error: %s\n') % (self.prog, message))
-- 
2.13.5

Add the scripts/ directory to sys.path so Python 2.6 will be able to
import argparse.

Cc: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Acked-by: Fam Zheng <famz@redhat.com>
Message-id: 20170825155732.15665-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 tests/migration/guestperf/shell.py | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tests/migration/guestperf/shell.py b/tests/migration/guestperf/shell.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/migration/guestperf/shell.py
+++ b/tests/migration/guestperf/shell.py
@@ -XXX,XX +XXX,XX @@
 #
 
 
-import argparse
-import fnmatch
 import os
 import os.path
-import platform
 import sys
+sys.path.append(os.path.join(os.path.dirname(__file__),
+                             '..', '..', '..', 'scripts'))
+import argparse
+import fnmatch
+import platform
 
 from guestperf.hardware import Hardware
 from guestperf.engine import Engine
-- 
2.13.5

From: Fred Rolland <rollandf@gmail.com>

Update doc with the usage of UUID for initiator name.

Related-To: https://bugzilla.redhat.com/1006468
Signed-off-by: Fred Rolland <frolland@redhat.com>
Message-id: 20170823084830.30500-1-frolland@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qemu-doc.texi | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/qemu-doc.texi b/qemu-doc.texi
index XXXXXXX..XXXXXXX 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -XXX,XX +XXX,XX @@ in a configuration file provided via '-readconfig' or directly on the
 command line.
 
 If the initiator-name is not specified qemu will use a default name
-of 'iqn.2008-11.org.linux-kvm[:<name>'] where <name> is the name of the
+of 'iqn.2008-11.org.linux-kvm[:<uuid>'] where <uuid> is the UUID of the
+virtual machine. If the UUID is not specified qemu will use
+'iqn.2008-11.org.linux-kvm[:<name>'] where <name> is the name of the
 virtual machine.
 
-
 @example
 Setting a specific initiator name to use when logging in to the target
 -iscsi initiator-name=iqn.qemu.test:my-initiator
-- 
2.13.5

Most qcow2 files are uncompressed so it is wasteful to allocate (32 + 1)
* cluster_size + 512 bytes upfront.  Allocate s->cluster_cache and
s->cluster_data when the first read operation is performance on a
compressed cluster.

The buffers are freed in .bdrv_close().  .bdrv_open() no longer has any
code paths that can allocate these buffers, so remove the free functions
in the error code path.

This patch can result in significant memory savings when many qcow2
disks are attached or backing file chains are long:

Before 12.81% (1,023,193,088B)
After   5.36% (393,893,888B)

Reported-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20170821135530.32344-1-stefanha@redhat.com
Cc: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/qcow2-cluster.c | 17 +++++++++++++++++
 block/qcow2.c         | 12 ------------
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -XXX,XX +XXX,XX @@ int qcow2_decompress_cluster(BlockDriverState *bs, uint64_t cluster_offset)
         nb_csectors = ((cluster_offset >> s->csize_shift) & s->csize_mask) + 1;
         sector_offset = coffset & 511;
         csize = nb_csectors * 512 - sector_offset;
+
+        /* Allocate buffers on first decompress operation, most images are
+         * uncompressed and the memory overhead can be avoided.  The buffers
+         * are freed in .bdrv_close().
+         */
+        if (!s->cluster_data) {
+            /* one more sector for decompressed data alignment */
+            s->cluster_data = qemu_try_blockalign(bs->file->bs,
+                    QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size + 512);
+            if (!s->cluster_data) {
+                return -ENOMEM;
+            }
+        }
+        if (!s->cluster_cache) {
+            s->cluster_cache = g_malloc(s->cluster_size);
+        }
+
         BLKDBG_EVENT(bs->file, BLKDBG_READ_COMPRESSED);
         ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data,
                         nb_csectors);
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags,
         goto fail;
     }
 
-    s->cluster_cache = g_malloc(s->cluster_size);
-    /* one more sector for decompressed data alignment */
-    s->cluster_data = qemu_try_blockalign(bs->file->bs, QCOW_MAX_CRYPT_CLUSTERS
-                                                    * s->cluster_size + 512);
-    if (s->cluster_data == NULL) {
-        error_setg(errp, "Could not allocate temporary cluster buffer");
-        ret = -ENOMEM;
-        goto fail;
-    }
-
     s->cluster_cache_offset = -1;
     s->flags = flags;
 
@@ -XXX,XX +XXX,XX @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags,
     if (s->refcount_block_cache) {
         qcow2_cache_destroy(bs, s->refcount_block_cache);
     }
-    g_free(s->cluster_cache);
-    qemu_vfree(s->cluster_data);
     qcrypto_block_free(s->crypto);
     qapi_free_QCryptoBlockOpenOptions(s->crypto_opts);
     return ret;
-- 
2.13.5

The following changes since commit c1eb2ddf0f8075faddc5f7c3d39feae3e8e9d6b4:

Update version for v8.0.0 release (2023-04-19 17:27:13 +0100)

are available in the Git repository at:

https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 36e5e9b22abe56aa00ca067851555ad8127a7966:

tracing: install trace events file only if necessary (2023-04-20 07:39:43 -0400)

----------------------------------------------------------------
Pull request

Sam Li's zoned storage work and fixes I collected during the 8.0 freeze.

----------------------------------------------------------------

Carlos Santos (1):
  tracing: install trace events file only if necessary

Philippe Mathieu-Daudé (1):
  block/dmg: Declare a type definition for DMG uncompress function

Sam Li (17):
  block/block-common: add zoned device structs
  block/file-posix: introduce helper functions for sysfs attributes
  block/block-backend: add block layer APIs resembling Linux
    ZonedBlockDevice ioctls
  block/raw-format: add zone operations to pass through requests
  block: add zoned BlockDriver check to block layer
  iotests: test new zone operations
  block: add some trace events for new block layer APIs
  docs/zoned-storage: add zoned device documentation
  file-posix: add tracking of the zone write pointers
  block: introduce zone append write for zoned devices
  qemu-iotests: test zone append operation
  block: add some trace events for zone append
  include: update virtio_blk headers to v6.3-rc1
  virtio-blk: add zoned storage emulation for zoned devices
  block: add accounting for zone append operation
  virtio-blk: add some trace events for zoned emulation
  docs/zoned-storage:add zoned emulation use case

Thomas De Schampheleire (1):
  tracetool: use relative paths for '#line' preprocessor directives

-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-2-faithilikerun@gmail.com
[Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
<philmd@linaro.org>.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/block/block-common.h | 43 ++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/include/block/block-common.h b/include/block/block-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block-common.h
+++ b/include/block/block-common.h
@@ -XXX,XX +XXX,XX @@ typedef struct BlockDriver BlockDriver;
 typedef struct BdrvChild BdrvChild;
 typedef struct BdrvChildClass BdrvChildClass;
 
+typedef enum BlockZoneOp {
+    BLK_ZO_OPEN,
+    BLK_ZO_CLOSE,
+    BLK_ZO_FINISH,
+    BLK_ZO_RESET,
+} BlockZoneOp;
+
+typedef enum BlockZoneModel {
+    BLK_Z_NONE = 0x0, /* Regular block device */
+    BLK_Z_HM = 0x1, /* Host-managed zoned block device */
+    BLK_Z_HA = 0x2, /* Host-aware zoned block device */
+} BlockZoneModel;
+
+typedef enum BlockZoneState {
+    BLK_ZS_NOT_WP = 0x0,
+    BLK_ZS_EMPTY = 0x1,
+    BLK_ZS_IOPEN = 0x2,
+    BLK_ZS_EOPEN = 0x3,
+    BLK_ZS_CLOSED = 0x4,
+    BLK_ZS_RDONLY = 0xD,
+    BLK_ZS_FULL = 0xE,
+    BLK_ZS_OFFLINE = 0xF,
+} BlockZoneState;
+
+typedef enum BlockZoneType {
+    BLK_ZT_CONV = 0x1, /* Conventional random writes supported */
+    BLK_ZT_SWR = 0x2, /* Sequential writes required */
+    BLK_ZT_SWP = 0x3, /* Sequential writes preferred */
+} BlockZoneType;
+
+/*
+ * Zone descriptor data structure.
+ * Provides information on a zone with all position and size values in bytes.
+ */
+typedef struct BlockZoneDescriptor {
+    uint64_t start;
+    uint64_t length;
+    uint64_t cap;
+    uint64_t wp;
+    BlockZoneType type;
+    BlockZoneState state;
+} BlockZoneDescriptor;
+
 typedef struct BlockDriverInfo {
     /* in bytes, 0 if irrelevant */
     int cluster_size;
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Use get_sysfs_str_val() to get the string value of device
zoned model. Then get_sysfs_zoned_model() can convert it to
BlockZoneModel type of QEMU.

Use get_sysfs_long_val() to get the long value of zoned device
information.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-3-faithilikerun@gmail.com
[Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
<philmd@linaro.org>.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/block/block_int-common.h |   3 +
 block/file-posix.c               | 130 ++++++++++++++++++++++---------
 2 files changed, 95 insertions(+), 38 deletions(-)

diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ typedef struct BlockLimits {
      * an explicit monitor command to load the disk inside the guest).
      */
     bool has_variable_length;
+
+    /* device zone model */
+    BlockZoneModel zoned;
 } BlockLimits;
 
 typedef struct BdrvOpBlocker BdrvOpBlocker;
diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int hdev_get_max_hw_transfer(int fd, struct stat *st)
 #endif
 }
 
-static int hdev_get_max_segments(int fd, struct stat *st)
+/*
+ * Get a sysfs attribute value as character string.
+ */
+static int get_sysfs_str_val(struct stat *st, const char *attribute,
+                             char **val) {
+#ifdef CONFIG_LINUX
+    g_autofree char *sysfspath = NULL;
+    int ret;
+    size_t len;
+
+    if (!S_ISBLK(st->st_mode)) {
+        return -ENOTSUP;
+    }
+
+    sysfspath = g_strdup_printf("/sys/dev/block/%u:%u/queue/%s",
+                                major(st->st_rdev), minor(st->st_rdev),
+                                attribute);
+    ret = g_file_get_contents(sysfspath, val, &len, NULL);
+    if (ret == -1) {
+        return -ENOENT;
+    }
+
+    /* The file is ended with '\n' */
+    char *p;
+    p = *val;
+    if (*(p + len - 1) == '\n') {
+        *(p + len - 1) = '\0';
+    }
+    return ret;
+#else
+    return -ENOTSUP;
+#endif
+}
+
+static int get_sysfs_zoned_model(struct stat *st, BlockZoneModel *zoned)
+{
+    g_autofree char *val = NULL;
+    int ret;
+
+    ret = get_sysfs_str_val(st, "zoned", &val);
+    if (ret < 0) {
+        return ret;
+    }
+
+    if (strcmp(val, "host-managed") == 0) {
+        *zoned = BLK_Z_HM;
+    } else if (strcmp(val, "host-aware") == 0) {
+        *zoned = BLK_Z_HA;
+    } else if (strcmp(val, "none") == 0) {
+        *zoned = BLK_Z_NONE;
+    } else {
+        return -ENOTSUP;
+    }
+    return 0;
+}
+
+/*
+ * Get a sysfs attribute value as a long integer.
+ */
+static long get_sysfs_long_val(struct stat *st, const char *attribute)
 {
 #ifdef CONFIG_LINUX
-    char buf[32];
+    g_autofree char *str = NULL;
     const char *end;
-    char *sysfspath = NULL;
+    long val;
+    int ret;
+
+    ret = get_sysfs_str_val(st, attribute, &str);
+    if (ret < 0) {
+        return ret;
+    }
+
+    /* The file is ended with '\n', pass 'end' to accept that. */
+    ret = qemu_strtol(str, &end, 10, &val);
+    if (ret == 0 && end && *end == '\0') {
+        ret = val;
+    }
+    return ret;
+#else
+    return -ENOTSUP;
+#endif
+}
+
+static int hdev_get_max_segments(int fd, struct stat *st)
+{
+#ifdef CONFIG_LINUX
     int ret;
-    int sysfd = -1;
-    long max_segments;
 
     if (S_ISCHR(st->st_mode)) {
         if (ioctl(fd, SG_GET_SG_TABLESIZE, &ret) == 0) {
@@ -XXX,XX +XXX,XX @@ static int hdev_get_max_segments(int fd, struct stat *st)
         }
         return -ENOTSUP;
     }
-
-    if (!S_ISBLK(st->st_mode)) {
-        return -ENOTSUP;
-    }
-
-    sysfspath = g_strdup_printf("/sys/dev/block/%u:%u/queue/max_segments",
-                                major(st->st_rdev), minor(st->st_rdev));
-    sysfd = open(sysfspath, O_RDONLY);
-    if (sysfd == -1) {
-        ret = -errno;
-        goto out;
-    }
-    ret = RETRY_ON_EINTR(read(sysfd, buf, sizeof(buf) - 1));
-    if (ret < 0) {
-        ret = -errno;
-        goto out;
-    } else if (ret == 0) {
-        ret = -EIO;
-        goto out;
-    }
-    buf[ret] = 0;
-    /* The file is ended with '\n', pass 'end' to accept that. */
-    ret = qemu_strtol(buf, &end, 10, &max_segments);
-    if (ret == 0 && end && *end == '\n') {
-        ret = max_segments;
-    }
-
-out:
-    if (sysfd != -1) {
-        close(sysfd);
-    }
-    g_free(sysfspath);
-    return ret;
+    return get_sysfs_long_val(st, "max_segments");
 #else
     return -ENOTSUP;
 #endif
@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     BDRVRawState *s = bs->opaque;
     struct stat st;
+    int ret;
+    BlockZoneModel zoned;
 
     s->needs_alignment = raw_needs_alignment(bs);
     raw_probe_alignment(bs, s->fd, errp);
@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
             bs->bl.max_hw_iov = ret;
         }
     }
+
+    ret = get_sysfs_zoned_model(&st, &zoned);
+    if (ret < 0) {
+        zoned = BLK_Z_NONE;
+    }
+    bs->bl.zoned = zoned;
 }
 
 static int check_for_dasd(int fd)
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Add zoned device option to host_device BlockDriver. It will be presented only
for zoned host block devices. By adding zone management operations to the
host_block_device BlockDriver, users can use the new block layer APIs
including Report Zone and four zone management operations
(open, close, finish, reset, reset_all).

Qemu-io uses the new APIs to perform zoned storage commands of the device:
zone_report(zrp), zone_open(zo), zone_close(zc), zone_reset(zrs),
zone_finish(zf).

For example, to test zone_report, use following command:
$ ./build/qemu-io --image-opts -n driver=host_device, filename=/dev/nullb0
-c "zrp offset nr_zones"

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-4-faithilikerun@gmail.com
[Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
<philmd@linaro.org> and remove spurious ret = -errno in
raw_co_zone_mgmt().
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 meson.build                       |   4 +
 include/block/block-io.h          |   9 +
 include/block/block_int-common.h  |  21 ++
 include/block/raw-aio.h           |   6 +-
 include/sysemu/block-backend-io.h |  18 ++
 block/block-backend.c             | 133 +++++++++++++
 block/file-posix.c                | 306 +++++++++++++++++++++++++++++-
 block/io.c                        |  41 ++++
 qemu-io-cmds.c                    | 149 +++++++++++++++
 9 files changed, 684 insertions(+), 3 deletions(-)

diff --git a/meson.build b/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/meson.build
+++ b/meson.build
@@ -XXX,XX +XXX,XX @@ config_host_data.set('CONFIG_REPLICATION', get_option('replication').allowed())
 # has_header
 config_host_data.set('CONFIG_EPOLL', cc.has_header('sys/epoll.h'))
 config_host_data.set('CONFIG_LINUX_MAGIC_H', cc.has_header('linux/magic.h'))
+config_host_data.set('CONFIG_BLKZONED', cc.has_header('linux/blkzoned.h'))
 config_host_data.set('CONFIG_VALGRIND_H', cc.has_header('valgrind/valgrind.h'))
 config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
 config_host_data.set('HAVE_DRM_H', cc.has_header('libdrm/drm.h'))
@@ -XXX,XX +XXX,XX @@ config_host_data.set('HAVE_SIGEV_NOTIFY_THREAD_ID',
 config_host_data.set('HAVE_STRUCT_STAT_ST_ATIM',
                      cc.has_member('struct stat', 'st_atim',
                                    prefix: '#include <sys/stat.h>'))
+config_host_data.set('HAVE_BLK_ZONE_REP_CAPACITY',
+                     cc.has_member('struct blk_zone', 'capacity',
+                                   prefix: '#include <linux/blkzoned.h>'))
 
 # has_type
 config_host_data.set('CONFIG_IOVEC',
diff --git a/include/block/block-io.h b/include/block/block-io.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block-io.h
+++ b/include/block/block-io.h
@@ -XXX,XX +XXX,XX @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_flush(BlockDriverState *bs);
 int coroutine_fn GRAPH_RDLOCK bdrv_co_pdiscard(BdrvChild *child, int64_t offset,
                                                int64_t bytes);
 
+/* Report zone information of zone block device. */
+int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_report(BlockDriverState *bs,
+                                                  int64_t offset,
+                                                  unsigned int *nr_zones,
+                                                  BlockZoneDescriptor *zones);
+int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_mgmt(BlockDriverState *bs,
+                                                BlockZoneOp op,
+                                                int64_t offset, int64_t len);
+
 bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
 int bdrv_block_status(BlockDriverState *bs, int64_t offset,
                       int64_t bytes, int64_t *pnum, int64_t *map,
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
     int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_load_vmstate)(
         BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
 
+    int coroutine_fn (*bdrv_co_zone_report)(BlockDriverState *bs,
+            int64_t offset, unsigned int *nr_zones,
+            BlockZoneDescriptor *zones);
+    int coroutine_fn (*bdrv_co_zone_mgmt)(BlockDriverState *bs, BlockZoneOp op,
+            int64_t offset, int64_t len);
+
     /* removable device specific */
     bool coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_is_inserted)(
         BlockDriverState *bs);
@@ -XXX,XX +XXX,XX @@ typedef struct BlockLimits {
 
     /* device zone model */
     BlockZoneModel zoned;
+
+    /* zone size expressed in bytes */
+    uint32_t zone_size;
+
+    /* total number of zones */
+    uint32_t nr_zones;
+
+    /* maximum sectors of a zone append write operation */
+    int64_t max_append_sectors;
+
+    /* maximum number of open zones */
+    int64_t max_open_zones;
+
+    /* maximum number of active zones */
+    int64_t max_active_zones;
 } BlockLimits;
 
 typedef struct BdrvOpBlocker BdrvOpBlocker;
diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/raw-aio.h
+++ b/include/block/raw-aio.h
@@ -XXX,XX +XXX,XX @@
 #define QEMU_AIO_WRITE_ZEROES 0x0020
 #define QEMU_AIO_COPY_RANGE   0x0040
 #define QEMU_AIO_TRUNCATE     0x0080
+#define QEMU_AIO_ZONE_REPORT  0x0100
+#define QEMU_AIO_ZONE_MGMT    0x0200
 #define QEMU_AIO_TYPE_MASK \
         (QEMU_AIO_READ | \
          QEMU_AIO_WRITE | \
@@ -XXX,XX +XXX,XX @@
          QEMU_AIO_DISCARD | \
          QEMU_AIO_WRITE_ZEROES | \
          QEMU_AIO_COPY_RANGE | \
-         QEMU_AIO_TRUNCATE)
+         QEMU_AIO_TRUNCATE | \
+         QEMU_AIO_ZONE_REPORT | \
+         QEMU_AIO_ZONE_MGMT)
 
 /* AIO flags */
 #define QEMU_AIO_MISALIGNED   0x1000
diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backend-io.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/block-backend-io.h
+++ b/include/sysemu/block-backend-io.h
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_pwritev(BlockBackend *blk, int64_t offset,
                             BlockCompletionFunc *cb, void *opaque);
 BlockAIOCB *blk_aio_flush(BlockBackend *blk,
                           BlockCompletionFunc *cb, void *opaque);
+BlockAIOCB *blk_aio_zone_report(BlockBackend *blk, int64_t offset,
+                                unsigned int *nr_zones,
+                                BlockZoneDescriptor *zones,
+                                BlockCompletionFunc *cb, void *opaque);
+BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
+                              int64_t offset, int64_t len,
+                              BlockCompletionFunc *cb, void *opaque);
 BlockAIOCB *blk_aio_pdiscard(BlockBackend *blk, int64_t offset, int64_t bytes,
                              BlockCompletionFunc *cb, void *opaque);
 void blk_aio_cancel_async(BlockAIOCB *acb);
@@ -XXX,XX +XXX,XX @@ int co_wrapper_mixed blk_pwrite_zeroes(BlockBackend *blk, int64_t offset,
 int coroutine_fn blk_co_pwrite_zeroes(BlockBackend *blk, int64_t offset,
                                       int64_t bytes, BdrvRequestFlags flags);
 
+int coroutine_fn blk_co_zone_report(BlockBackend *blk, int64_t offset,
+                                    unsigned int *nr_zones,
+                                    BlockZoneDescriptor *zones);
+int co_wrapper_mixed blk_zone_report(BlockBackend *blk, int64_t offset,
+                                         unsigned int *nr_zones,
+                                         BlockZoneDescriptor *zones);
+int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
+                                  int64_t offset, int64_t len);
+int co_wrapper_mixed blk_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
+                                       int64_t offset, int64_t len);
+
 int co_wrapper_mixed blk_pdiscard(BlockBackend *blk, int64_t offset,
                                   int64_t bytes);
 int coroutine_fn blk_co_pdiscard(BlockBackend *blk, int64_t offset,
diff --git a/block/block-backend.c b/block/block-backend.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_flush(BlockBackend *blk)
     return ret;
 }
 
+static void coroutine_fn blk_aio_zone_report_entry(void *opaque)
+{
+    BlkAioEmAIOCB *acb = opaque;
+    BlkRwCo *rwco = &acb->rwco;
+
+    rwco->ret = blk_co_zone_report(rwco->blk, rwco->offset,
+                                   (unsigned int*)acb->bytes,rwco->iobuf);
+    blk_aio_complete(acb);
+}
+
+BlockAIOCB *blk_aio_zone_report(BlockBackend *blk, int64_t offset,
+                                unsigned int *nr_zones,
+                                BlockZoneDescriptor  *zones,
+                                BlockCompletionFunc *cb, void *opaque)
+{
+    BlkAioEmAIOCB *acb;
+    Coroutine *co;
+    IO_CODE();
+
+    blk_inc_in_flight(blk);
+    acb = blk_aio_get(&blk_aio_em_aiocb_info, blk, cb, opaque);
+    acb->rwco = (BlkRwCo) {
+        .blk    = blk,
+        .offset = offset,
+        .iobuf  = zones,
+        .ret    = NOT_DONE,
+    };
+    acb->bytes = (int64_t)nr_zones,
+    acb->has_returned = false;
+
+    co = qemu_coroutine_create(blk_aio_zone_report_entry, acb);
+    aio_co_enter(blk_get_aio_context(blk), co);
+
+    acb->has_returned = true;
+    if (acb->rwco.ret != NOT_DONE) {
+        replay_bh_schedule_oneshot_event(blk_get_aio_context(blk),
+                                         blk_aio_complete_bh, acb);
+    }
+
+    return &acb->common;
+}
+
+static void coroutine_fn blk_aio_zone_mgmt_entry(void *opaque)
+{
+    BlkAioEmAIOCB *acb = opaque;
+    BlkRwCo *rwco = &acb->rwco;
+
+    rwco->ret = blk_co_zone_mgmt(rwco->blk, (BlockZoneOp)rwco->iobuf,
+                                 rwco->offset, acb->bytes);
+    blk_aio_complete(acb);
+}
+
+BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
+                              int64_t offset, int64_t len,
+                              BlockCompletionFunc *cb, void *opaque) {
+    BlkAioEmAIOCB *acb;
+    Coroutine *co;
+    IO_CODE();
+
+    blk_inc_in_flight(blk);
+    acb = blk_aio_get(&blk_aio_em_aiocb_info, blk, cb, opaque);
+    acb->rwco = (BlkRwCo) {
+        .blk    = blk,
+        .offset = offset,
+        .iobuf  = (void *)op,
+        .ret    = NOT_DONE,
+    };
+    acb->bytes = len;
+    acb->has_returned = false;
+
+    co = qemu_coroutine_create(blk_aio_zone_mgmt_entry, acb);
+    aio_co_enter(blk_get_aio_context(blk), co);
+
+    acb->has_returned = true;
+    if (acb->rwco.ret != NOT_DONE) {
+        replay_bh_schedule_oneshot_event(blk_get_aio_context(blk),
+                                         blk_aio_complete_bh, acb);
+    }
+
+    return &acb->common;
+}
+
+/*
+ * Send a zone_report command.
+ * offset is a byte offset from the start of the device. No alignment
+ * required for offset.
+ * nr_zones represents IN maximum and OUT actual.
+ */
+int coroutine_fn blk_co_zone_report(BlockBackend *blk, int64_t offset,
+                                    unsigned int *nr_zones,
+                                    BlockZoneDescriptor *zones)
+{
+    int ret;
+    IO_CODE();
+
+    blk_inc_in_flight(blk); /* increase before waiting */
+    blk_wait_while_drained(blk);
+    if (!blk_is_available(blk)) {
+        blk_dec_in_flight(blk);
+        return -ENOMEDIUM;
+    }
+    ret = bdrv_co_zone_report(blk_bs(blk), offset, nr_zones, zones);
+    blk_dec_in_flight(blk);
+    return ret;
+}
+
+/*
+ * Send a zone_management command.
+ * op is the zone operation;
+ * offset is the byte offset from the start of the zoned device;
+ * len is the maximum number of bytes the command should operate on. It
+ * should be aligned with the device zone size.
+ */
+int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
+        int64_t offset, int64_t len)
+{
+    int ret;
+    IO_CODE();
+
+    blk_inc_in_flight(blk);
+    blk_wait_while_drained(blk);
+
+    ret = blk_check_byte_request(blk, offset, len);
+    if (ret < 0) {
+        blk_dec_in_flight(blk);
+        return ret;
+    }
+
+    ret = bdrv_co_zone_mgmt(blk_bs(blk), op, offset, len);
+    blk_dec_in_flight(blk);
+    return ret;
+}
+
 void blk_drain(BlockBackend *blk)
 {
     BlockDriverState *bs = blk_bs(blk);
diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@
 #include <sys/param.h>
 #include <sys/syscall.h>
 #include <sys/vfs.h>
+#if defined(CONFIG_BLKZONED)
+#include <linux/blkzoned.h>
+#endif
 #include <linux/cdrom.h>
 #include <linux/fd.h>
 #include <linux/fs.h>
@@ -XXX,XX +XXX,XX @@ typedef struct RawPosixAIOData {
             PreallocMode prealloc;
             Error **errp;
         } truncate;
+        struct {
+            unsigned int *nr_zones;
+            BlockZoneDescriptor *zones;
+        } zone_report;
+        struct {
+            unsigned long op;
+        } zone_mgmt;
     };
 } RawPosixAIOData;
 
@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
         zoned = BLK_Z_NONE;
     }
     bs->bl.zoned = zoned;
+    if (zoned != BLK_Z_NONE) {
+        /*
+         * The zoned device must at least have zone size and nr_zones fields.
+         */
+        ret = get_sysfs_long_val(&st, "chunk_sectors");
+        if (ret < 0) {
+            error_setg_errno(errp, -ret, "Unable to read chunk_sectors "
+                                         "sysfs attribute");
+            goto out;
+        } else if (!ret) {
+            error_setg(errp, "Read 0 from chunk_sectors sysfs attribute");
+            goto out;
+        }
+        bs->bl.zone_size = ret << BDRV_SECTOR_BITS;
+
+        ret = get_sysfs_long_val(&st, "nr_zones");
+        if (ret < 0) {
+            error_setg_errno(errp, -ret, "Unable to read nr_zones "
+                                         "sysfs attribute");
+            goto out;
+        } else if (!ret) {
+            error_setg(errp, "Read 0 from nr_zones sysfs attribute");
+            goto out;
+        }
+        bs->bl.nr_zones = ret;
+
+        ret = get_sysfs_long_val(&st, "zone_append_max_bytes");
+        if (ret > 0) {
+            bs->bl.max_append_sectors = ret >> BDRV_SECTOR_BITS;
+        }
+
+        ret = get_sysfs_long_val(&st, "max_open_zones");
+        if (ret >= 0) {
+            bs->bl.max_open_zones = ret;
+        }
+
+        ret = get_sysfs_long_val(&st, "max_active_zones");
+        if (ret >= 0) {
+            bs->bl.max_active_zones = ret;
+        }
+        return;
+    }
+out:
+    bs->bl.zoned = BLK_Z_NONE;
 }
 
 static int check_for_dasd(int fd)
@@ -XXX,XX +XXX,XX @@ static int hdev_probe_blocksizes(BlockDriverState *bs, BlockSizes *bsz)
     BDRVRawState *s = bs->opaque;
     int ret;
 
-    /* If DASD, get blocksizes */
+    /* If DASD or zoned devices, get blocksizes */
     if (check_for_dasd(s->fd) < 0) {
-        return -ENOTSUP;
+        /* zoned devices are not DASD */
+        if (bs->bl.zoned == BLK_Z_NONE) {
+            return -ENOTSUP;
+        }
     }
     ret = probe_logical_blocksize(s->fd, &bsz->log);
     if (ret < 0) {
@@ -XXX,XX +XXX,XX @@ static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd,
 }
 #endif
 
+/*
+ * parse_zone - Fill a zone descriptor
+ */
+#if defined(CONFIG_BLKZONED)
+static inline int parse_zone(struct BlockZoneDescriptor *zone,
+                              const struct blk_zone *blkz) {
+    zone->start = blkz->start << BDRV_SECTOR_BITS;
+    zone->length = blkz->len << BDRV_SECTOR_BITS;
+    zone->wp = blkz->wp << BDRV_SECTOR_BITS;
+
+#ifdef HAVE_BLK_ZONE_REP_CAPACITY
+    zone->cap = blkz->capacity << BDRV_SECTOR_BITS;
+#else
+    zone->cap = blkz->len << BDRV_SECTOR_BITS;
+#endif
+
+    switch (blkz->type) {
+    case BLK_ZONE_TYPE_SEQWRITE_REQ:
+        zone->type = BLK_ZT_SWR;
+        break;
+    case BLK_ZONE_TYPE_SEQWRITE_PREF:
+        zone->type = BLK_ZT_SWP;
+        break;
+    case BLK_ZONE_TYPE_CONVENTIONAL:
+        zone->type = BLK_ZT_CONV;
+        break;
+    default:
+        error_report("Unsupported zone type: 0x%x", blkz->type);
+        return -ENOTSUP;
+    }
+
+    switch (blkz->cond) {
+    case BLK_ZONE_COND_NOT_WP:
+        zone->state = BLK_ZS_NOT_WP;
+        break;
+    case BLK_ZONE_COND_EMPTY:
+        zone->state = BLK_ZS_EMPTY;
+        break;
+    case BLK_ZONE_COND_IMP_OPEN:
+        zone->state = BLK_ZS_IOPEN;
+        break;
+    case BLK_ZONE_COND_EXP_OPEN:
+        zone->state = BLK_ZS_EOPEN;
+        break;
+    case BLK_ZONE_COND_CLOSED:
+        zone->state = BLK_ZS_CLOSED;
+        break;
+    case BLK_ZONE_COND_READONLY:
+        zone->state = BLK_ZS_RDONLY;
+        break;
+    case BLK_ZONE_COND_FULL:
+        zone->state = BLK_ZS_FULL;
+        break;
+    case BLK_ZONE_COND_OFFLINE:
+        zone->state = BLK_ZS_OFFLINE;
+        break;
+    default:
+        error_report("Unsupported zone state: 0x%x", blkz->cond);
+        return -ENOTSUP;
+    }
+    return 0;
+}
+#endif
+
+#if defined(CONFIG_BLKZONED)
+static int handle_aiocb_zone_report(void *opaque)
+{
+    RawPosixAIOData *aiocb = opaque;
+    int fd = aiocb->aio_fildes;
+    unsigned int *nr_zones = aiocb->zone_report.nr_zones;
+    BlockZoneDescriptor *zones = aiocb->zone_report.zones;
+    /* zoned block devices use 512-byte sectors */
+    uint64_t sector = aiocb->aio_offset / 512;
+
+    struct blk_zone *blkz;
+    size_t rep_size;
+    unsigned int nrz;
+    int ret, n = 0, i = 0;
+
+    nrz = *nr_zones;
+    rep_size = sizeof(struct blk_zone_report) + nrz * sizeof(struct blk_zone);
+    g_autofree struct blk_zone_report *rep = NULL;
+    rep = g_malloc(rep_size);
+
+    blkz = (struct blk_zone *)(rep + 1);
+    while (n < nrz) {
+        memset(rep, 0, rep_size);
+        rep->sector = sector;
+        rep->nr_zones = nrz - n;
+
+        do {
+            ret = ioctl(fd, BLKREPORTZONE, rep);
+        } while (ret != 0 && errno == EINTR);
+        if (ret != 0) {
+            error_report("%d: ioctl BLKREPORTZONE at %" PRId64 " failed %d",
+                         fd, sector, errno);
+            return -errno;
+        }
+
+        if (!rep->nr_zones) {
+            break;
+        }
+
+        for (i = 0; i < rep->nr_zones; i++, n++) {
+            ret = parse_zone(&zones[n], &blkz[i]);
+            if (ret != 0) {
+                return ret;
+            }
+
+            /* The next report should start after the last zone reported */
+            sector = blkz[i].start + blkz[i].len;
+        }
+    }
+
+    *nr_zones = n;
+    return 0;
+}
+#endif
+
+#if defined(CONFIG_BLKZONED)
+static int handle_aiocb_zone_mgmt(void *opaque)
+{
+    RawPosixAIOData *aiocb = opaque;
+    int fd = aiocb->aio_fildes;
+    uint64_t sector = aiocb->aio_offset / 512;
+    int64_t nr_sectors = aiocb->aio_nbytes / 512;
+    struct blk_zone_range range;
+    int ret;
+
+    /* Execute the operation */
+    range.sector = sector;
+    range.nr_sectors = nr_sectors;
+    do {
+        ret = ioctl(fd, aiocb->zone_mgmt.op, &range);
+    } while (ret != 0 && errno == EINTR);
+
+    return ret;
+}
+#endif
+
 static int handle_aiocb_copy_range(void *opaque)
 {
     RawPosixAIOData *aiocb = opaque;
@@ -XXX,XX +XXX,XX @@ static void raw_account_discard(BDRVRawState *s, uint64_t nbytes, int ret)
     }
 }
 
+/*
+ * zone report - Get a zone block device's information in the form
+ * of an array of zone descriptors.
+ * zones is an array of zone descriptors to hold zone information on reply;
+ * offset can be any byte within the entire size of the device;
+ * nr_zones is the maxium number of sectors the command should operate on.
+ */
+#if defined(CONFIG_BLKZONED)
+static int coroutine_fn raw_co_zone_report(BlockDriverState *bs, int64_t offset,
+                                           unsigned int *nr_zones,
+                                           BlockZoneDescriptor *zones) {
+    BDRVRawState *s = bs->opaque;
+    RawPosixAIOData acb = (RawPosixAIOData) {
+        .bs         = bs,
+        .aio_fildes = s->fd,
+        .aio_type   = QEMU_AIO_ZONE_REPORT,
+        .aio_offset = offset,
+        .zone_report    = {
+            .nr_zones       = nr_zones,
+            .zones          = zones,
+        },
+    };
+
+    return raw_thread_pool_submit(bs, handle_aiocb_zone_report, &acb);
+}
+#endif
+
+/*
+ * zone management operations - Execute an operation on a zone
+ */
+#if defined(CONFIG_BLKZONED)
+static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
+        int64_t offset, int64_t len) {
+    BDRVRawState *s = bs->opaque;
+    RawPosixAIOData acb;
+    int64_t zone_size, zone_size_mask;
+    const char *op_name;
+    unsigned long zo;
+    int ret;
+    int64_t capacity = bs->total_sectors << BDRV_SECTOR_BITS;
+
+    zone_size = bs->bl.zone_size;
+    zone_size_mask = zone_size - 1;
+    if (offset & zone_size_mask) {
+        error_report("sector offset %" PRId64 " is not aligned to zone size "
+                     "%" PRId64 "", offset / 512, zone_size / 512);
+        return -EINVAL;
+    }
+
+    if (((offset + len) < capacity && len & zone_size_mask) ||
+        offset + len > capacity) {
+        error_report("number of sectors %" PRId64 " is not aligned to zone size"
+                      " %" PRId64 "", len / 512, zone_size / 512);
+        return -EINVAL;
+    }
+
+    switch (op) {
+    case BLK_ZO_OPEN:
+        op_name = "BLKOPENZONE";
+        zo = BLKOPENZONE;
+        break;
+    case BLK_ZO_CLOSE:
+        op_name = "BLKCLOSEZONE";
+        zo = BLKCLOSEZONE;
+        break;
+    case BLK_ZO_FINISH:
+        op_name = "BLKFINISHZONE";
+        zo = BLKFINISHZONE;
+        break;
+    case BLK_ZO_RESET:
+        op_name = "BLKRESETZONE";
+        zo = BLKRESETZONE;
+        break;
+    default:
+        error_report("Unsupported zone op: 0x%x", op);
+        return -ENOTSUP;
+    }
+
+    acb = (RawPosixAIOData) {
+        .bs             = bs,
+        .aio_fildes     = s->fd,
+        .aio_type       = QEMU_AIO_ZONE_MGMT,
+        .aio_offset     = offset,
+        .aio_nbytes     = len,
+        .zone_mgmt  = {
+            .op = zo,
+        },
+    };
+
+    ret = raw_thread_pool_submit(bs, handle_aiocb_zone_mgmt, &acb);
+    if (ret != 0) {
+        error_report("ioctl %s failed %d", op_name, ret);
+    }
+
+    return ret;
+}
+#endif
+
 static coroutine_fn int
 raw_do_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes,
                 bool blkdev)
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_host_device = {
 #ifdef __linux__
     .bdrv_co_ioctl          = hdev_co_ioctl,
 #endif
+
+    /* zoned device */
+#if defined(CONFIG_BLKZONED)
+    /* zone management operations */
+    .bdrv_co_zone_report = raw_co_zone_report,
+    .bdrv_co_zone_mgmt = raw_co_zone_mgmt,
+#endif
 };
 
 #if defined(__linux__) || defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ out:
     return co.ret;
 }
 
+int coroutine_fn bdrv_co_zone_report(BlockDriverState *bs, int64_t offset,
+                        unsigned int *nr_zones,
+                        BlockZoneDescriptor *zones)
+{
+    BlockDriver *drv = bs->drv;
+    CoroutineIOCompletion co = {
+            .coroutine = qemu_coroutine_self(),
+    };
+    IO_CODE();
+
+    bdrv_inc_in_flight(bs);
+    if (!drv || !drv->bdrv_co_zone_report || bs->bl.zoned == BLK_Z_NONE) {
+        co.ret = -ENOTSUP;
+        goto out;
+    }
+    co.ret = drv->bdrv_co_zone_report(bs, offset, nr_zones, zones);
+out:
+    bdrv_dec_in_flight(bs);
+    return co.ret;
+}
+
+int coroutine_fn bdrv_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
+        int64_t offset, int64_t len)
+{
+    BlockDriver *drv = bs->drv;
+    CoroutineIOCompletion co = {
+            .coroutine = qemu_coroutine_self(),
+    };
+    IO_CODE();
+
+    bdrv_inc_in_flight(bs);
+    if (!drv || !drv->bdrv_co_zone_mgmt || bs->bl.zoned == BLK_Z_NONE) {
+        co.ret = -ENOTSUP;
+        goto out;
+    }
+    co.ret = drv->bdrv_co_zone_mgmt(bs, op, offset, len);
+out:
+    bdrv_dec_in_flight(bs);
+    return co.ret;
+}
+
 void *qemu_blockalign(BlockDriverState *bs, size_t size)
 {
     IO_CODE();
diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t flush_cmd = {
     .oneline    = "flush all in-core file state to disk",
 };
 
+static inline int64_t tosector(int64_t bytes)
+{
+    return bytes >> BDRV_SECTOR_BITS;
+}
+
+static int zone_report_f(BlockBackend *blk, int argc, char **argv)
+{
+    int ret;
+    int64_t offset;
+    unsigned int nr_zones;
+
+    ++optind;
+    offset = cvtnum(argv[optind]);
+    ++optind;
+    nr_zones = cvtnum(argv[optind]);
+
+    g_autofree BlockZoneDescriptor *zones = NULL;
+    zones = g_new(BlockZoneDescriptor, nr_zones);
+    ret = blk_zone_report(blk, offset, &nr_zones, zones);
+    if (ret < 0) {
+        printf("zone report failed: %s\n", strerror(-ret));
+    } else {
+        for (int i = 0; i < nr_zones; ++i) {
+            printf("start: 0x%" PRIx64 ", len 0x%" PRIx64 ", "
+                   "cap"" 0x%" PRIx64 ", wptr 0x%" PRIx64 ", "
+                   "zcond:%u, [type: %u]\n",
+                    tosector(zones[i].start), tosector(zones[i].length),
+                    tosector(zones[i].cap), tosector(zones[i].wp),
+                    zones[i].state, zones[i].type);
+        }
+    }
+    return ret;
+}
+
+static const cmdinfo_t zone_report_cmd = {
+    .name = "zone_report",
+    .altname = "zrp",
+    .cfunc = zone_report_f,
+    .argmin = 2,
+    .argmax = 2,
+    .args = "offset number",
+    .oneline = "report zone information",
+};
+
+static int zone_open_f(BlockBackend *blk, int argc, char **argv)
+{
+    int ret;
+    int64_t offset, len;
+    ++optind;
+    offset = cvtnum(argv[optind]);
+    ++optind;
+    len = cvtnum(argv[optind]);
+    ret = blk_zone_mgmt(blk, BLK_ZO_OPEN, offset, len);
+    if (ret < 0) {
+        printf("zone open failed: %s\n", strerror(-ret));
+    }
+    return ret;
+}
+
+static const cmdinfo_t zone_open_cmd = {
+    .name = "zone_open",
+    .altname = "zo",
+    .cfunc = zone_open_f,
+    .argmin = 2,
+    .argmax = 2,
+    .args = "offset len",
+    .oneline = "explicit open a range of zones in zone block device",
+};
+
+static int zone_close_f(BlockBackend *blk, int argc, char **argv)
+{
+    int ret;
+    int64_t offset, len;
+    ++optind;
+    offset = cvtnum(argv[optind]);
+    ++optind;
+    len = cvtnum(argv[optind]);
+    ret = blk_zone_mgmt(blk, BLK_ZO_CLOSE, offset, len);
+    if (ret < 0) {
+        printf("zone close failed: %s\n", strerror(-ret));
+    }
+    return ret;
+}
+
+static const cmdinfo_t zone_close_cmd = {
+    .name = "zone_close",
+    .altname = "zc",
+    .cfunc = zone_close_f,
+    .argmin = 2,
+    .argmax = 2,
+    .args = "offset len",
+    .oneline = "close a range of zones in zone block device",
+};
+
+static int zone_finish_f(BlockBackend *blk, int argc, char **argv)
+{
+    int ret;
+    int64_t offset, len;
+    ++optind;
+    offset = cvtnum(argv[optind]);
+    ++optind;
+    len = cvtnum(argv[optind]);
+    ret = blk_zone_mgmt(blk, BLK_ZO_FINISH, offset, len);
+    if (ret < 0) {
+        printf("zone finish failed: %s\n", strerror(-ret));
+    }
+    return ret;
+}
+
+static const cmdinfo_t zone_finish_cmd = {
+    .name = "zone_finish",
+    .altname = "zf",
+    .cfunc = zone_finish_f,
+    .argmin = 2,
+    .argmax = 2,
+    .args = "offset len",
+    .oneline = "finish a range of zones in zone block device",
+};
+
+static int zone_reset_f(BlockBackend *blk, int argc, char **argv)
+{
+    int ret;
+    int64_t offset, len;
+    ++optind;
+    offset = cvtnum(argv[optind]);
+    ++optind;
+    len = cvtnum(argv[optind]);
+    ret = blk_zone_mgmt(blk, BLK_ZO_RESET, offset, len);
+    if (ret < 0) {
+        printf("zone reset failed: %s\n", strerror(-ret));
+    }
+    return ret;
+}
+
+static const cmdinfo_t zone_reset_cmd = {
+    .name = "zone_reset",
+    .altname = "zrs",
+    .cfunc = zone_reset_f,
+    .argmin = 2,
+    .argmax = 2,
+    .args = "offset len",
+    .oneline = "reset a zone write pointer in zone block device",
+};
+
 static int truncate_f(BlockBackend *blk, int argc, char **argv);
 static const cmdinfo_t truncate_cmd = {
     .name       = "truncate",
@@ -XXX,XX +XXX,XX @@ static void __attribute((constructor)) init_qemuio_commands(void)
     qemuio_add_command(&aio_write_cmd);
     qemuio_add_command(&aio_flush_cmd);
     qemuio_add_command(&flush_cmd);
+    qemuio_add_command(&zone_report_cmd);
+    qemuio_add_command(&zone_open_cmd);
+    qemuio_add_command(&zone_close_cmd);
+    qemuio_add_command(&zone_finish_cmd);
+    qemuio_add_command(&zone_reset_cmd);
     qemuio_add_command(&truncate_cmd);
     qemuio_add_command(&length_cmd);
     qemuio_add_command(&info_cmd);
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

raw-format driver usually sits on top of file-posix driver. It needs to
pass through requests of zone commands.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-5-faithilikerun@gmail.com
[Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
<philmd@linaro.org>.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/raw-format.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/block/raw-format.c b/block/raw-format.c
index XXXXXXX..XXXXXXX 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -XXX,XX +XXX,XX @@ raw_co_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes)
     return bdrv_co_pdiscard(bs->file, offset, bytes);
 }
 
+static int coroutine_fn GRAPH_RDLOCK
+raw_co_zone_report(BlockDriverState *bs, int64_t offset,
+                   unsigned int *nr_zones,
+                   BlockZoneDescriptor *zones)
+{
+    return bdrv_co_zone_report(bs->file->bs, offset, nr_zones, zones);
+}
+
+static int coroutine_fn GRAPH_RDLOCK
+raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
+                 int64_t offset, int64_t len)
+{
+    return bdrv_co_zone_mgmt(bs->file->bs, op, offset, len);
+}
+
 static int64_t coroutine_fn GRAPH_RDLOCK
 raw_co_getlength(BlockDriverState *bs)
 {
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_raw = {
     .bdrv_co_pwritev      = &raw_co_pwritev,
     .bdrv_co_pwrite_zeroes = &raw_co_pwrite_zeroes,
     .bdrv_co_pdiscard     = &raw_co_pdiscard,
+    .bdrv_co_zone_report  = &raw_co_zone_report,
+    .bdrv_co_zone_mgmt  = &raw_co_zone_mgmt,
     .bdrv_co_block_status = &raw_co_block_status,
     .bdrv_co_copy_range_from = &raw_co_copy_range_from,
     .bdrv_co_copy_range_to  = &raw_co_copy_range_to,
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Putting zoned/non-zoned BlockDrivers on top of each other is not
allowed.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-6-faithilikerun@gmail.com
[Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
<philmd@linaro.org> and clarify that the check is about zoned
BlockDrivers.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/block/block_int-common.h |  5 +++++
 block.c                          | 19 +++++++++++++++++++
 block/file-posix.c               | 12 ++++++++++++
 block/raw-format.c               |  1 +
 4 files changed, 37 insertions(+)

diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
      */
     bool is_format;
 
+    /*
+     * Set to true if the BlockDriver supports zoned children.
+     */
+    bool supports_zoned_children;
+
     /*
      * Drivers not implementing bdrv_parse_filename nor bdrv_open should have
      * this field set to true, except ones that are defined only by their
diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_add_child(BlockDriverState *parent_bs, BlockDriverState *child_bs,
         return;
     }
 
+    /*
+     * Non-zoned block drivers do not follow zoned storage constraints
+     * (i.e. sequential writes to zones). Refuse mixing zoned and non-zoned
+     * drivers in a graph.
+     */
+    if (!parent_bs->drv->supports_zoned_children &&
+        child_bs->bl.zoned == BLK_Z_HM) {
+        /*
+         * The host-aware model allows zoned storage constraints and random
+         * write. Allow mixing host-aware and non-zoned drivers. Using
+         * host-aware device as a regular device.
+         */
+        error_setg(errp, "Cannot add a %s child to a %s parent",
+                   child_bs->bl.zoned == BLK_Z_HM ? "zoned" : "non-zoned",
+                   parent_bs->drv->supports_zoned_children ?
+                   "support zoned children" : "not support zoned children");
+        return;
+    }
+
     if (!QLIST_EMPTY(&child_bs->parents)) {
         error_setg(errp, "The node %s already has a parent",
                    child_bs->node_name);
diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
             goto fail;
         }
     }
+#ifdef CONFIG_BLKZONED
+    /*
+     * The kernel page cache does not reliably work for writes to SWR zones
+     * of zoned block device because it can not guarantee the order of writes.
+     */
+    if ((bs->bl.zoned != BLK_Z_NONE) &&
+        (!(s->open_flags & O_DIRECT))) {
+        error_setg(errp, "The driver supports zoned devices, and it requires "
+                         "cache.direct=on, which was not specified.");
+        return -EINVAL; /* No host kernel page cache */
+    }
+#endif
 
     if (S_ISBLK(st.st_mode)) {
 #ifdef __linux__
diff --git a/block/raw-format.c b/block/raw-format.c
index XXXXXXX..XXXXXXX 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -XXX,XX +XXX,XX @@ static void raw_child_perm(BlockDriverState *bs, BdrvChild *c,
 BlockDriver bdrv_raw = {
     .format_name          = "raw",
     .instance_size        = sizeof(BDRVRawState),
+    .supports_zoned_children = true,
     .bdrv_probe           = &raw_probe,
     .bdrv_reopen_prepare  = &raw_reopen_prepare,
     .bdrv_reopen_commit   = &raw_reopen_commit,
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

The new block layer APIs of zoned block devices can be tested by:
$ tests/qemu-iotests/check zoned
Run each zone operation on a newly created null_blk device
and see whether it outputs the same zone information.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-7-faithilikerun@gmail.com
[Adjust commit message prefix as suggested by Philippe Mathieu-Daudé
<philmd@linaro.org>.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 tests/qemu-iotests/tests/zoned     | 89 ++++++++++++++++++++++++++++++
 tests/qemu-iotests/tests/zoned.out | 53 ++++++++++++++++++
 2 files changed, 142 insertions(+)
 create mode 100755 tests/qemu-iotests/tests/zoned
 create mode 100644 tests/qemu-iotests/tests/zoned.out

diff --git a/tests/qemu-iotests/tests/zoned b/tests/qemu-iotests/tests/zoned
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/tests/zoned
@@ -XXX,XX +XXX,XX @@
+#!/usr/bin/env bash
+#
+# Test zone management operations.
+#
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+status=1 # failure is the default!
+
+_cleanup()
+{
+  _cleanup_test_img
+  sudo -n rmmod null_blk
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ../common.rc
+. ../common.filter
+. ../common.qemu
+
+# This test only runs on Linux hosts with raw image files.
+_supported_fmt raw
+_supported_proto file
+_supported_os Linux
+
+sudo -n true || \
+    _notrun 'Password-less sudo required'
+
+IMG="--image-opts -n driver=host_device,filename=/dev/nullb0"
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+echo "Testing a null_blk device:"
+echo "case 1: if the operations work"
+sudo -n modprobe null_blk nr_devices=1 zoned=1
+sudo -n chmod 0666 /dev/nullb0
+
+echo "(1) report the first zone:"
+$QEMU_IO $IMG -c "zrp 0 1"
+echo
+echo "report the first 10 zones"
+$QEMU_IO $IMG -c "zrp 0 10"
+echo
+echo "report the last zone:"
+$QEMU_IO $IMG -c "zrp 0x3e70000000 2" # 0x3e70000000 / 512 = 0x1f380000
+echo
+echo
+echo "(2) opening the first zone"
+$QEMU_IO $IMG -c "zo 0 268435456"  # 268435456 / 512 = 524288
+echo "report after:"
+$QEMU_IO $IMG -c "zrp 0 1"
+echo
+echo "opening the second zone"
+$QEMU_IO $IMG -c "zo 268435456 268435456" #
+echo "report after:"
+$QEMU_IO $IMG -c "zrp 268435456 1"
+echo
+echo "opening the last zone"
+$QEMU_IO $IMG -c "zo 0x3e70000000 268435456"
+echo "report after:"
+$QEMU_IO $IMG -c "zrp 0x3e70000000 2"
+echo
+echo
+echo "(3) closing the first zone"
+$QEMU_IO $IMG -c "zc 0 268435456"
+echo "report after:"
+$QEMU_IO $IMG -c "zrp 0 1"
+echo
+echo "closing the last zone"
+$QEMU_IO $IMG -c "zc 0x3e70000000 268435456"
+echo "report after:"
+$QEMU_IO $IMG -c "zrp 0x3e70000000 2"
+echo
+echo
+echo "(4) finishing the second zone"
+$QEMU_IO $IMG -c "zf 268435456 268435456"
+echo "After finishing a zone:"
+$QEMU_IO $IMG -c "zrp 268435456 1"
+echo
+echo
+echo "(5) resetting the second zone"
+$QEMU_IO $IMG -c "zrs 268435456 268435456"
+echo "After resetting a zone:"
+$QEMU_IO $IMG -c "zrp 268435456 1"
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/tests/zoned.out b/tests/qemu-iotests/tests/zoned.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/tests/zoned.out
@@ -XXX,XX +XXX,XX @@
+QA output created by zoned
+Testing a null_blk device:
+case 1: if the operations work
+(1) report the first zone:
+start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
+
+report the first 10 zones
+start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
+start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:1, [type: 2]
+start: 0x100000, len 0x80000, cap 0x80000, wptr 0x100000, zcond:1, [type: 2]
+start: 0x180000, len 0x80000, cap 0x80000, wptr 0x180000, zcond:1, [type: 2]
+start: 0x200000, len 0x80000, cap 0x80000, wptr 0x200000, zcond:1, [type: 2]
+start: 0x280000, len 0x80000, cap 0x80000, wptr 0x280000, zcond:1, [type: 2]
+start: 0x300000, len 0x80000, cap 0x80000, wptr 0x300000, zcond:1, [type: 2]
+start: 0x380000, len 0x80000, cap 0x80000, wptr 0x380000, zcond:1, [type: 2]
+start: 0x400000, len 0x80000, cap 0x80000, wptr 0x400000, zcond:1, [type: 2]
+start: 0x480000, len 0x80000, cap 0x80000, wptr 0x480000, zcond:1, [type: 2]
+
+report the last zone:
+start: 0x1f380000, len 0x80000, cap 0x80000, wptr 0x1f380000, zcond:1, [type: 2]
+
+
+(2) opening the first zone
+report after:
+start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:3, [type: 2]
+
+opening the second zone
+report after:
+start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:3, [type: 2]
+
+opening the last zone
+report after:
+start: 0x1f380000, len 0x80000, cap 0x80000, wptr 0x1f380000, zcond:3, [type: 2]
+
+
+(3) closing the first zone
+report after:
+start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
+
+closing the last zone
+report after:
+start: 0x1f380000, len 0x80000, cap 0x80000, wptr 0x1f380000, zcond:1, [type: 2]
+
+
+(4) finishing the second zone
+After finishing a zone:
+start: 0x80000, len 0x80000, cap 0x80000, wptr 0x100000, zcond:14, [type: 2]
+
+
+(5) resetting the second zone
+After resetting a zone:
+start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:1, [type: 2]
+*** done
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-8-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/file-posix.c | 3 +++
 block/trace-events | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_report(BlockDriverState *bs, int64_t offset,
         },
     };
 
+    trace_zbd_zone_report(bs, *nr_zones, offset >> BDRV_SECTOR_BITS);
     return raw_thread_pool_submit(bs, handle_aiocb_zone_report, &acb);
 }
 #endif
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
         },
     };
 
+    trace_zbd_zone_mgmt(bs, op_name, offset >> BDRV_SECTOR_BITS,
+                        len >> BDRV_SECTOR_BITS);
     ret = raw_thread_pool_submit(bs, handle_aiocb_zone_mgmt, &acb);
     if (ret != 0) {
         error_report("ioctl %s failed %d", op_name, ret);
diff --git a/block/trace-events b/block/trace-events
index XXXXXXX..XXXXXXX 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -XXX,XX +XXX,XX @@ file_FindEjectableOpticalMedia(const char *media) "Matching using %s"
 file_setup_cdrom(const char *partition) "Using %s as optical disc"
 file_hdev_is_sg(int type, int version) "SG device found: type=%d, version=%d"
 file_flush_fdatasync_failed(int err) "errno %d"
+zbd_zone_report(void *bs, unsigned int nr_zones, int64_t sector) "bs %p report %d zones starting at sector offset 0x%" PRIx64 ""
+zbd_zone_mgmt(void *bs, const char *op_name, int64_t sector, int64_t len) "bs %p %s starts at sector offset 0x%" PRIx64 " over a range of 0x%" PRIx64 " sectors"
 
 # ssh.c
 sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Add the documentation about the zoned device support to virtio-blk
emulation.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230324090605.28361-9-faithilikerun@gmail.com
[Add index-api.rst to fix "zoned-storage.rst:document isn't included in
any toctree" error.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/devel/index-api.rst               |  1 +
 docs/devel/zoned-storage.rst           | 43 ++++++++++++++++++++++++++
 docs/system/qemu-block-drivers.rst.inc |  6 ++++
 3 files changed, 50 insertions(+)
 create mode 100644 docs/devel/zoned-storage.rst

diff --git a/docs/devel/index-api.rst b/docs/devel/index-api.rst
index XXXXXXX..XXXXXXX 100644
--- a/docs/devel/index-api.rst
+++ b/docs/devel/index-api.rst
@@ -XXX,XX +XXX,XX @@ generated from in-code annotations to function prototypes.
    memory
    modules
    ui
+   zoned-storage
diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/docs/devel/zoned-storage.rst
@@ -XXX,XX +XXX,XX @@
+=============
+zoned-storage
+=============
+
+Zoned Block Devices (ZBDs) divide the LBA space into block regions called zones
+that are larger than the LBA size. They can only allow sequential writes, which
+can reduce write amplification in SSDs, and potentially lead to higher
+throughput and increased capacity. More details about ZBDs can be found at:
+
+https://zonedstorage.io/docs/introduction/zoned-storage
+
+1. Block layer APIs for zoned storage
+-------------------------------------
+QEMU block layer supports three zoned storage models:
+- BLK_Z_HM: The host-managed zoned model only allows sequential writes access
+to zones. It supports ZBD-specific I/O commands that can be used by a host to
+manage the zones of a device.
+- BLK_Z_HA: The host-aware zoned model allows random write operations in
+zones, making it backward compatible with regular block devices.
+- BLK_Z_NONE: The non-zoned model has no zones support. It includes both
+regular and drive-managed ZBD devices. ZBD-specific I/O commands are not
+supported.
+
+The block device information resides inside BlockDriverState. QEMU uses
+BlockLimits struct(BlockDriverState::bl) that is continuously accessed by the
+block layer while processing I/O requests. A BlockBackend has a root pointer to
+a BlockDriverState graph(for example, raw format on top of file-posix). The
+zoned storage information can be propagated from the leaf BlockDriverState all
+the way up to the BlockBackend. If the zoned storage model in file-posix is
+set to BLK_Z_HM, then block drivers will declare support for zoned host device.
+
+The block layer APIs support commands needed for zoned storage devices,
+including report zones, four zone operations, and zone append.
+
+2. Emulating zoned storage controllers
+--------------------------------------
+When the BlockBackend's BlockLimits model reports a zoned storage device, users
+like the virtio-blk emulation or the qemu-io-cmds.c utility can use block layer
+APIs for zoned storage emulation or testing.
+
+For example, to test zone_report on a null_blk device using qemu-io is:
+$ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0
+-c "zrp offset nr_zones"
diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
index XXXXXXX..XXXXXXX 100644
--- a/docs/system/qemu-block-drivers.rst.inc
+++ b/docs/system/qemu-block-drivers.rst.inc
@@ -XXX,XX +XXX,XX @@ Hard disks
   you may corrupt your host data (use the ``-snapshot`` command
   line option or modify the device permissions accordingly).
 
+Zoned block devices
+  Zoned block devices can be passed through to the guest if the emulated storage
+  controller supports zoned storage. Use ``--blockdev host_device,
+  node-name=drive0,filename=/dev/nullb0,cache.direct=on`` to pass through
+  ``/dev/nullb0`` as ``drive0``.
+
 Windows
 ^^^^^^^
 
-- 
2.39.2

From: Philippe Mathieu-Daudé <philmd@linaro.org>

Introduce the BdrvDmgUncompressFunc type defintion. To emphasis
dmg_uncompress_bz2 and dmg_uncompress_lzfse are pointer to functions,
declare them using this new typedef.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20230320152610.32052-1-philmd@linaro.org
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/dmg.h | 8 ++++----
 block/dmg.c | 7 ++-----
 2 files changed, 6 insertions(+), 9 deletions(-)

diff --git a/block/dmg.h b/block/dmg.h
index XXXXXXX..XXXXXXX 100644
--- a/block/dmg.h
+++ b/block/dmg.h
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVDMGState {
     z_stream zstream;
 } BDRVDMGState;
 
-extern int (*dmg_uncompress_bz2)(char *next_in, unsigned int avail_in,
-                                 char *next_out, unsigned int avail_out);
+typedef int BdrvDmgUncompressFunc(char *next_in, unsigned int avail_in,
+                                  char *next_out, unsigned int avail_out);
 
-extern int (*dmg_uncompress_lzfse)(char *next_in, unsigned int avail_in,
-                                   char *next_out, unsigned int avail_out);
+extern BdrvDmgUncompressFunc *dmg_uncompress_bz2;
+extern BdrvDmgUncompressFunc *dmg_uncompress_lzfse;
 
 #endif
diff --git a/block/dmg.c b/block/dmg.c
index XXXXXXX..XXXXXXX 100644
--- a/block/dmg.c
+++ b/block/dmg.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/memalign.h"
 #include "dmg.h"
 
-int (*dmg_uncompress_bz2)(char *next_in, unsigned int avail_in,
-                          char *next_out, unsigned int avail_out);
-
-int (*dmg_uncompress_lzfse)(char *next_in, unsigned int avail_in,
-                            char *next_out, unsigned int avail_out);
+BdrvDmgUncompressFunc *dmg_uncompress_bz2;
+BdrvDmgUncompressFunc *dmg_uncompress_lzfse;
 
 enum {
     /* Limit chunk sizes to prevent unreasonable amounts of memory being used
-- 
2.39.2

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

The event filename is an absolute path. Convert it to a relative path when
writing '#line' directives, to preserve reproducibility of the generated
output when different base paths are used.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230406080045.21696-1-thomas.de_schampheleire@nokia.com>
---
 scripts/tracetool/backend/ftrace.py | 4 +++-
 scripts/tracetool/backend/log.py    | 4 +++-
 scripts/tracetool/backend/syslog.py | 4 +++-
 3 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/scripts/tracetool/backend/ftrace.py b/scripts/tracetool/backend/ftrace.py
index XXXXXXX..XXXXXXX 100644
--- a/scripts/tracetool/backend/ftrace.py
+++ b/scripts/tracetool/backend/ftrace.py
@@ -XXX,XX +XXX,XX @@
 __email__      = "stefanha@redhat.com"
 
 
+import os.path
+
 from tracetool import out
 
 
@@ -XXX,XX +XXX,XX @@ def generate_h(event, group):
         args=event.args,
         event_id="TRACE_" + event.name.upper(),
         event_lineno=event.lineno,
-        event_filename=event.filename,
+        event_filename=os.path.relpath(event.filename),
         fmt=event.fmt.rstrip("\n"),
         argnames=argnames)
 
diff --git a/scripts/tracetool/backend/log.py b/scripts/tracetool/backend/log.py
index XXXXXXX..XXXXXXX 100644
--- a/scripts/tracetool/backend/log.py
+++ b/scripts/tracetool/backend/log.py
@@ -XXX,XX +XXX,XX @@
 __email__      = "stefanha@redhat.com"
 
 
+import os.path
+
 from tracetool import out
 
 
@@ -XXX,XX +XXX,XX @@ def generate_h(event, group):
         '    }',
         cond=cond,
         event_lineno=event.lineno,
-        event_filename=event.filename,
+        event_filename=os.path.relpath(event.filename),
         name=event.name,
         fmt=event.fmt.rstrip("\n"),
         argnames=argnames)
diff --git a/scripts/tracetool/backend/syslog.py b/scripts/tracetool/backend/syslog.py
index XXXXXXX..XXXXXXX 100644
--- a/scripts/tracetool/backend/syslog.py
+++ b/scripts/tracetool/backend/syslog.py
@@ -XXX,XX +XXX,XX @@
 __email__      = "stefanha@redhat.com"
 
 
+import os.path
+
 from tracetool import out
 
 
@@ -XXX,XX +XXX,XX @@ def generate_h(event, group):
         '    }',
         cond=cond,
         event_lineno=event.lineno,
-        event_filename=event.filename,
+        event_filename=os.path.relpath(event.filename),
         name=event.name,
         fmt=event.fmt.rstrip("\n"),
         argnames=argnames)
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Since Linux doesn't have a user API to issue zone append operations to
zoned devices from user space, the file-posix driver is modified to add
zone append emulation using regular writes. To do this, the file-posix
driver tracks the wp location of all zones of the device. It uses an
array of uint64_t. The most significant bit of each wp location indicates
if the zone type is conventional zones.

The zones wp can be changed due to the following operations issued:
- zone reset: change the wp to the start offset of that zone
- zone finish: change to the end location of that zone
- write to a zone
- zone append

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Message-id: 20230407081657.17947-2-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/block/block-common.h     |  14 +++
 include/block/block_int-common.h |   5 +
 block/file-posix.c               | 173 ++++++++++++++++++++++++++++++-
 3 files changed, 189 insertions(+), 3 deletions(-)

diff --git a/include/block/block-common.h b/include/block/block-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block-common.h
+++ b/include/block/block-common.h
@@ -XXX,XX +XXX,XX @@ typedef struct BlockZoneDescriptor {
     BlockZoneState state;
 } BlockZoneDescriptor;
 
+/*
+ * Track write pointers of a zone in bytes.
+ */
+typedef struct BlockZoneWps {
+    CoMutex colock;
+    uint64_t wp[];
+} BlockZoneWps;
+
 typedef struct BlockDriverInfo {
     /* in bytes, 0 if irrelevant */
     int cluster_size;
@@ -XXX,XX +XXX,XX @@ typedef enum {
 #define BDRV_SECTOR_BITS   9
 #define BDRV_SECTOR_SIZE   (1ULL << BDRV_SECTOR_BITS)
 
+/*
+ * Get the first most significant bit of wp. If it is zero, then
+ * the zone type is SWR.
+ */
+#define BDRV_ZT_IS_CONV(wp)    (wp & (1ULL << 63))
+
 #define BDRV_REQUEST_MAX_SECTORS MIN_CONST(SIZE_MAX >> BDRV_SECTOR_BITS, \
                                            INT_MAX >> BDRV_SECTOR_BITS)
 #define BDRV_REQUEST_MAX_BYTES (BDRV_REQUEST_MAX_SECTORS << BDRV_SECTOR_BITS)
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ typedef struct BlockLimits {
 
     /* maximum number of active zones */
     int64_t max_active_zones;
+
+    int64_t write_granularity;
 } BlockLimits;
 
 typedef struct BdrvOpBlocker BdrvOpBlocker;
@@ -XXX,XX +XXX,XX @@ struct BlockDriverState {
     CoMutex bsc_modify_lock;
     /* Always non-NULL, but must only be dereferenced under an RCU read guard */
     BdrvBlockStatusCache *block_status_cache;
+
+    /* array of write pointers' location of each zone in the zoned device. */
+    BlockZoneWps *wps;
 };
 
 struct BlockBackendRootState {
diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int hdev_get_max_segments(int fd, struct stat *st)
 #endif
 }
 
+#if defined(CONFIG_BLKZONED)
+/*
+ * If the reset_all flag is true, then the wps of zone whose state is
+ * not readonly or offline should be all reset to the start sector.
+ * Else, take the real wp of the device.
+ */
+static int get_zones_wp(BlockDriverState *bs, int fd, int64_t offset,
+                        unsigned int nrz, bool reset_all)
+{
+    struct blk_zone *blkz;
+    size_t rep_size;
+    uint64_t sector = offset >> BDRV_SECTOR_BITS;
+    BlockZoneWps *wps = bs->wps;
+    int j = offset / bs->bl.zone_size;
+    int ret, n = 0, i = 0;
+    rep_size = sizeof(struct blk_zone_report) + nrz * sizeof(struct blk_zone);
+    g_autofree struct blk_zone_report *rep = NULL;
+
+    rep = g_malloc(rep_size);
+    blkz = (struct blk_zone *)(rep + 1);
+    while (n < nrz) {
+        memset(rep, 0, rep_size);
+        rep->sector = sector;
+        rep->nr_zones = nrz - n;
+
+        do {
+            ret = ioctl(fd, BLKREPORTZONE, rep);
+        } while (ret != 0 && errno == EINTR);
+        if (ret != 0) {
+            error_report("%d: ioctl BLKREPORTZONE at %" PRId64 " failed %d",
+                    fd, offset, errno);
+            return -errno;
+        }
+
+        if (!rep->nr_zones) {
+            break;
+        }
+
+        for (i = 0; i < rep->nr_zones; ++i, ++n, ++j) {
+            /*
+             * The wp tracking cares only about sequential writes required and
+             * sequential write preferred zones so that the wp can advance to
+             * the right location.
+             * Use the most significant bit of the wp location to indicate the
+             * zone type: 0 for SWR/SWP zones and 1 for conventional zones.
+             */
+            if (blkz[i].type == BLK_ZONE_TYPE_CONVENTIONAL) {
+                wps->wp[j] |= 1ULL << 63;
+            } else {
+                switch(blkz[i].cond) {
+                case BLK_ZONE_COND_FULL:
+                case BLK_ZONE_COND_READONLY:
+                    /* Zone not writable */
+                    wps->wp[j] = (blkz[i].start + blkz[i].len) << BDRV_SECTOR_BITS;
+                    break;
+                case BLK_ZONE_COND_OFFLINE:
+                    /* Zone not writable nor readable */
+                    wps->wp[j] = (blkz[i].start) << BDRV_SECTOR_BITS;
+                    break;
+                default:
+                    if (reset_all) {
+                        wps->wp[j] = blkz[i].start << BDRV_SECTOR_BITS;
+                    } else {
+                        wps->wp[j] = blkz[i].wp << BDRV_SECTOR_BITS;
+                    }
+                    break;
+                }
+            }
+        }
+        sector = blkz[i - 1].start + blkz[i - 1].len;
+    }
+
+    return 0;
+}
+
+static void update_zones_wp(BlockDriverState *bs, int fd, int64_t offset,
+                            unsigned int nrz)
+{
+    if (get_zones_wp(bs, fd, offset, nrz, 0) < 0) {
+        error_report("update zone wp failed");
+    }
+}
+#endif
+
 static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
 {
     BDRVRawState *s = bs->opaque;
@@ -XXX,XX +XXX,XX @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
         if (ret >= 0) {
             bs->bl.max_active_zones = ret;
         }
+
+        ret = get_sysfs_long_val(&st, "physical_block_size");
+        if (ret >= 0) {
+            bs->bl.write_granularity = ret;
+        }
+
+        /* The refresh_limits() function can be called multiple times. */
+        g_free(bs->wps);
+        bs->wps = g_malloc(sizeof(BlockZoneWps) +
+                sizeof(int64_t) * bs->bl.nr_zones);
+        ret = get_zones_wp(bs, s->fd, 0, bs->bl.nr_zones, 0);
+        if (ret < 0) {
+            error_setg_errno(errp, -ret, "report wps failed");
+            bs->wps = NULL;
+            return;
+        }
+        qemu_co_mutex_init(&bs->wps->colock);
         return;
     }
 out:
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
 {
     BDRVRawState *s = bs->opaque;
     RawPosixAIOData acb;
+    int ret;
 
     if (fd_open(bs) < 0)
         return -EIO;
+#if defined(CONFIG_BLKZONED)
+    if (type & QEMU_AIO_WRITE && bs->wps) {
+        qemu_co_mutex_lock(&bs->wps->colock);
+    }
+#endif
 
     /*
      * When using O_DIRECT, the request must be aligned to be able to use
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
     } else if (s->use_linux_io_uring) {
         LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs));
         assert(qiov->size == bytes);
-        return luring_co_submit(bs, aio, s->fd, offset, qiov, type);
+        ret = luring_co_submit(bs, aio, s->fd, offset, qiov, type);
+        goto out;
 #endif
 #ifdef CONFIG_LINUX_AIO
     } else if (s->use_linux_aio) {
         LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
         assert(qiov->size == bytes);
-        return laio_co_submit(bs, aio, s->fd, offset, qiov, type,
+        ret = laio_co_submit(bs, aio, s->fd, offset, qiov, type,
                               s->aio_max_batch);
+        goto out;
 #endif
     }
 
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
     };
 
     assert(qiov->size == bytes);
-    return raw_thread_pool_submit(bs, handle_aiocb_rw, &acb);
+    ret = raw_thread_pool_submit(bs, handle_aiocb_rw, &acb);
+
+out:
+#if defined(CONFIG_BLKZONED)
+    BlockZoneWps *wps = bs->wps;
+    if (ret == 0) {
+        if (type & QEMU_AIO_WRITE && wps && bs->bl.zone_size) {
+            uint64_t *wp = &wps->wp[offset / bs->bl.zone_size];
+            if (!BDRV_ZT_IS_CONV(*wp)) {
+                /* Advance the wp if needed */
+                if (offset + bytes > *wp) {
+                    *wp = offset + bytes;
+                }
+            }
+        }
+    } else {
+        if (type & QEMU_AIO_WRITE) {
+            update_zones_wp(bs, s->fd, 0, 1);
+        }
+    }
+
+    if (type & QEMU_AIO_WRITE && wps) {
+        qemu_co_mutex_unlock(&wps->colock);
+    }
+#endif
+    return ret;
 }
 
 static int coroutine_fn raw_co_preadv(BlockDriverState *bs, int64_t offset,
@@ -XXX,XX +XXX,XX @@ static void raw_close(BlockDriverState *bs)
     BDRVRawState *s = bs->opaque;
 
     if (s->fd >= 0) {
+#if defined(CONFIG_BLKZONED)
+        g_free(bs->wps);
+#endif
         qemu_close(s->fd);
         s->fd = -1;
     }
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
     const char *op_name;
     unsigned long zo;
     int ret;
+    BlockZoneWps *wps = bs->wps;
     int64_t capacity = bs->total_sectors << BDRV_SECTOR_BITS;
 
     zone_size = bs->bl.zone_size;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
         return -EINVAL;
     }
 
+    QEMU_LOCK_GUARD(&wps->colock);
+    uint32_t i = offset / bs->bl.zone_size;
+    uint32_t nrz = len / bs->bl.zone_size;
+    uint64_t *wp = &wps->wp[i];
+    if (BDRV_ZT_IS_CONV(*wp) && len != capacity) {
+        error_report("zone mgmt operations are not allowed for conventional zones");
+        return -EIO;
+    }
+
     switch (op) {
     case BLK_ZO_OPEN:
         op_name = "BLKOPENZONE";
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
                         len >> BDRV_SECTOR_BITS);
     ret = raw_thread_pool_submit(bs, handle_aiocb_zone_mgmt, &acb);
     if (ret != 0) {
+        update_zones_wp(bs, s->fd, offset, i);
         error_report("ioctl %s failed %d", op_name, ret);
+        return ret;
+    }
+
+    if (zo == BLKRESETZONE && len == capacity) {
+        ret = get_zones_wp(bs, s->fd, 0, bs->bl.nr_zones, 1);
+        if (ret < 0) {
+            error_report("reporting single wp failed");
+            return ret;
+        }
+    } else if (zo == BLKRESETZONE) {
+        for (int j = 0; j < nrz; ++j) {
+            wp[j] = offset + j * zone_size;
+        }
+    } else if (zo == BLKFINISHZONE) {
+        for (int j = 0; j < nrz; ++j) {
+            /* The zoned device allows the last zone smaller that the
+             * zone size. */
+            wp[j] = MIN(offset + (j + 1) * zone_size, offset + len);
+        }
     }
 
     return ret;
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

A zone append command is a write operation that specifies the first
logical block of a zone as the write position. When writing to a zoned
block device using zone append, the byte offset of the call may point at
any position within the zone to which the data is being appended. Upon
completion the device will respond with the position where the data has
been written in the zone.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20230407081657.17947-3-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/block/block-io.h          |  4 +++
 include/block/block_int-common.h  |  3 ++
 include/block/raw-aio.h           |  4 ++-
 include/sysemu/block-backend-io.h |  9 +++++
 block/block-backend.c             | 60 +++++++++++++++++++++++++++++++
 block/file-posix.c                | 58 ++++++++++++++++++++++++++----
 block/io.c                        | 27 ++++++++++++++
 block/io_uring.c                  |  4 +++
 block/linux-aio.c                 |  3 ++
 block/raw-format.c                |  8 +++++
 10 files changed, 172 insertions(+), 8 deletions(-)

diff --git a/include/block/block-io.h b/include/block/block-io.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block-io.h
+++ b/include/block/block-io.h
@@ -XXX,XX +XXX,XX @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_report(BlockDriverState *bs,
 int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_mgmt(BlockDriverState *bs,
                                                 BlockZoneOp op,
                                                 int64_t offset, int64_t len);
+int coroutine_fn GRAPH_RDLOCK bdrv_co_zone_append(BlockDriverState *bs,
+                                                  int64_t *offset,
+                                                  QEMUIOVector *qiov,
+                                                  BdrvRequestFlags flags);
 
 bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
 int bdrv_block_status(BlockDriverState *bs, int64_t offset,
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
             BlockZoneDescriptor *zones);
     int coroutine_fn (*bdrv_co_zone_mgmt)(BlockDriverState *bs, BlockZoneOp op,
             int64_t offset, int64_t len);
+    int coroutine_fn (*bdrv_co_zone_append)(BlockDriverState *bs,
+            int64_t *offset, QEMUIOVector *qiov,
+            BdrvRequestFlags flags);
 
     /* removable device specific */
     bool coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_is_inserted)(
diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/raw-aio.h
+++ b/include/block/raw-aio.h
@@ -XXX,XX +XXX,XX @@
 #define QEMU_AIO_TRUNCATE     0x0080
 #define QEMU_AIO_ZONE_REPORT  0x0100
 #define QEMU_AIO_ZONE_MGMT    0x0200
+#define QEMU_AIO_ZONE_APPEND  0x0400
 #define QEMU_AIO_TYPE_MASK \
         (QEMU_AIO_READ | \
          QEMU_AIO_WRITE | \
@@ -XXX,XX +XXX,XX @@
          QEMU_AIO_COPY_RANGE | \
          QEMU_AIO_TRUNCATE | \
          QEMU_AIO_ZONE_REPORT | \
-         QEMU_AIO_ZONE_MGMT)
+         QEMU_AIO_ZONE_MGMT | \
+         QEMU_AIO_ZONE_APPEND)
 
 /* AIO flags */
 #define QEMU_AIO_MISALIGNED   0x1000
diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backend-io.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/block-backend-io.h
+++ b/include/sysemu/block-backend-io.h
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_zone_report(BlockBackend *blk, int64_t offset,
 BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
                               int64_t offset, int64_t len,
                               BlockCompletionFunc *cb, void *opaque);
+BlockAIOCB *blk_aio_zone_append(BlockBackend *blk, int64_t *offset,
+                                QEMUIOVector *qiov, BdrvRequestFlags flags,
+                                BlockCompletionFunc *cb, void *opaque);
 BlockAIOCB *blk_aio_pdiscard(BlockBackend *blk, int64_t offset, int64_t bytes,
                              BlockCompletionFunc *cb, void *opaque);
 void blk_aio_cancel_async(BlockAIOCB *acb);
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
                                   int64_t offset, int64_t len);
 int co_wrapper_mixed blk_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
                                        int64_t offset, int64_t len);
+int coroutine_fn blk_co_zone_append(BlockBackend *blk, int64_t *offset,
+                                    QEMUIOVector *qiov,
+                                    BdrvRequestFlags flags);
+int co_wrapper_mixed blk_zone_append(BlockBackend *blk, int64_t *offset,
+                                         QEMUIOVector *qiov,
+                                         BdrvRequestFlags flags);
 
 int co_wrapper_mixed blk_pdiscard(BlockBackend *blk, int64_t offset,
                                   int64_t bytes);
diff --git a/block/block-backend.c b/block/block-backend.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *blk_aio_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
     return &acb->common;
 }
 
+static void coroutine_fn blk_aio_zone_append_entry(void *opaque)
+{
+    BlkAioEmAIOCB *acb = opaque;
+    BlkRwCo *rwco = &acb->rwco;
+
+    rwco->ret = blk_co_zone_append(rwco->blk, (int64_t *)acb->bytes,
+                                   rwco->iobuf, rwco->flags);
+    blk_aio_complete(acb);
+}
+
+BlockAIOCB *blk_aio_zone_append(BlockBackend *blk, int64_t *offset,
+                                QEMUIOVector *qiov, BdrvRequestFlags flags,
+                                BlockCompletionFunc *cb, void *opaque) {
+    BlkAioEmAIOCB *acb;
+    Coroutine *co;
+    IO_CODE();
+
+    blk_inc_in_flight(blk);
+    acb = blk_aio_get(&blk_aio_em_aiocb_info, blk, cb, opaque);
+    acb->rwco = (BlkRwCo) {
+        .blk    = blk,
+        .ret    = NOT_DONE,
+        .flags  = flags,
+        .iobuf  = qiov,
+    };
+    acb->bytes = (int64_t)offset;
+    acb->has_returned = false;
+
+    co = qemu_coroutine_create(blk_aio_zone_append_entry, acb);
+    aio_co_enter(blk_get_aio_context(blk), co);
+    acb->has_returned = true;
+    if (acb->rwco.ret != NOT_DONE) {
+        replay_bh_schedule_oneshot_event(blk_get_aio_context(blk),
+                                         blk_aio_complete_bh, acb);
+    }
+
+    return &acb->common;
+}
+
 /*
  * Send a zone_report command.
  * offset is a byte offset from the start of the device. No alignment
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_zone_mgmt(BlockBackend *blk, BlockZoneOp op,
     return ret;
 }
 
+/*
+ * Send a zone_append command.
+ */
+int coroutine_fn blk_co_zone_append(BlockBackend *blk, int64_t *offset,
+        QEMUIOVector *qiov, BdrvRequestFlags flags)
+{
+    int ret;
+    IO_CODE();
+
+    blk_inc_in_flight(blk);
+    blk_wait_while_drained(blk);
+    if (!blk_is_available(blk)) {
+        blk_dec_in_flight(blk);
+        return -ENOMEDIUM;
+    }
+
+    ret = bdrv_co_zone_append(blk_bs(blk), offset, qiov, flags);
+    blk_dec_in_flight(blk);
+    return ret;
+}
+
 void blk_drain(BlockBackend *blk)
 {
     BlockDriverState *bs = blk_bs(blk);
diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVRawState {
     bool has_write_zeroes:1;
     bool use_linux_aio:1;
     bool use_linux_io_uring:1;
+    int64_t *offset; /* offset of zone append operation */
     int page_cache_inconsistent; /* errno from fdatasync failure */
     bool has_fallocate;
     bool needs_alignment;
@@ -XXX,XX +XXX,XX @@ static ssize_t handle_aiocb_rw_vector(RawPosixAIOData *aiocb)
     ssize_t len;
 
     len = RETRY_ON_EINTR(
-        (aiocb->aio_type & QEMU_AIO_WRITE) ?
+        (aiocb->aio_type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) ?
             qemu_pwritev(aiocb->aio_fildes,
                            aiocb->io.iov,
                            aiocb->io.niov,
@@ -XXX,XX +XXX,XX @@ static ssize_t handle_aiocb_rw_linear(RawPosixAIOData *aiocb, char *buf)
     ssize_t len;
 
     while (offset < aiocb->aio_nbytes) {
-        if (aiocb->aio_type & QEMU_AIO_WRITE) {
+        if (aiocb->aio_type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) {
             len = pwrite(aiocb->aio_fildes,
                          (const char *)buf + offset,
                          aiocb->aio_nbytes - offset,
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_rw(void *opaque)
     }
 
     nbytes = handle_aiocb_rw_linear(aiocb, buf);
-    if (!(aiocb->aio_type & QEMU_AIO_WRITE)) {
+    if (!(aiocb->aio_type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND))) {
         char *p = buf;
         size_t count = aiocb->aio_nbytes, copy;
         int i;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
     if (fd_open(bs) < 0)
         return -EIO;
 #if defined(CONFIG_BLKZONED)
-    if (type & QEMU_AIO_WRITE && bs->wps) {
+    if ((type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) && bs->wps) {
         qemu_co_mutex_lock(&bs->wps->colock);
+        if (type & QEMU_AIO_ZONE_APPEND && bs->bl.zone_size) {
+            int index = offset / bs->bl.zone_size;
+            offset = bs->wps->wp[index];
+        }
     }
 #endif
 
@@ -XXX,XX +XXX,XX @@ out:
 #if defined(CONFIG_BLKZONED)
     BlockZoneWps *wps = bs->wps;
     if (ret == 0) {
-        if (type & QEMU_AIO_WRITE && wps && bs->bl.zone_size) {
+        if ((type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND))
+            && wps && bs->bl.zone_size) {
             uint64_t *wp = &wps->wp[offset / bs->bl.zone_size];
             if (!BDRV_ZT_IS_CONV(*wp)) {
+                if (type & QEMU_AIO_ZONE_APPEND) {
+                    *s->offset = *wp;
+                }
                 /* Advance the wp if needed */
                 if (offset + bytes > *wp) {
                     *wp = offset + bytes;
@@ -XXX,XX +XXX,XX @@ out:
             }
         }
     } else {
-        if (type & QEMU_AIO_WRITE) {
+        if (type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) {
             update_zones_wp(bs, s->fd, 0, 1);
         }
     }
 
-    if (type & QEMU_AIO_WRITE && wps) {
+    if ((type & (QEMU_AIO_WRITE | QEMU_AIO_ZONE_APPEND)) && wps) {
         qemu_co_mutex_unlock(&wps->colock);
     }
 #endif
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
 }
 #endif
 
+#if defined(CONFIG_BLKZONED)
+static int coroutine_fn raw_co_zone_append(BlockDriverState *bs,
+                                           int64_t *offset,
+                                           QEMUIOVector *qiov,
+                                           BdrvRequestFlags flags) {
+    assert(flags == 0);
+    int64_t zone_size_mask = bs->bl.zone_size - 1;
+    int64_t iov_len = 0;
+    int64_t len = 0;
+    BDRVRawState *s = bs->opaque;
+    s->offset = offset;
+
+    if (*offset & zone_size_mask) {
+        error_report("sector offset %" PRId64 " is not aligned to zone size "
+                     "%" PRId32 "", *offset / 512, bs->bl.zone_size / 512);
+        return -EINVAL;
+    }
+
+    int64_t wg = bs->bl.write_granularity;
+    int64_t wg_mask = wg - 1;
+    for (int i = 0; i < qiov->niov; i++) {
+        iov_len = qiov->iov[i].iov_len;
+        if (iov_len & wg_mask) {
+            error_report("len of IOVector[%d] %" PRId64 " is not aligned to "
+                         "block size %" PRId64 "", i, iov_len, wg);
+            return -EINVAL;
+        }
+        len += iov_len;
+    }
+
+    return raw_co_prw(bs, *offset, len, qiov, QEMU_AIO_ZONE_APPEND);
+}
+#endif
+
 static coroutine_fn int
 raw_do_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes,
                 bool blkdev)
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_host_device = {
     /* zone management operations */
     .bdrv_co_zone_report = raw_co_zone_report,
     .bdrv_co_zone_mgmt = raw_co_zone_mgmt,
+    .bdrv_co_zone_append = raw_co_zone_append,
 #endif
 };
 
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ out:
     return co.ret;
 }
 
+int coroutine_fn bdrv_co_zone_append(BlockDriverState *bs, int64_t *offset,
+                        QEMUIOVector *qiov,
+                        BdrvRequestFlags flags)
+{
+    int ret;
+    BlockDriver *drv = bs->drv;
+    CoroutineIOCompletion co = {
+            .coroutine = qemu_coroutine_self(),
+    };
+    IO_CODE();
+
+    ret = bdrv_check_qiov_request(*offset, qiov->size, qiov, 0, NULL);
+    if (ret < 0) {
+        return ret;
+    }
+
+    bdrv_inc_in_flight(bs);
+    if (!drv || !drv->bdrv_co_zone_append || bs->bl.zoned == BLK_Z_NONE) {
+        co.ret = -ENOTSUP;
+        goto out;
+    }
+    co.ret = drv->bdrv_co_zone_append(bs, offset, qiov, flags);
+out:
+    bdrv_dec_in_flight(bs);
+    return co.ret;
+}
+
 void *qemu_blockalign(BlockDriverState *bs, size_t size)
 {
     IO_CODE();
diff --git a/block/io_uring.c b/block/io_uring.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io_uring.c
+++ b/block/io_uring.c
@@ -XXX,XX +XXX,XX @@ static int luring_do_submit(int fd, LuringAIOCB *luringcb, LuringState *s,
         io_uring_prep_writev(sqes, fd, luringcb->qiov->iov,
                              luringcb->qiov->niov, offset);
         break;
+    case QEMU_AIO_ZONE_APPEND:
+        io_uring_prep_writev(sqes, fd, luringcb->qiov->iov,
+                             luringcb->qiov->niov, offset);
+        break;
     case QEMU_AIO_READ:
         io_uring_prep_readv(sqes, fd, luringcb->qiov->iov,
                             luringcb->qiov->niov, offset);
diff --git a/block/linux-aio.c b/block/linux-aio.c
index XXXXXXX..XXXXXXX 100644
--- a/block/linux-aio.c
+++ b/block/linux-aio.c
@@ -XXX,XX +XXX,XX @@ static int laio_do_submit(int fd, struct qemu_laiocb *laiocb, off_t offset,
     case QEMU_AIO_WRITE:
         io_prep_pwritev(iocbs, fd, qiov->iov, qiov->niov, offset);
         break;
+    case QEMU_AIO_ZONE_APPEND:
+        io_prep_pwritev(iocbs, fd, qiov->iov, qiov->niov, offset);
+        break;
     case QEMU_AIO_READ:
         io_prep_preadv(iocbs, fd, qiov->iov, qiov->niov, offset);
         break;
diff --git a/block/raw-format.c b/block/raw-format.c
index XXXXXXX..XXXXXXX 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -XXX,XX +XXX,XX @@ raw_co_zone_mgmt(BlockDriverState *bs, BlockZoneOp op,
     return bdrv_co_zone_mgmt(bs->file->bs, op, offset, len);
 }
 
+static int coroutine_fn GRAPH_RDLOCK
+raw_co_zone_append(BlockDriverState *bs,int64_t *offset, QEMUIOVector *qiov,
+                   BdrvRequestFlags flags)
+{
+    return bdrv_co_zone_append(bs->file->bs, offset, qiov, flags);
+}
+
 static int64_t coroutine_fn GRAPH_RDLOCK
 raw_co_getlength(BlockDriverState *bs)
 {
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_raw = {
     .bdrv_co_pdiscard     = &raw_co_pdiscard,
     .bdrv_co_zone_report  = &raw_co_zone_report,
     .bdrv_co_zone_mgmt  = &raw_co_zone_mgmt,
+    .bdrv_co_zone_append = &raw_co_zone_append,
     .bdrv_co_block_status = &raw_co_block_status,
     .bdrv_co_copy_range_from = &raw_co_copy_range_from,
     .bdrv_co_copy_range_to  = &raw_co_copy_range_to,
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

The patch tests zone append writes by reporting the zone wp after
the completion of the call. "zap -p" option can print the sector
offset value after completion, which should be the start sector
where the append write begins.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20230407081657.17947-4-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qemu-io-cmds.c                     | 75 ++++++++++++++++++++++++++++++
 tests/qemu-iotests/tests/zoned     | 16 +++++++
 tests/qemu-iotests/tests/zoned.out | 16 +++++++
 3 files changed, 107 insertions(+)

diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t zone_reset_cmd = {
     .oneline = "reset a zone write pointer in zone block device",
 };
 
+static int do_aio_zone_append(BlockBackend *blk, QEMUIOVector *qiov,
+                              int64_t *offset, int flags, int *total)
+{
+    int async_ret = NOT_DONE;
+
+    blk_aio_zone_append(blk, offset, qiov, flags, aio_rw_done, &async_ret);
+    while (async_ret == NOT_DONE) {
+        main_loop_wait(false);
+    }
+
+    *total = qiov->size;
+    return async_ret < 0 ? async_ret : 1;
+}
+
+static int zone_append_f(BlockBackend *blk, int argc, char **argv)
+{
+    int ret;
+    bool pflag = false;
+    int flags = 0;
+    int total = 0;
+    int64_t offset;
+    char *buf;
+    int c, nr_iov;
+    int pattern = 0xcd;
+    QEMUIOVector qiov;
+
+    if (optind > argc - 3) {
+        return -EINVAL;
+    }
+
+    if ((c = getopt(argc, argv, "p")) != -1) {
+        pflag = true;
+    }
+
+    offset = cvtnum(argv[optind]);
+    if (offset < 0) {
+        print_cvtnum_err(offset, argv[optind]);
+        return offset;
+    }
+    optind++;
+    nr_iov = argc - optind;
+    buf = create_iovec(blk, &qiov, &argv[optind], nr_iov, pattern,
+                       flags & BDRV_REQ_REGISTERED_BUF);
+    if (buf == NULL) {
+        return -EINVAL;
+    }
+    ret = do_aio_zone_append(blk, &qiov, &offset, flags, &total);
+    if (ret < 0) {
+        printf("zone append failed: %s\n", strerror(-ret));
+        goto out;
+    }
+
+    if (pflag) {
+        printf("After zap done, the append sector is 0x%" PRIx64 "\n",
+               tosector(offset));
+    }
+
+out:
+    qemu_io_free(blk, buf, qiov.size,
+                 flags & BDRV_REQ_REGISTERED_BUF);
+    qemu_iovec_destroy(&qiov);
+    return ret;
+}
+
+static const cmdinfo_t zone_append_cmd = {
+    .name = "zone_append",
+    .altname = "zap",
+    .cfunc = zone_append_f,
+    .argmin = 3,
+    .argmax = 4,
+    .args = "offset len [len..]",
+    .oneline = "append write a number of bytes at a specified offset",
+};
+
 static int truncate_f(BlockBackend *blk, int argc, char **argv);
 static const cmdinfo_t truncate_cmd = {
     .name       = "truncate",
@@ -XXX,XX +XXX,XX @@ static void __attribute((constructor)) init_qemuio_commands(void)
     qemuio_add_command(&zone_close_cmd);
     qemuio_add_command(&zone_finish_cmd);
     qemuio_add_command(&zone_reset_cmd);
+    qemuio_add_command(&zone_append_cmd);
     qemuio_add_command(&truncate_cmd);
     qemuio_add_command(&length_cmd);
     qemuio_add_command(&info_cmd);
diff --git a/tests/qemu-iotests/tests/zoned b/tests/qemu-iotests/tests/zoned
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/tests/zoned
+++ b/tests/qemu-iotests/tests/zoned
@@ -XXX,XX +XXX,XX @@ echo "(5) resetting the second zone"
 $QEMU_IO $IMG -c "zrs 268435456 268435456"
 echo "After resetting a zone:"
 $QEMU_IO $IMG -c "zrp 268435456 1"
+echo
+echo
+echo "(6) append write" # the physical block size of the device is 4096
+$QEMU_IO $IMG -c "zrp 0 1"
+$QEMU_IO $IMG -c "zap -p 0 0x1000 0x2000"
+echo "After appending the first zone firstly:"
+$QEMU_IO $IMG -c "zrp 0 1"
+$QEMU_IO $IMG -c "zap -p 0 0x1000 0x2000"
+echo "After appending the first zone secondly:"
+$QEMU_IO $IMG -c "zrp 0 1"
+$QEMU_IO $IMG -c "zap -p 268435456 0x1000 0x2000"
+echo "After appending the second zone firstly:"
+$QEMU_IO $IMG -c "zrp 268435456 1"
+$QEMU_IO $IMG -c "zap -p 268435456 0x1000 0x2000"
+echo "After appending the second zone secondly:"
+$QEMU_IO $IMG -c "zrp 268435456 1"
 
 # success, all done
 echo "*** done"
diff --git a/tests/qemu-iotests/tests/zoned.out b/tests/qemu-iotests/tests/zoned.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/tests/zoned.out
+++ b/tests/qemu-iotests/tests/zoned.out
@@ -XXX,XX +XXX,XX @@ start: 0x80000, len 0x80000, cap 0x80000, wptr 0x100000, zcond:14, [type: 2]
 (5) resetting the second zone
 After resetting a zone:
 start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80000, zcond:1, [type: 2]
+
+
+(6) append write
+start: 0x0, len 0x80000, cap 0x80000, wptr 0x0, zcond:1, [type: 2]
+After zap done, the append sector is 0x0
+After appending the first zone firstly:
+start: 0x0, len 0x80000, cap 0x80000, wptr 0x18, zcond:2, [type: 2]
+After zap done, the append sector is 0x18
+After appending the first zone secondly:
+start: 0x0, len 0x80000, cap 0x80000, wptr 0x30, zcond:2, [type: 2]
+After zap done, the append sector is 0x80000
+After appending the second zone firstly:
+start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80018, zcond:2, [type: 2]
+After zap done, the append sector is 0x80018
+After appending the second zone secondly:
+start: 0x80000, len 0x80000, cap 0x80000, wptr 0x80030, zcond:2, [type: 2]
 *** done
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ out:
             if (!BDRV_ZT_IS_CONV(*wp)) {
                 if (type & QEMU_AIO_ZONE_APPEND) {
                     *s->offset = *wp;
+                    trace_zbd_zone_append_complete(bs, *s->offset
+                        >> BDRV_SECTOR_BITS);
                 }
                 /* Advance the wp if needed */
                 if (offset + bytes > *wp) {
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn raw_co_zone_append(BlockDriverState *bs,
         len += iov_len;
     }
 
+    trace_zbd_zone_append(bs, *offset >> BDRV_SECTOR_BITS);
     return raw_co_prw(bs, *offset, len, qiov, QEMU_AIO_ZONE_APPEND);
 }
 #endif
diff --git a/block/trace-events b/block/trace-events
index XXXXXXX..XXXXXXX 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -XXX,XX +XXX,XX @@ file_hdev_is_sg(int type, int version) "SG device found: type=%d, version=%d"
 file_flush_fdatasync_failed(int err) "errno %d"
 zbd_zone_report(void *bs, unsigned int nr_zones, int64_t sector) "bs %p report %d zones starting at sector offset 0x%" PRIx64 ""
 zbd_zone_mgmt(void *bs, const char *op_name, int64_t sector, int64_t len) "bs %p %s starts at sector offset 0x%" PRIx64 " over a range of 0x%" PRIx64 " sectors"
+zbd_zone_append(void *bs, int64_t sector) "bs %p append at sector offset 0x%" PRIx64 ""
+zbd_zone_append_complete(void *bs, int64_t sector) "bs %p returns append sector 0x%" PRIx64 ""
 
 # ssh.c
 sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Use scripts/update-linux-headers.sh to update headers to 6.3-rc1.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Message-id: 20230407082528.18841-2-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/standard-headers/drm/drm_fourcc.h    |  12 +++
 include/standard-headers/linux/ethtool.h     |  48 ++++++++-
 include/standard-headers/linux/fuse.h        |  45 +++++++-
 include/standard-headers/linux/pci_regs.h    |   1 +
 include/standard-headers/linux/vhost_types.h |   2 +
 include/standard-headers/linux/virtio_blk.h  | 105 +++++++++++++++++++
 linux-headers/asm-arm64/kvm.h                |   1 +
 linux-headers/asm-x86/kvm.h                  |  34 +++++-
 linux-headers/linux/kvm.h                    |   9 ++
 linux-headers/linux/vfio.h                   |  15 +--
 linux-headers/linux/vhost.h                  |   8 ++
 11 files changed, 270 insertions(+), 10 deletions(-)

diff --git a/include/standard-headers/drm/drm_fourcc.h b/include/standard-headers/drm/drm_fourcc.h
index XXXXXXX..XXXXXXX 100644
--- a/include/standard-headers/drm/drm_fourcc.h
+++ b/include/standard-headers/drm/drm_fourcc.h
@@ -XXX,XX +XXX,XX @@ extern "C" {
  *
  * The authoritative list of format modifier codes is found in
  * `include/uapi/drm/drm_fourcc.h`
+ *
+ * Open Source User Waiver
+ * -----------------------
+ *
+ * Because this is the authoritative source for pixel formats and modifiers
+ * referenced by GL, Vulkan extensions and other standards and hence used both
+ * by open source and closed source driver stacks, the usual requirement for an
+ * upstream in-kernel or open source userspace user does not apply.
+ *
+ * To ensure, as much as feasible, compatibility across stacks and avoid
+ * confusion with incompatible enumerations stakeholders for all relevant driver
+ * stacks should approve additions.
  */
 
 #define fourcc_code(a, b, c, d) ((uint32_t)(a) | ((uint32_t)(b) << 8) | \
diff --git a/include/standard-headers/linux/ethtool.h b/include/standard-headers/linux/ethtool.h
index XXXXXXX..XXXXXXX 100644
--- a/include/standard-headers/linux/ethtool.h
+++ b/include/standard-headers/linux/ethtool.h
@@ -XXX,XX +XXX,XX @@ enum ethtool_stringset {
 	ETH_SS_COUNT
 };
 
+/**
+ * enum ethtool_mac_stats_src - source of ethtool MAC statistics
+ * @ETHTOOL_MAC_STATS_SRC_AGGREGATE:
+ *	if device supports a MAC merge layer, this retrieves the aggregate
+ *	statistics of the eMAC and pMAC. Otherwise, it retrieves just the
+ *	statistics of the single (express) MAC.
+ * @ETHTOOL_MAC_STATS_SRC_EMAC:
+ *	if device supports a MM layer, this retrieves the eMAC statistics.
+ *	Otherwise, it retrieves the statistics of the single (express) MAC.
+ * @ETHTOOL_MAC_STATS_SRC_PMAC:
+ *	if device supports a MM layer, this retrieves the pMAC statistics.
+ */
+enum ethtool_mac_stats_src {
+	ETHTOOL_MAC_STATS_SRC_AGGREGATE,
+	ETHTOOL_MAC_STATS_SRC_EMAC,
+	ETHTOOL_MAC_STATS_SRC_PMAC,
+};
+
 /**
  * enum ethtool_module_power_mode_policy - plug-in module power mode policy
  * @ETHTOOL_MODULE_POWER_MODE_POLICY_HIGH: Module is always in high power mode.
@@ -XXX,XX +XXX,XX @@ enum ethtool_podl_pse_pw_d_status {
 	ETHTOOL_PODL_PSE_PW_D_STATUS_ERROR,
 };
 
+/**
+ * enum ethtool_mm_verify_status - status of MAC Merge Verify function
+ * @ETHTOOL_MM_VERIFY_STATUS_UNKNOWN:
+ *	verification status is unknown
+ * @ETHTOOL_MM_VERIFY_STATUS_INITIAL:
+ *	the 802.3 Verify State diagram is in the state INIT_VERIFICATION
+ * @ETHTOOL_MM_VERIFY_STATUS_VERIFYING:
+ *	the Verify State diagram is in the state VERIFICATION_IDLE,
+ *	SEND_VERIFY or WAIT_FOR_RESPONSE
+ * @ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED:
+ *	indicates that the Verify State diagram is in the state VERIFIED
+ * @ETHTOOL_MM_VERIFY_STATUS_FAILED:
+ *	the Verify State diagram is in the state VERIFY_FAIL
+ * @ETHTOOL_MM_VERIFY_STATUS_DISABLED:
+ *	verification of preemption operation is disabled
+ */
+enum ethtool_mm_verify_status {
+	ETHTOOL_MM_VERIFY_STATUS_UNKNOWN,
+	ETHTOOL_MM_VERIFY_STATUS_INITIAL,
+	ETHTOOL_MM_VERIFY_STATUS_VERIFYING,
+	ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED,
+	ETHTOOL_MM_VERIFY_STATUS_FAILED,
+	ETHTOOL_MM_VERIFY_STATUS_DISABLED,
+};
+
 /**
  * struct ethtool_gstrings - string set for data tagging
  * @cmd: Command number = %ETHTOOL_GSTRINGS
@@ -XXX,XX +XXX,XX @@ struct ethtool_rxnfc {
 		uint32_t			rule_cnt;
 		uint32_t			rss_context;
 	};
-	uint32_t				rule_locs[0];
+	uint32_t				rule_locs[];
 };
 
 
@@ -XXX,XX +XXX,XX @@ enum ethtool_link_mode_bit_indices {
 	ETHTOOL_LINK_MODE_800000baseDR8_2_Full_BIT	 = 96,
 	ETHTOOL_LINK_MODE_800000baseSR8_Full_BIT	 = 97,
 	ETHTOOL_LINK_MODE_800000baseVR8_Full_BIT	 = 98,
+	ETHTOOL_LINK_MODE_10baseT1S_Full_BIT		 = 99,
+	ETHTOOL_LINK_MODE_10baseT1S_Half_BIT		 = 100,
+	ETHTOOL_LINK_MODE_10baseT1S_P2MP_Half_BIT	 = 101,
 
 	/* must be last entry */
 	__ETHTOOL_LINK_MODE_MASK_NBITS
diff --git a/include/standard-headers/linux/fuse.h b/include/standard-headers/linux/fuse.h
index XXXXXXX..XXXXXXX 100644
--- a/include/standard-headers/linux/fuse.h
+++ b/include/standard-headers/linux/fuse.h
@@ -XXX,XX +XXX,XX @@
  *  7.38
  *  - add FUSE_EXPIRE_ONLY flag to fuse_notify_inval_entry
  *  - add FOPEN_PARALLEL_DIRECT_WRITES
+ *  - add total_extlen to fuse_in_header
+ *  - add FUSE_MAX_NR_SECCTX
+ *  - add extension header
+ *  - add FUSE_EXT_GROUPS
+ *  - add FUSE_CREATE_SUPP_GROUP
  */
 
 #ifndef _LINUX_FUSE_H
@@ -XXX,XX +XXX,XX @@ struct fuse_file_lock {
  * FUSE_SECURITY_CTX:	add security context to create, mkdir, symlink, and
  *			mknod
  * FUSE_HAS_INODE_DAX:  use per inode DAX
+ * FUSE_CREATE_SUPP_GROUP: add supplementary group info to create, mkdir,
+ *			symlink and mknod (single group that matches parent)
  */
 #define FUSE_ASYNC_READ		(1 << 0)
 #define FUSE_POSIX_LOCKS	(1 << 1)
@@ -XXX,XX +XXX,XX @@ struct fuse_file_lock {
 /* bits 32..63 get shifted down 32 bits into the flags2 field */
 #define FUSE_SECURITY_CTX	(1ULL << 32)
 #define FUSE_HAS_INODE_DAX	(1ULL << 33)
+#define FUSE_CREATE_SUPP_GROUP	(1ULL << 34)
 
 /**
  * CUSE INIT request/reply flags
@@ -XXX,XX +XXX,XX @@ struct fuse_file_lock {
  */
 #define FUSE_EXPIRE_ONLY		(1 << 0)
 
+/**
+ * extension type
+ * FUSE_MAX_NR_SECCTX: maximum value of &fuse_secctx_header.nr_secctx
+ * FUSE_EXT_GROUPS: &fuse_supp_groups extension
+ */
+enum fuse_ext_type {
+	/* Types 0..31 are reserved for fuse_secctx_header */
+	FUSE_MAX_NR_SECCTX	= 31,
+	FUSE_EXT_GROUPS		= 32,
+};
+
 enum fuse_opcode {
 	FUSE_LOOKUP		= 1,
 	FUSE_FORGET		= 2,  /* no reply */
@@ -XXX,XX +XXX,XX @@ struct fuse_in_header {
 	uint32_t	uid;
 	uint32_t	gid;
 	uint32_t	pid;
-	uint32_t	padding;
+	uint16_t	total_extlen; /* length of extensions in 8byte units */
+	uint16_t	padding;
 };
 
 struct fuse_out_header {
@@ -XXX,XX +XXX,XX @@ struct fuse_secctx_header {
 	uint32_t	nr_secctx;
 };
 
+/**
+ * struct fuse_ext_header - extension header
+ * @size: total size of this extension including this header
+ * @type: type of extension
+ *
+ * This is made compatible with fuse_secctx_header by using type values >
+ * FUSE_MAX_NR_SECCTX
+ */
+struct fuse_ext_header {
+	uint32_t	size;
+	uint32_t	type;
+};
+
+/**
+ * struct fuse_supp_groups - Supplementary group extension
+ * @nr_groups: number of supplementary groups
+ * @groups: flexible array of group IDs
+ */
+struct fuse_supp_groups {
+	uint32_t	nr_groups;
+	uint32_t	groups[];
+};
+
 #endif /* _LINUX_FUSE_H */
diff --git a/include/standard-headers/linux/pci_regs.h b/include/standard-headers/linux/pci_regs.h
index XXXXXXX..XXXXXXX 100644
--- a/include/standard-headers/linux/pci_regs.h
+++ b/include/standard-headers/linux/pci_regs.h
@@ -XXX,XX +XXX,XX @@
 #define  PCI_EXP_LNKCTL2_TX_MARGIN	0x0380 /* Transmit Margin */
 #define  PCI_EXP_LNKCTL2_HASD		0x0020 /* HW Autonomous Speed Disable */
 #define PCI_EXP_LNKSTA2		0x32	/* Link Status 2 */
+#define  PCI_EXP_LNKSTA2_FLIT		0x0400 /* Flit Mode Status */
 #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2	0x32	/* end of v2 EPs w/ link */
 #define PCI_EXP_SLTCAP2		0x34	/* Slot Capabilities 2 */
 #define  PCI_EXP_SLTCAP2_IBPD	0x00000001 /* In-band PD Disable Supported */
diff --git a/include/standard-headers/linux/vhost_types.h b/include/standard-headers/linux/vhost_types.h
index XXXXXXX..XXXXXXX 100644
--- a/include/standard-headers/linux/vhost_types.h
+++ b/include/standard-headers/linux/vhost_types.h
@@ -XXX,XX +XXX,XX @@ struct vhost_vdpa_iova_range {
 #define VHOST_BACKEND_F_IOTLB_ASID  0x3
 /* Device can be suspended */
 #define VHOST_BACKEND_F_SUSPEND  0x4
+/* Device can be resumed */
+#define VHOST_BACKEND_F_RESUME  0x5
 
 #endif
diff --git a/include/standard-headers/linux/virtio_blk.h b/include/standard-headers/linux/virtio_blk.h
index XXXXXXX..XXXXXXX 100644
--- a/include/standard-headers/linux/virtio_blk.h
+++ b/include/standard-headers/linux/virtio_blk.h
@@ -XXX,XX +XXX,XX @@
 #define VIRTIO_BLK_F_DISCARD	13	/* DISCARD is supported */
 #define VIRTIO_BLK_F_WRITE_ZEROES	14	/* WRITE ZEROES is supported */
 #define VIRTIO_BLK_F_SECURE_ERASE	16 /* Secure Erase is supported */
+#define VIRTIO_BLK_F_ZONED		17	/* Zoned block device */
 
 /* Legacy feature bits */
 #ifndef VIRTIO_BLK_NO_LEGACY
@@ -XXX,XX +XXX,XX @@ struct virtio_blk_config {
 	/* Secure erase commands must be aligned to this number of sectors. */
 	__virtio32 secure_erase_sector_alignment;
 
+	/* Zoned block device characteristics (if VIRTIO_BLK_F_ZONED) */
+	struct virtio_blk_zoned_characteristics {
+		uint32_t zone_sectors;
+		uint32_t max_open_zones;
+		uint32_t max_active_zones;
+		uint32_t max_append_sectors;
+		uint32_t write_granularity;
+		uint8_t model;
+		uint8_t unused2[3];
+	} zoned;
 } QEMU_PACKED;
 
 /*
@@ -XXX,XX +XXX,XX @@ struct virtio_blk_config {
 /* Secure erase command */
 #define VIRTIO_BLK_T_SECURE_ERASE	14
 
+/* Zone append command */
+#define VIRTIO_BLK_T_ZONE_APPEND    15
+
+/* Report zones command */
+#define VIRTIO_BLK_T_ZONE_REPORT    16
+
+/* Open zone command */
+#define VIRTIO_BLK_T_ZONE_OPEN      18
+
+/* Close zone command */
+#define VIRTIO_BLK_T_ZONE_CLOSE     20
+
+/* Finish zone command */
+#define VIRTIO_BLK_T_ZONE_FINISH    22
+
+/* Reset zone command */
+#define VIRTIO_BLK_T_ZONE_RESET     24
+
+/* Reset All zones command */
+#define VIRTIO_BLK_T_ZONE_RESET_ALL 26
+
 #ifndef VIRTIO_BLK_NO_LEGACY
 /* Barrier before this op. */
 #define VIRTIO_BLK_T_BARRIER	0x80000000
@@ -XXX,XX +XXX,XX @@ struct virtio_blk_outhdr {
 	__virtio64 sector;
 };
 
+/*
+ * Supported zoned device models.
+ */
+
+/* Regular block device */
+#define VIRTIO_BLK_Z_NONE      0
+/* Host-managed zoned device */
+#define VIRTIO_BLK_Z_HM        1
+/* Host-aware zoned device */
+#define VIRTIO_BLK_Z_HA        2
+
+/*
+ * Zone descriptor. A part of VIRTIO_BLK_T_ZONE_REPORT command reply.
+ */
+struct virtio_blk_zone_descriptor {
+	/* Zone capacity */
+	uint64_t z_cap;
+	/* The starting sector of the zone */
+	uint64_t z_start;
+	/* Zone write pointer position in sectors */
+	uint64_t z_wp;
+	/* Zone type */
+	uint8_t z_type;
+	/* Zone state */
+	uint8_t z_state;
+	uint8_t reserved[38];
+};
+
+struct virtio_blk_zone_report {
+	uint64_t nr_zones;
+	uint8_t reserved[56];
+	struct virtio_blk_zone_descriptor zones[];
+};
+
+/*
+ * Supported zone types.
+ */
+
+/* Conventional zone */
+#define VIRTIO_BLK_ZT_CONV         1
+/* Sequential Write Required zone */
+#define VIRTIO_BLK_ZT_SWR          2
+/* Sequential Write Preferred zone */
+#define VIRTIO_BLK_ZT_SWP          3
+
+/*
+ * Zone states that are available for zones of all types.
+ */
+
+/* Not a write pointer (conventional zones only) */
+#define VIRTIO_BLK_ZS_NOT_WP       0
+/* Empty */
+#define VIRTIO_BLK_ZS_EMPTY        1
+/* Implicitly Open */
+#define VIRTIO_BLK_ZS_IOPEN        2
+/* Explicitly Open */
+#define VIRTIO_BLK_ZS_EOPEN        3
+/* Closed */
+#define VIRTIO_BLK_ZS_CLOSED       4
+/* Read-Only */
+#define VIRTIO_BLK_ZS_RDONLY       13
+/* Full */
+#define VIRTIO_BLK_ZS_FULL         14
+/* Offline */
+#define VIRTIO_BLK_ZS_OFFLINE      15
+
 /* Unmap this range (only valid for write zeroes command) */
 #define VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP	0x00000001
 
@@ -XXX,XX +XXX,XX @@ struct virtio_scsi_inhdr {
 #define VIRTIO_BLK_S_OK		0
 #define VIRTIO_BLK_S_IOERR	1
 #define VIRTIO_BLK_S_UNSUPP	2
+
+/* Error codes that are specific to zoned block devices */
+#define VIRTIO_BLK_S_ZONE_INVALID_CMD     3
+#define VIRTIO_BLK_S_ZONE_UNALIGNED_WP    4
+#define VIRTIO_BLK_S_ZONE_OPEN_RESOURCE   5
+#define VIRTIO_BLK_S_ZONE_ACTIVE_RESOURCE 6
+
 #endif /* _LINUX_VIRTIO_BLK_H */
diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h
index XXXXXXX..XXXXXXX 100644
--- a/linux-headers/asm-arm64/kvm.h
+++ b/linux-headers/asm-arm64/kvm.h
@@ -XXX,XX +XXX,XX @@ struct kvm_regs {
 #define KVM_ARM_VCPU_SVE		4 /* enable SVE for this CPU */
 #define KVM_ARM_VCPU_PTRAUTH_ADDRESS	5 /* VCPU uses address authentication */
 #define KVM_ARM_VCPU_PTRAUTH_GENERIC	6 /* VCPU uses generic authentication */
+#define KVM_ARM_VCPU_HAS_EL2		7 /* Support nested virtualization */
 
 struct kvm_vcpu_init {
 	__u32 target;
diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
index XXXXXXX..XXXXXXX 100644
--- a/linux-headers/asm-x86/kvm.h
+++ b/linux-headers/asm-x86/kvm.h
@@ -XXX,XX +XXX,XX @@
 
 #include <linux/types.h>
 #include <linux/ioctl.h>
+#include <linux/stddef.h>
 
 #define KVM_PIO_PAGE_OFFSET 1
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 2
@@ -XXX,XX +XXX,XX @@ struct kvm_nested_state {
 	 * KVM_{GET,PUT}_NESTED_STATE ioctl values.
 	 */
 	union {
-		struct kvm_vmx_nested_state_data vmx[0];
-		struct kvm_svm_nested_state_data svm[0];
+		__DECLARE_FLEX_ARRAY(struct kvm_vmx_nested_state_data, vmx);
+		__DECLARE_FLEX_ARRAY(struct kvm_svm_nested_state_data, svm);
 	} data;
 };
 
@@ -XXX,XX +XXX,XX @@ struct kvm_pmu_event_filter {
 #define KVM_PMU_EVENT_ALLOW 0
 #define KVM_PMU_EVENT_DENY 1
 
+#define KVM_PMU_EVENT_FLAG_MASKED_EVENTS BIT(0)
+#define KVM_PMU_EVENT_FLAGS_VALID_MASK (KVM_PMU_EVENT_FLAG_MASKED_EVENTS)
+
+/*
+ * Masked event layout.
+ * Bits   Description
+ * ----   -----------
+ * 7:0    event select (low bits)
+ * 15:8   umask match
+ * 31:16  unused
+ * 35:32  event select (high bits)
+ * 36:54  unused
+ * 55     exclude bit
+ * 63:56  umask mask
+ */
+
+#define KVM_PMU_ENCODE_MASKED_ENTRY(event_select, mask, match, exclude) \
+	(((event_select) & 0xFFULL) | (((event_select) & 0XF00ULL) << 24) | \
+	(((mask) & 0xFFULL) << 56) | \
+	(((match) & 0xFFULL) << 8) | \
+	((__u64)(!!(exclude)) << 55))
+
+#define KVM_PMU_MASKED_ENTRY_EVENT_SELECT \
+	(GENMASK_ULL(7, 0) | GENMASK_ULL(35, 32))
+#define KVM_PMU_MASKED_ENTRY_UMASK_MASK		(GENMASK_ULL(63, 56))
+#define KVM_PMU_MASKED_ENTRY_UMASK_MATCH	(GENMASK_ULL(15, 8))
+#define KVM_PMU_MASKED_ENTRY_EXCLUDE		(BIT_ULL(55))
+#define KVM_PMU_MASKED_ENTRY_UMASK_MASK_SHIFT	(56)
+
 /* for KVM_{GET,SET,HAS}_DEVICE_ATTR */
 #define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
 #define   KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index XXXXXXX..XXXXXXX 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -XXX,XX +XXX,XX @@ struct kvm_s390_mem_op {
 		struct {
 			__u8 ar;	/* the access register number */
 			__u8 key;	/* access key, ignored if flag unset */
+			__u8 pad1[6];	/* ignored */
+			__u64 old_addr;	/* ignored if cmpxchg flag unset */
 		};
 		__u32 sida_offset; /* offset into the sida */
 		__u8 reserved[32]; /* ignored */
@@ -XXX,XX +XXX,XX @@ struct kvm_s390_mem_op {
 #define KVM_S390_MEMOP_SIDA_WRITE	3
 #define KVM_S390_MEMOP_ABSOLUTE_READ	4
 #define KVM_S390_MEMOP_ABSOLUTE_WRITE	5
+#define KVM_S390_MEMOP_ABSOLUTE_CMPXCHG	6
+
 /* flags for kvm_s390_mem_op->flags */
 #define KVM_S390_MEMOP_F_CHECK_ONLY		(1ULL << 0)
 #define KVM_S390_MEMOP_F_INJECT_EXCEPTION	(1ULL << 1)
 #define KVM_S390_MEMOP_F_SKEY_PROTECTION	(1ULL << 2)
 
+/* flags specifying extension support via KVM_CAP_S390_MEM_OP_EXTENSION */
+#define KVM_S390_MEMOP_EXTENSION_CAP_BASE	(1 << 0)
+#define KVM_S390_MEMOP_EXTENSION_CAP_CMPXCHG	(1 << 1)
+
 /* for KVM_INTERRUPT */
 struct kvm_interrupt {
 	/* in */
@@ -XXX,XX +XXX,XX @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223
 #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
 #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
+#define KVM_CAP_PMU_EVENT_MASKED_EVENTS 226
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index XXXXXXX..XXXXXXX 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -XXX,XX +XXX,XX @@
 /* Supports VFIO_DMA_UNMAP_FLAG_ALL */
 #define VFIO_UNMAP_ALL			9
 
-/* Supports the vaddr flag for DMA map and unmap */
+/*
+ * Supports the vaddr flag for DMA map and unmap.  Not supported for mediated
+ * devices, so this capability is subject to change as groups are added or
+ * removed.
+ */
 #define VFIO_UPDATE_VADDR		10
 
 /*
@@ -XXX,XX +XXX,XX @@ struct vfio_iommu_type1_info_dma_avail {
  * Map process virtual addresses to IO virtual addresses using the
  * provided struct vfio_dma_map. Caller sets argsz. READ &/ WRITE required.
  *
- * If flags & VFIO_DMA_MAP_FLAG_VADDR, update the base vaddr for iova, and
- * unblock translation of host virtual addresses in the iova range.  The vaddr
+ * If flags & VFIO_DMA_MAP_FLAG_VADDR, update the base vaddr for iova. The vaddr
  * must have previously been invalidated with VFIO_DMA_UNMAP_FLAG_VADDR.  To
  * maintain memory consistency within the user application, the updated vaddr
  * must address the same memory object as originally mapped.  Failure to do so
@@ -XXX,XX +XXX,XX @@ struct vfio_bitmap {
  * must be 0.  This cannot be combined with the get-dirty-bitmap flag.
  *
  * If flags & VFIO_DMA_UNMAP_FLAG_VADDR, do not unmap, but invalidate host
- * virtual addresses in the iova range.  Tasks that attempt to translate an
- * iova's vaddr will block.  DMA to already-mapped pages continues.  This
- * cannot be combined with the get-dirty-bitmap flag.
+ * virtual addresses in the iova range.  DMA to already-mapped pages continues.
+ * Groups may not be added to the container while any addresses are invalid.
+ * This cannot be combined with the get-dirty-bitmap flag.
  */
 struct vfio_iommu_type1_dma_unmap {
 	__u32	argsz;
diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h
index XXXXXXX..XXXXXXX 100644
--- a/linux-headers/linux/vhost.h
+++ b/linux-headers/linux/vhost.h
@@ -XXX,XX +XXX,XX @@
  */
 #define VHOST_VDPA_SUSPEND		_IO(VHOST_VIRTIO, 0x7D)
 
+/* Resume a device so it can resume processing virtqueue requests
+ *
+ * After the return of this ioctl the device will have restored all the
+ * necessary states and it is fully operational to continue processing the
+ * virtqueue descriptors.
+ */
+#define VHOST_VDPA_RESUME		_IO(VHOST_VIRTIO, 0x7E)
+
 #endif
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

This patch extends virtio-blk emulation to handle zoned device commands
by calling the new block layer APIs to perform zoned device I/O on
behalf of the guest. It supports Report Zone, four zone oparations (open,
close, finish, reset), and Append Zone.

The VIRTIO_BLK_F_ZONED feature bit will only be set if the host does
support zoned block devices. Regular block devices(conventional zones)
will not be set.

The guest os can use blktests, fio to test those commands on zoned devices.
Furthermore, using zonefs to test zone append write is also supported.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Message-id: 20230407082528.18841-3-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/virtio-blk-common.c |   2 +
 hw/block/virtio-blk.c        | 389 +++++++++++++++++++++++++++++++++++
 hw/virtio/virtio-qmp.c       |   2 +
 3 files changed, 393 insertions(+)

diff --git a/hw/block/virtio-blk-common.c b/hw/block/virtio-blk-common.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/virtio-blk-common.c
+++ b/hw/block/virtio-blk-common.c
@@ -XXX,XX +XXX,XX @@ static const VirtIOFeature feature_sizes[] = {
      .end = endof(struct virtio_blk_config, discard_sector_alignment)},
     {.flags = 1ULL << VIRTIO_BLK_F_WRITE_ZEROES,
      .end = endof(struct virtio_blk_config, write_zeroes_may_unmap)},
+    {.flags = 1ULL << VIRTIO_BLK_F_ZONED,
+     .end = endof(struct virtio_blk_config, zoned)},
     {}
 };
 
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/module.h"
 #include "qemu/error-report.h"
 #include "qemu/main-loop.h"
+#include "block/block_int.h"
 #include "trace.h"
 #include "hw/block/block.h"
 #include "hw/qdev-properties.h"
@@ -XXX,XX +XXX,XX @@ err:
     return err_status;
 }
 
+typedef struct ZoneCmdData {
+    VirtIOBlockReq *req;
+    struct iovec *in_iov;
+    unsigned in_num;
+    union {
+        struct {
+            unsigned int nr_zones;
+            BlockZoneDescriptor *zones;
+        } zone_report_data;
+        struct {
+            int64_t offset;
+        } zone_append_data;
+    };
+} ZoneCmdData;
+
+/*
+ * check zoned_request: error checking before issuing requests. If all checks
+ * passed, return true.
+ * append: true if only zone append requests issued.
+ */
+static bool check_zoned_request(VirtIOBlock *s, int64_t offset, int64_t len,
+                             bool append, uint8_t *status) {
+    BlockDriverState *bs = blk_bs(s->blk);
+    int index;
+
+    if (!virtio_has_feature(s->host_features, VIRTIO_BLK_F_ZONED)) {
+        *status = VIRTIO_BLK_S_UNSUPP;
+        return false;
+    }
+
+    if (offset < 0 || len < 0 || len > (bs->total_sectors << BDRV_SECTOR_BITS)
+        || offset > (bs->total_sectors << BDRV_SECTOR_BITS) - len) {
+        *status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+        return false;
+    }
+
+    if (append) {
+        if (bs->bl.write_granularity) {
+            if ((offset % bs->bl.write_granularity) != 0) {
+                *status = VIRTIO_BLK_S_ZONE_UNALIGNED_WP;
+                return false;
+            }
+        }
+
+        index = offset / bs->bl.zone_size;
+        if (BDRV_ZT_IS_CONV(bs->wps->wp[index])) {
+            *status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+            return false;
+        }
+
+        if (len / 512 > bs->bl.max_append_sectors) {
+            if (bs->bl.max_append_sectors == 0) {
+                *status = VIRTIO_BLK_S_UNSUPP;
+            } else {
+                *status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+            }
+            return false;
+        }
+    }
+    return true;
+}
+
+static void virtio_blk_zone_report_complete(void *opaque, int ret)
+{
+    ZoneCmdData *data = opaque;
+    VirtIOBlockReq *req = data->req;
+    VirtIOBlock *s = req->dev;
+    VirtIODevice *vdev = VIRTIO_DEVICE(req->dev);
+    struct iovec *in_iov = data->in_iov;
+    unsigned in_num = data->in_num;
+    int64_t zrp_size, n, j = 0;
+    int64_t nz = data->zone_report_data.nr_zones;
+    int8_t err_status = VIRTIO_BLK_S_OK;
+
+    if (ret) {
+        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+        goto out;
+    }
+
+    struct virtio_blk_zone_report zrp_hdr = (struct virtio_blk_zone_report) {
+        .nr_zones = cpu_to_le64(nz),
+    };
+    zrp_size = sizeof(struct virtio_blk_zone_report)
+               + sizeof(struct virtio_blk_zone_descriptor) * nz;
+    n = iov_from_buf(in_iov, in_num, 0, &zrp_hdr, sizeof(zrp_hdr));
+    if (n != sizeof(zrp_hdr)) {
+        virtio_error(vdev, "Driver provided input buffer that is too small!");
+        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+        goto out;
+    }
+
+    for (size_t i = sizeof(zrp_hdr); i < zrp_size;
+        i += sizeof(struct virtio_blk_zone_descriptor), ++j) {
+        struct virtio_blk_zone_descriptor desc =
+            (struct virtio_blk_zone_descriptor) {
+                .z_start = cpu_to_le64(data->zone_report_data.zones[j].start
+                    >> BDRV_SECTOR_BITS),
+                .z_cap = cpu_to_le64(data->zone_report_data.zones[j].cap
+                    >> BDRV_SECTOR_BITS),
+                .z_wp = cpu_to_le64(data->zone_report_data.zones[j].wp
+                    >> BDRV_SECTOR_BITS),
+        };
+
+        switch (data->zone_report_data.zones[j].type) {
+        case BLK_ZT_CONV:
+            desc.z_type = VIRTIO_BLK_ZT_CONV;
+            break;
+        case BLK_ZT_SWR:
+            desc.z_type = VIRTIO_BLK_ZT_SWR;
+            break;
+        case BLK_ZT_SWP:
+            desc.z_type = VIRTIO_BLK_ZT_SWP;
+            break;
+        default:
+            g_assert_not_reached();
+        }
+
+        switch (data->zone_report_data.zones[j].state) {
+        case BLK_ZS_RDONLY:
+            desc.z_state = VIRTIO_BLK_ZS_RDONLY;
+            break;
+        case BLK_ZS_OFFLINE:
+            desc.z_state = VIRTIO_BLK_ZS_OFFLINE;
+            break;
+        case BLK_ZS_EMPTY:
+            desc.z_state = VIRTIO_BLK_ZS_EMPTY;
+            break;
+        case BLK_ZS_CLOSED:
+            desc.z_state = VIRTIO_BLK_ZS_CLOSED;
+            break;
+        case BLK_ZS_FULL:
+            desc.z_state = VIRTIO_BLK_ZS_FULL;
+            break;
+        case BLK_ZS_EOPEN:
+            desc.z_state = VIRTIO_BLK_ZS_EOPEN;
+            break;
+        case BLK_ZS_IOPEN:
+            desc.z_state = VIRTIO_BLK_ZS_IOPEN;
+            break;
+        case BLK_ZS_NOT_WP:
+            desc.z_state = VIRTIO_BLK_ZS_NOT_WP;
+            break;
+        default:
+            g_assert_not_reached();
+        }
+
+        /* TODO: it takes O(n^2) time complexity. Optimizations required. */
+        n = iov_from_buf(in_iov, in_num, i, &desc, sizeof(desc));
+        if (n != sizeof(desc)) {
+            virtio_error(vdev, "Driver provided input buffer "
+                               "for descriptors that is too small!");
+            err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+        }
+    }
+
+out:
+    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
+    virtio_blk_req_complete(req, err_status);
+    virtio_blk_free_request(req);
+    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
+    g_free(data->zone_report_data.zones);
+    g_free(data);
+}
+
+static void virtio_blk_handle_zone_report(VirtIOBlockReq *req,
+                                         struct iovec *in_iov,
+                                         unsigned in_num)
+{
+    VirtIOBlock *s = req->dev;
+    VirtIODevice *vdev = VIRTIO_DEVICE(s);
+    unsigned int nr_zones;
+    ZoneCmdData *data;
+    int64_t zone_size, offset;
+    uint8_t err_status;
+
+    if (req->in_len < sizeof(struct virtio_blk_inhdr) +
+            sizeof(struct virtio_blk_zone_report) +
+            sizeof(struct virtio_blk_zone_descriptor)) {
+        virtio_error(vdev, "in buffer too small for zone report");
+        return;
+    }
+
+    /* start byte offset of the zone report */
+    offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
+    if (!check_zoned_request(s, offset, 0, false, &err_status)) {
+        goto out;
+    }
+    nr_zones = (req->in_len - sizeof(struct virtio_blk_inhdr) -
+                sizeof(struct virtio_blk_zone_report)) /
+               sizeof(struct virtio_blk_zone_descriptor);
+
+    zone_size = sizeof(BlockZoneDescriptor) * nr_zones;
+    data = g_malloc(sizeof(ZoneCmdData));
+    data->req = req;
+    data->in_iov = in_iov;
+    data->in_num = in_num;
+    data->zone_report_data.nr_zones = nr_zones;
+    data->zone_report_data.zones = g_malloc(zone_size),
+
+    blk_aio_zone_report(s->blk, offset, &data->zone_report_data.nr_zones,
+                        data->zone_report_data.zones,
+                        virtio_blk_zone_report_complete, data);
+    return;
+out:
+    virtio_blk_req_complete(req, err_status);
+    virtio_blk_free_request(req);
+}
+
+static void virtio_blk_zone_mgmt_complete(void *opaque, int ret)
+{
+    VirtIOBlockReq *req = opaque;
+    VirtIOBlock *s = req->dev;
+    int8_t err_status = VIRTIO_BLK_S_OK;
+
+    if (ret) {
+        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+    }
+
+    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
+    virtio_blk_req_complete(req, err_status);
+    virtio_blk_free_request(req);
+    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
+}
+
+static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, BlockZoneOp op)
+{
+    VirtIOBlock *s = req->dev;
+    VirtIODevice *vdev = VIRTIO_DEVICE(s);
+    BlockDriverState *bs = blk_bs(s->blk);
+    int64_t offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
+    uint64_t len;
+    uint64_t capacity = bs->total_sectors << BDRV_SECTOR_BITS;
+    uint8_t err_status = VIRTIO_BLK_S_OK;
+
+    uint32_t type = virtio_ldl_p(vdev, &req->out.type);
+    if (type == VIRTIO_BLK_T_ZONE_RESET_ALL) {
+        /* Entire drive capacity */
+        offset = 0;
+        len = capacity;
+    } else {
+        if (bs->bl.zone_size > capacity - offset) {
+            /* The zoned device allows the last smaller zone. */
+            len = capacity - bs->bl.zone_size * (bs->bl.nr_zones - 1);
+        } else {
+            len = bs->bl.zone_size;
+        }
+    }
+
+    if (!check_zoned_request(s, offset, len, false, &err_status)) {
+        goto out;
+    }
+
+    blk_aio_zone_mgmt(s->blk, op, offset, len,
+                      virtio_blk_zone_mgmt_complete, req);
+
+    return 0;
+out:
+    virtio_blk_req_complete(req, err_status);
+    virtio_blk_free_request(req);
+    return err_status;
+}
+
+static void virtio_blk_zone_append_complete(void *opaque, int ret)
+{
+    ZoneCmdData *data = opaque;
+    VirtIOBlockReq *req = data->req;
+    VirtIOBlock *s = req->dev;
+    VirtIODevice *vdev = VIRTIO_DEVICE(req->dev);
+    int64_t append_sector, n;
+    uint8_t err_status = VIRTIO_BLK_S_OK;
+
+    if (ret) {
+        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+        goto out;
+    }
+
+    virtio_stq_p(vdev, &append_sector,
+                 data->zone_append_data.offset >> BDRV_SECTOR_BITS);
+    n = iov_from_buf(data->in_iov, data->in_num, 0, &append_sector,
+                     sizeof(append_sector));
+    if (n != sizeof(append_sector)) {
+        virtio_error(vdev, "Driver provided input buffer less than size of "
+                           "append_sector");
+        err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+        goto out;
+    }
+
+out:
+    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
+    virtio_blk_req_complete(req, err_status);
+    virtio_blk_free_request(req);
+    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
+    g_free(data);
+}
+
+static int virtio_blk_handle_zone_append(VirtIOBlockReq *req,
+                                         struct iovec *out_iov,
+                                         struct iovec *in_iov,
+                                         uint64_t out_num,
+                                         unsigned in_num) {
+    VirtIOBlock *s = req->dev;
+    VirtIODevice *vdev = VIRTIO_DEVICE(s);
+    uint8_t err_status = VIRTIO_BLK_S_OK;
+
+    int64_t offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
+    int64_t len = iov_size(out_iov, out_num);
+
+    if (!check_zoned_request(s, offset, len, true, &err_status)) {
+        goto out;
+    }
+
+    ZoneCmdData *data = g_malloc(sizeof(ZoneCmdData));
+    data->req = req;
+    data->in_iov = in_iov;
+    data->in_num = in_num;
+    data->zone_append_data.offset = offset;
+    qemu_iovec_init_external(&req->qiov, out_iov, out_num);
+    blk_aio_zone_append(s->blk, &data->zone_append_data.offset, &req->qiov, 0,
+                        virtio_blk_zone_append_complete, data);
+    return 0;
+
+out:
+    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
+    virtio_blk_req_complete(req, err_status);
+    virtio_blk_free_request(req);
+    aio_context_release(blk_get_aio_context(s->conf.conf.blk));
+    return err_status;
+}
+
 static int virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
 {
     uint32_t type;
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
     case VIRTIO_BLK_T_FLUSH:
         virtio_blk_handle_flush(req, mrb);
         break;
+    case VIRTIO_BLK_T_ZONE_REPORT:
+        virtio_blk_handle_zone_report(req, in_iov, in_num);
+        break;
+    case VIRTIO_BLK_T_ZONE_OPEN:
+        virtio_blk_handle_zone_mgmt(req, BLK_ZO_OPEN);
+        break;
+    case VIRTIO_BLK_T_ZONE_CLOSE:
+        virtio_blk_handle_zone_mgmt(req, BLK_ZO_CLOSE);
+        break;
+    case VIRTIO_BLK_T_ZONE_FINISH:
+        virtio_blk_handle_zone_mgmt(req, BLK_ZO_FINISH);
+        break;
+    case VIRTIO_BLK_T_ZONE_RESET:
+        virtio_blk_handle_zone_mgmt(req, BLK_ZO_RESET);
+        break;
+    case VIRTIO_BLK_T_ZONE_RESET_ALL:
+        virtio_blk_handle_zone_mgmt(req, BLK_ZO_RESET);
+        break;
     case VIRTIO_BLK_T_SCSI_CMD:
         virtio_blk_handle_scsi(req);
         break;
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
         virtio_blk_free_request(req);
         break;
     }
+    case VIRTIO_BLK_T_ZONE_APPEND & ~VIRTIO_BLK_T_OUT:
+        /*
+         * Passing out_iov/out_num and in_iov/in_num is not safe
+         * to access req->elem.out_sg directly because it may be
+         * modified by virtio_blk_handle_request().
+         */
+        virtio_blk_handle_zone_append(req, out_iov, in_iov, out_num, in_num);
+        break;
     /*
      * VIRTIO_BLK_T_DISCARD and VIRTIO_BLK_T_WRITE_ZEROES are defined with
      * VIRTIO_BLK_T_OUT flag set. We masked this flag in the switch statement,
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
 {
     VirtIOBlock *s = VIRTIO_BLK(vdev);
     BlockConf *conf = &s->conf.conf;
+    BlockDriverState *bs = blk_bs(s->blk);
     struct virtio_blk_config blkcfg;
     uint64_t capacity;
     int64_t length;
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_update_config(VirtIODevice *vdev, uint8_t *config)
         blkcfg.write_zeroes_may_unmap = 1;
         virtio_stl_p(vdev, &blkcfg.max_write_zeroes_seg, 1);
     }
+    if (bs->bl.zoned != BLK_Z_NONE) {
+        switch (bs->bl.zoned) {
+        case BLK_Z_HM:
+            blkcfg.zoned.model = VIRTIO_BLK_Z_HM;
+            break;
+        case BLK_Z_HA:
+            blkcfg.zoned.model = VIRTIO_BLK_Z_HA;
+            break;
+        default:
+            g_assert_not_reached();
+        }
+
+        virtio_stl_p(vdev, &blkcfg.zoned.zone_sectors,
+                     bs->bl.zone_size / 512);
+        virtio_stl_p(vdev, &blkcfg.zoned.max_active_zones,
+                     bs->bl.max_active_zones);
+        virtio_stl_p(vdev, &blkcfg.zoned.max_open_zones,
+                     bs->bl.max_open_zones);
+        virtio_stl_p(vdev, &blkcfg.zoned.write_granularity, blk_size);
+        virtio_stl_p(vdev, &blkcfg.zoned.max_append_sectors,
+                     bs->bl.max_append_sectors);
+    } else {
+        blkcfg.zoned.model = VIRTIO_BLK_Z_NONE;
+    }
     memcpy(config, &blkcfg, s->config_size);
 }
 
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp)
     VirtIODevice *vdev = VIRTIO_DEVICE(dev);
     VirtIOBlock *s = VIRTIO_BLK(dev);
     VirtIOBlkConf *conf = &s->conf;
+    BlockDriverState *bs = blk_bs(conf->conf.blk);
     Error *err = NULL;
     unsigned i;
 
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (bs->bl.zoned != BLK_Z_NONE) {
+        virtio_add_feature(&s->host_features, VIRTIO_BLK_F_ZONED);
+        if (bs->bl.zoned == BLK_Z_HM) {
+            virtio_clear_feature(&s->host_features, VIRTIO_BLK_F_DISCARD);
+        }
+    }
+
     if (virtio_has_feature(s->host_features, VIRTIO_BLK_F_DISCARD) &&
         (!conf->max_discard_sectors ||
          conf->max_discard_sectors > BDRV_REQUEST_MAX_SECTORS)) {
diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/virtio/virtio-qmp.c
+++ b/hw/virtio/virtio-qmp.c
@@ -XXX,XX +XXX,XX @@ static const qmp_virtio_feature_map_t virtio_blk_feature_map[] = {
             "VIRTIO_BLK_F_DISCARD: Discard command supported"),
     FEATURE_ENTRY(VIRTIO_BLK_F_WRITE_ZEROES, \
             "VIRTIO_BLK_F_WRITE_ZEROES: Write zeroes command supported"),
+    FEATURE_ENTRY(VIRTIO_BLK_F_ZONED, \
+            "VIRTIO_BLK_F_ZONED: Zoned block devices"),
 #ifndef VIRTIO_BLK_NO_LEGACY
     FEATURE_ENTRY(VIRTIO_BLK_F_BARRIER, \
             "VIRTIO_BLK_F_BARRIER: Request barriers supported"),
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Taking account of the new zone append write operation for zoned devices,
BLOCK_ACCT_ZONE_APPEND enum is introduced as other I/O request type (read,
write, flush).

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Message-id: 20230407082528.18841-4-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qapi/block-core.json       | 68 ++++++++++++++++++++++++++++++++------
 qapi/block.json            |  4 +++
 include/block/accounting.h |  1 +
 block/qapi-sysemu.c        | 11 ++++++
 block/qapi.c               | 18 ++++++++++
 hw/block/virtio-blk.c      |  4 +++
 6 files changed, 95 insertions(+), 11 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
 # @min_wr_latency_ns: Minimum latency of write operations in the
 #                     defined interval, in nanoseconds.
 #
+# @min_zone_append_latency_ns: Minimum latency of zone append operations
+#                              in the defined interval, in nanoseconds
+#                              (since 8.1)
+#
 # @min_flush_latency_ns: Minimum latency of flush operations in the
 #                        defined interval, in nanoseconds.
 #
@@ -XXX,XX +XXX,XX @@
 # @max_wr_latency_ns: Maximum latency of write operations in the
 #                     defined interval, in nanoseconds.
 #
+# @max_zone_append_latency_ns: Maximum latency of zone append operations
+#                              in the defined interval, in nanoseconds
+#                              (since 8.1)
+#
 # @max_flush_latency_ns: Maximum latency of flush operations in the
 #                        defined interval, in nanoseconds.
 #
@@ -XXX,XX +XXX,XX @@
 # @avg_wr_latency_ns: Average latency of write operations in the
 #                     defined interval, in nanoseconds.
 #
+# @avg_zone_append_latency_ns: Average latency of zone append operations
+#                              in the defined interval, in nanoseconds
+#                              (since 8.1)
+#
 # @avg_flush_latency_ns: Average latency of flush operations in the
 #                        defined interval, in nanoseconds.
 #
@@ -XXX,XX +XXX,XX @@
 # @avg_wr_queue_depth: Average number of pending write operations
 #                      in the defined interval.
 #
+# @avg_zone_append_queue_depth: Average number of pending zone append
+#                               operations in the defined interval
+#                               (since 8.1).
+#
 # Since: 2.5
 ##
 { 'struct': 'BlockDeviceTimedStats',
   'data': { 'interval_length': 'int', 'min_rd_latency_ns': 'int',
             'max_rd_latency_ns': 'int', 'avg_rd_latency_ns': 'int',
             'min_wr_latency_ns': 'int', 'max_wr_latency_ns': 'int',
-            'avg_wr_latency_ns': 'int', 'min_flush_latency_ns': 'int',
-            'max_flush_latency_ns': 'int', 'avg_flush_latency_ns': 'int',
-            'avg_rd_queue_depth': 'number', 'avg_wr_queue_depth': 'number' } }
+            'avg_wr_latency_ns': 'int', 'min_zone_append_latency_ns': 'int',
+            'max_zone_append_latency_ns': 'int',
+            'avg_zone_append_latency_ns': 'int',
+            'min_flush_latency_ns': 'int', 'max_flush_latency_ns': 'int',
+            'avg_flush_latency_ns': 'int', 'avg_rd_queue_depth': 'number',
+            'avg_wr_queue_depth': 'number',
+            'avg_zone_append_queue_depth': 'number'  } }
 
 ##
 # @BlockDeviceStats:
@@ -XXX,XX +XXX,XX @@
 #
 # @wr_bytes: The number of bytes written by the device.
 #
+# @zone_append_bytes: The number of bytes appended by the zoned devices
+#                     (since 8.1)
+#
 # @unmap_bytes: The number of bytes unmapped by the device (Since 4.2)
 #
 # @rd_operations: The number of read operations performed by the device.
 #
 # @wr_operations: The number of write operations performed by the device.
 #
+# @zone_append_operations: The number of zone append operations performed
+#                          by the zoned devices (since 8.1)
+#
 # @flush_operations: The number of cache flush operations performed by the
 #                    device (since 0.15)
 #
@@ -XXX,XX +XXX,XX @@
 #
 # @wr_total_time_ns: Total time spent on writes in nanoseconds (since 0.15).
 #
+# @zone_append_total_time_ns: Total time spent on zone append writes
+#                             in nanoseconds (since 8.1)
+#
 # @flush_total_time_ns: Total time spent on cache flushes in nanoseconds
 #                       (since 0.15).
 #
@@ -XXX,XX +XXX,XX @@
 # @wr_merged: Number of write requests that have been merged into another
 #             request (Since 2.3).
 #
+# @zone_append_merged: Number of zone append requests that have been merged
+#                      into another request (since 8.1)
+#
 # @unmap_merged: Number of unmap requests that have been merged into another
 #                request (Since 4.2)
 #
@@ -XXX,XX +XXX,XX @@
 # @failed_wr_operations: The number of failed write operations
 #                        performed by the device (Since 2.5)
 #
+# @failed_zone_append_operations: The number of failed zone append write
+#                                 operations performed by the zoned devices
+#                                 (since 8.1)
+#
 # @failed_flush_operations: The number of failed flush operations
 #                           performed by the device (Since 2.5)
 #
@@ -XXX,XX +XXX,XX @@
 # @invalid_wr_operations: The number of invalid write operations
 #                         performed by the device (Since 2.5)
 #
+# @invalid_zone_append_operations: The number of invalid zone append operations
+#                                  performed by the zoned device (since 8.1)
+#
 # @invalid_flush_operations: The number of invalid flush operations
 #                            performed by the device (Since 2.5)
 #
@@ -XXX,XX +XXX,XX @@
 #
 # @wr_latency_histogram: @BlockLatencyHistogramInfo. (Since 4.0)
 #
+# @zone_append_latency_histogram: @BlockLatencyHistogramInfo. (since 8.1)
+#
 # @flush_latency_histogram: @BlockLatencyHistogramInfo. (Since 4.0)
 #
 # Since: 0.14
 ##
 { 'struct': 'BlockDeviceStats',
-  'data': {'rd_bytes': 'int', 'wr_bytes': 'int', 'unmap_bytes' : 'int',
-           'rd_operations': 'int', 'wr_operations': 'int',
+  'data': {'rd_bytes': 'int', 'wr_bytes': 'int', 'zone_append_bytes': 'int',
+           'unmap_bytes' : 'int', 'rd_operations': 'int',
+           'wr_operations': 'int', 'zone_append_operations': 'int',
            'flush_operations': 'int', 'unmap_operations': 'int',
            'rd_total_time_ns': 'int', 'wr_total_time_ns': 'int',
-           'flush_total_time_ns': 'int', 'unmap_total_time_ns': 'int',
-           'wr_highest_offset': 'int',
-           'rd_merged': 'int', 'wr_merged': 'int', 'unmap_merged': 'int',
-           '*idle_time_ns': 'int',
+           'zone_append_total_time_ns': 'int', 'flush_total_time_ns': 'int',
+           'unmap_total_time_ns': 'int', 'wr_highest_offset': 'int',
+           'rd_merged': 'int', 'wr_merged': 'int', 'zone_append_merged': 'int',
+           'unmap_merged': 'int', '*idle_time_ns': 'int',
            'failed_rd_operations': 'int', 'failed_wr_operations': 'int',
-           'failed_flush_operations': 'int', 'failed_unmap_operations': 'int',
-           'invalid_rd_operations': 'int', 'invalid_wr_operations': 'int',
+           'failed_zone_append_operations': 'int',
+           'failed_flush_operations': 'int',
+           'failed_unmap_operations': 'int', 'invalid_rd_operations': 'int',
+           'invalid_wr_operations': 'int',
+           'invalid_zone_append_operations': 'int',
            'invalid_flush_operations': 'int', 'invalid_unmap_operations': 'int',
            'account_invalid': 'bool', 'account_failed': 'bool',
            'timed_stats': ['BlockDeviceTimedStats'],
            '*rd_latency_histogram': 'BlockLatencyHistogramInfo',
            '*wr_latency_histogram': 'BlockLatencyHistogramInfo',
+           '*zone_append_latency_histogram': 'BlockLatencyHistogramInfo',
            '*flush_latency_histogram': 'BlockLatencyHistogramInfo' } }
 
 ##
diff --git a/qapi/block.json b/qapi/block.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -XXX,XX +XXX,XX @@
 # @boundaries-write: list of interval boundary values for write latency
 #                    histogram.
 #
+# @boundaries-zap: list of interval boundary values for zone append write
+#                  latency histogram.
+#
 # @boundaries-flush: list of interval boundary values for flush latency
 #                    histogram.
 #
@@ -XXX,XX +XXX,XX @@
            '*boundaries': ['uint64'],
            '*boundaries-read': ['uint64'],
            '*boundaries-write': ['uint64'],
+           '*boundaries-zap': ['uint64'],
            '*boundaries-flush': ['uint64'] },
   'allow-preconfig': true }
diff --git a/include/block/accounting.h b/include/block/accounting.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/accounting.h
+++ b/include/block/accounting.h
@@ -XXX,XX +XXX,XX @@ enum BlockAcctType {
     BLOCK_ACCT_READ,
     BLOCK_ACCT_WRITE,
     BLOCK_ACCT_FLUSH,
+    BLOCK_ACCT_ZONE_APPEND,
     BLOCK_ACCT_UNMAP,
     BLOCK_MAX_IOTYPE,
 };
diff --git a/block/qapi-sysemu.c b/block/qapi-sysemu.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qapi-sysemu.c
+++ b/block/qapi-sysemu.c
@@ -XXX,XX +XXX,XX @@ void qmp_block_latency_histogram_set(
     bool has_boundaries, uint64List *boundaries,
     bool has_boundaries_read, uint64List *boundaries_read,
     bool has_boundaries_write, uint64List *boundaries_write,
+    bool has_boundaries_append, uint64List *boundaries_append,
     bool has_boundaries_flush, uint64List *boundaries_flush,
     Error **errp)
 {
@@ -XXX,XX +XXX,XX @@ void qmp_block_latency_histogram_set(
         }
     }
 
+    if (has_boundaries || has_boundaries_append) {
+        ret = block_latency_histogram_set(
+                stats, BLOCK_ACCT_ZONE_APPEND,
+                has_boundaries_append ? boundaries_append : boundaries);
+        if (ret) {
+            error_setg(errp, "Device '%s' set append write boundaries fail", id);
+            return;
+        }
+    }
+
     if (has_boundaries || has_boundaries_flush) {
         ret = block_latency_histogram_set(
             stats, BLOCK_ACCT_FLUSH,
diff --git a/block/qapi.c b/block/qapi.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
 
     ds->rd_bytes = stats->nr_bytes[BLOCK_ACCT_READ];
     ds->wr_bytes = stats->nr_bytes[BLOCK_ACCT_WRITE];
+    ds->zone_append_bytes = stats->nr_bytes[BLOCK_ACCT_ZONE_APPEND];
     ds->unmap_bytes = stats->nr_bytes[BLOCK_ACCT_UNMAP];
     ds->rd_operations = stats->nr_ops[BLOCK_ACCT_READ];
     ds->wr_operations = stats->nr_ops[BLOCK_ACCT_WRITE];
+    ds->zone_append_operations = stats->nr_ops[BLOCK_ACCT_ZONE_APPEND];
     ds->unmap_operations = stats->nr_ops[BLOCK_ACCT_UNMAP];
 
     ds->failed_rd_operations = stats->failed_ops[BLOCK_ACCT_READ];
     ds->failed_wr_operations = stats->failed_ops[BLOCK_ACCT_WRITE];
+    ds->failed_zone_append_operations =
+        stats->failed_ops[BLOCK_ACCT_ZONE_APPEND];
     ds->failed_flush_operations = stats->failed_ops[BLOCK_ACCT_FLUSH];
     ds->failed_unmap_operations = stats->failed_ops[BLOCK_ACCT_UNMAP];
 
     ds->invalid_rd_operations = stats->invalid_ops[BLOCK_ACCT_READ];
     ds->invalid_wr_operations = stats->invalid_ops[BLOCK_ACCT_WRITE];
+    ds->invalid_zone_append_operations =
+        stats->invalid_ops[BLOCK_ACCT_ZONE_APPEND];
     ds->invalid_flush_operations =
         stats->invalid_ops[BLOCK_ACCT_FLUSH];
     ds->invalid_unmap_operations = stats->invalid_ops[BLOCK_ACCT_UNMAP];
 
     ds->rd_merged = stats->merged[BLOCK_ACCT_READ];
     ds->wr_merged = stats->merged[BLOCK_ACCT_WRITE];
+    ds->zone_append_merged = stats->merged[BLOCK_ACCT_ZONE_APPEND];
     ds->unmap_merged = stats->merged[BLOCK_ACCT_UNMAP];
     ds->flush_operations = stats->nr_ops[BLOCK_ACCT_FLUSH];
     ds->wr_total_time_ns = stats->total_time_ns[BLOCK_ACCT_WRITE];
+    ds->zone_append_total_time_ns =
+        stats->total_time_ns[BLOCK_ACCT_ZONE_APPEND];
     ds->rd_total_time_ns = stats->total_time_ns[BLOCK_ACCT_READ];
     ds->flush_total_time_ns = stats->total_time_ns[BLOCK_ACCT_FLUSH];
     ds->unmap_total_time_ns = stats->total_time_ns[BLOCK_ACCT_UNMAP];
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
 
         TimedAverage *rd = &ts->latency[BLOCK_ACCT_READ];
         TimedAverage *wr = &ts->latency[BLOCK_ACCT_WRITE];
+        TimedAverage *zap = &ts->latency[BLOCK_ACCT_ZONE_APPEND];
         TimedAverage *fl = &ts->latency[BLOCK_ACCT_FLUSH];
 
         dev_stats->interval_length = ts->interval_length;
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
         dev_stats->max_wr_latency_ns = timed_average_max(wr);
         dev_stats->avg_wr_latency_ns = timed_average_avg(wr);
 
+        dev_stats->min_zone_append_latency_ns = timed_average_min(zap);
+        dev_stats->max_zone_append_latency_ns = timed_average_max(zap);
+        dev_stats->avg_zone_append_latency_ns = timed_average_avg(zap);
+
         dev_stats->min_flush_latency_ns = timed_average_min(fl);
         dev_stats->max_flush_latency_ns = timed_average_max(fl);
         dev_stats->avg_flush_latency_ns = timed_average_avg(fl);
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
             block_acct_queue_depth(ts, BLOCK_ACCT_READ);
         dev_stats->avg_wr_queue_depth =
             block_acct_queue_depth(ts, BLOCK_ACCT_WRITE);
+        dev_stats->avg_zone_append_queue_depth =
+            block_acct_queue_depth(ts, BLOCK_ACCT_ZONE_APPEND);
 
         QAPI_LIST_PREPEND(ds->timed_stats, dev_stats);
     }
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
         = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_READ]);
     ds->wr_latency_histogram
         = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_WRITE]);
+    ds->zone_append_latency_histogram
+        = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_ZONE_APPEND]);
     ds->flush_latency_histogram
         = bdrv_latency_histogram_stats(&hgram[BLOCK_ACCT_FLUSH]);
 }
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_append(VirtIOBlockReq *req,
     data->in_num = in_num;
     data->zone_append_data.offset = offset;
     qemu_iovec_init_external(&req->qiov, out_iov, out_num);
+
+    block_acct_start(blk_get_stats(s->blk), &req->acct, len,
+                     BLOCK_ACCT_ZONE_APPEND);
+
     blk_aio_zone_append(s->blk, &data->zone_append_data.offset, &req->qiov, 0,
                         virtio_blk_zone_append_complete, data);
     return 0;
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20230407082528.18841-5-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/virtio-blk.c | 12 ++++++++++++
 hw/block/trace-events |  7 +++++++
 2 files changed, 19 insertions(+)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_zone_report_complete(void *opaque, int ret)
     int64_t nz = data->zone_report_data.nr_zones;
     int8_t err_status = VIRTIO_BLK_S_OK;
 
+    trace_virtio_blk_zone_report_complete(vdev, req, nz, ret);
     if (ret) {
         err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
         goto out;
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_handle_zone_report(VirtIOBlockReq *req,
     nr_zones = (req->in_len - sizeof(struct virtio_blk_inhdr) -
                 sizeof(struct virtio_blk_zone_report)) /
                sizeof(struct virtio_blk_zone_descriptor);
+    trace_virtio_blk_handle_zone_report(vdev, req,
+                                        offset >> BDRV_SECTOR_BITS, nr_zones);
 
     zone_size = sizeof(BlockZoneDescriptor) * nr_zones;
     data = g_malloc(sizeof(ZoneCmdData));
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_zone_mgmt_complete(void *opaque, int ret)
 {
     VirtIOBlockReq *req = opaque;
     VirtIOBlock *s = req->dev;
+    VirtIODevice *vdev = VIRTIO_DEVICE(s);
     int8_t err_status = VIRTIO_BLK_S_OK;
+    trace_virtio_blk_zone_mgmt_complete(vdev, req,ret);
 
     if (ret) {
         err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, BlockZoneOp op)
         /* Entire drive capacity */
         offset = 0;
         len = capacity;
+        trace_virtio_blk_handle_zone_reset_all(vdev, req, 0,
+                                               bs->total_sectors);
     } else {
         if (bs->bl.zone_size > capacity - offset) {
             /* The zoned device allows the last smaller zone. */
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req, BlockZoneOp op)
         } else {
             len = bs->bl.zone_size;
         }
+        trace_virtio_blk_handle_zone_mgmt(vdev, req, op,
+                                          offset >> BDRV_SECTOR_BITS,
+                                          len >> BDRV_SECTOR_BITS);
     }
 
     if (!check_zoned_request(s, offset, len, false, &err_status)) {
@@ -XXX,XX +XXX,XX @@ static void virtio_blk_zone_append_complete(void *opaque, int ret)
         err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
         goto out;
     }
+    trace_virtio_blk_zone_append_complete(vdev, req, append_sector, ret);
 
 out:
     aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
@@ -XXX,XX +XXX,XX @@ static int virtio_blk_handle_zone_append(VirtIOBlockReq *req,
     int64_t offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
     int64_t len = iov_size(out_iov, out_num);
 
+    trace_virtio_blk_handle_zone_append(vdev, req, offset >> BDRV_SECTOR_BITS);
     if (!check_zoned_request(s, offset, len, true, &err_status)) {
         goto out;
     }
diff --git a/hw/block/trace-events b/hw/block/trace-events
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -XXX,XX +XXX,XX @@ pflash_write_unknown(const char *name, uint8_t cmd) "%s: unknown command 0x%02x"
 # virtio-blk.c
 virtio_blk_req_complete(void *vdev, void *req, int status) "vdev %p req %p status %d"
 virtio_blk_rw_complete(void *vdev, void *req, int ret) "vdev %p req %p ret %d"
+virtio_blk_zone_report_complete(void *vdev, void *req, unsigned int nr_zones, int ret) "vdev %p req %p nr_zones %u ret %d"
+virtio_blk_zone_mgmt_complete(void *vdev, void *req, int ret) "vdev %p req %p ret %d"
+virtio_blk_zone_append_complete(void *vdev, void *req, int64_t sector, int ret) "vdev %p req %p, append sector 0x%" PRIx64 " ret %d"
 virtio_blk_handle_write(void *vdev, void *req, uint64_t sector, size_t nsectors) "vdev %p req %p sector %"PRIu64" nsectors %zu"
 virtio_blk_handle_read(void *vdev, void *req, uint64_t sector, size_t nsectors) "vdev %p req %p sector %"PRIu64" nsectors %zu"
 virtio_blk_submit_multireq(void *vdev, void *mrb, int start, int num_reqs, uint64_t offset, size_t size, bool is_write) "vdev %p mrb %p start %d num_reqs %d offset %"PRIu64" size %zu is_write %d"
+virtio_blk_handle_zone_report(void *vdev, void *req, int64_t sector, unsigned int nr_zones) "vdev %p req %p sector 0x%" PRIx64 " nr_zones %u"
+virtio_blk_handle_zone_mgmt(void *vdev, void *req, uint8_t op, int64_t sector, int64_t len) "vdev %p req %p op 0x%x sector 0x%" PRIx64 " len 0x%" PRIx64 ""
+virtio_blk_handle_zone_reset_all(void *vdev, void *req, int64_t sector, int64_t len) "vdev %p req %p sector 0x%" PRIx64 " cap 0x%" PRIx64 ""
+virtio_blk_handle_zone_append(void *vdev, void *req, int64_t sector) "vdev %p req %p, append sector 0x%" PRIx64 ""
 
 # hd-geometry.c
 hd_geometry_lchs_guess(void *blk, int cyls, int heads, int secs) "blk %p LCHS %d %d %d"
-- 
2.39.2

From: Sam Li <faithilikerun@gmail.com>

Add the documentation about the example of using virtio-blk driver
to pass the zoned block devices through to the guest.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Message-id: 20230407082528.18841-6-faithilikerun@gmail.com
[Fix Sphinx indentation error by turning command-lines into
pre-formatted text.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/devel/zoned-storage.rst | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst
index XXXXXXX..XXXXXXX 100644
--- a/docs/devel/zoned-storage.rst
+++ b/docs/devel/zoned-storage.rst
@@ -XXX,XX +XXX,XX @@ When the BlockBackend's BlockLimits model reports a zoned storage device, users
 like the virtio-blk emulation or the qemu-io-cmds.c utility can use block layer
 APIs for zoned storage emulation or testing.
 
-For example, to test zone_report on a null_blk device using qemu-io is:
-$ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0
--c "zrp offset nr_zones"
+For example, to test zone_report on a null_blk device using qemu-io is::
+
+  $ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0 -c "zrp offset nr_zones"
+
+To expose the host's zoned block device through virtio-blk, the command line
+can be (includes the -device parameter)::
+
+  -blockdev node-name=drive0,driver=host_device,filename=/dev/nullb0,cache.direct=on \
+      -device virtio-blk-pci,drive=drive0
+
+Or only use the -drive parameter::
+
+  -driver driver=host_device,file=/dev/nullb0,if=virtio,cache.direct=on
+
+Additionally, QEMU has several ways of supporting zoned storage, including:
+(1) Using virtio-scsi: --device scsi-block allows for the passing through of
+SCSI ZBC devices, enabling the attachment of ZBC or ZAC HDDs to QEMU.
+(2) PCI device pass-through: While NVMe ZNS emulation is available for testing
+purposes, it cannot yet pass through a zoned device from the host. To pass on
+the NVMe ZNS device to the guest, use VFIO PCI pass the entire NVMe PCI adapter
+through to the guest. Likewise, an HDD HBA can be passed on to QEMU all HDDs
+attached to the HBA.
-- 
2.39.2