[PULL v2 00/13] misc patches removing deprecated features

Daniel P. Berrangé posted 13 patches 3 years, 1 month ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210318092512.250725-1-berrange@redhat.com
block/dirty-bitmap.c                          |  38 --
block/file-posix.c                            |  17 +-
block/qapi.c                                  |   6 -
chardev/char-socket.c                         |  12 +-
docs/devel/migration.rst                      |   2 +-
docs/qdev-device-use.txt                      |   2 +-
docs/rdma.txt                                 |   2 +-
docs/system/deprecated.rst                    | 108 -----
docs/system/removed-features.rst              | 108 +++++
docs/xbzrle.txt                               |   5 -
hmp-commands-info.hx                          |  13 -
hmp-commands.hx                               | 121 ------
hw/core/machine-hmp-cmds.c                    |   8 +-
hw/core/machine-qmp-cmds.c                    | 120 ------
hw/i386/pc.c                                  |   2 -
hw/ide/qdev.c                                 |  38 --
hw/ppc/mac_newworld.c                         |  13 -
hw/ppc/mac_oldworld.c                         |  13 -
hw/scsi/scsi-disk.c                           |  62 ---
hw/sparc64/sun4u.c                            |  15 -
include/block/dirty-bitmap.h                  |   1 -
include/monitor/hmp.h                         |   4 -
migration/migration.c                         |  45 ---
migration/ram.c                               |   2 +-
monitor/hmp-cmds.c                            |  34 --
monitor/misc.c                                | 187 ---------
monitor/monitor.c                             |   4 +-
monitor/qmp-cmds-control.c                    |  24 --
qapi/block-core.json                          |  64 +--
qapi/control.json                             |  45 ---
qapi/machine.json                             | 181 +--------
qapi/migration.json                           |  98 -----
qemu-options.hx                               |   5 +-
scripts/device-crash-test                     |   2 -
softmmu/vl.c                                  |   1 -
tests/acceptance/pc_cpu_hotplug_props.py      |   2 +-
tests/acceptance/x86_cpu_model_versions.py    |   2 +-
tests/migration/guestperf/engine.py           |  18 +-
tests/qemu-iotests/051                        |   4 -
tests/qemu-iotests/051.pc.out                 |  20 -
tests/qemu-iotests/124                        |   4 -
tests/qemu-iotests/181                        |   2 +-
tests/qemu-iotests/184.out                    |   6 +-
tests/qemu-iotests/191.out                    |  48 +--
tests/qemu-iotests/194                        |   4 +-
tests/qemu-iotests/194.out                    |   4 +-
tests/qemu-iotests/226.out                    |  10 +-
tests/qemu-iotests/236                        |   2 +-
tests/qemu-iotests/236.out                    |  42 +-
tests/qemu-iotests/246                        |   3 +-
tests/qemu-iotests/246.out                    |  66 +--
tests/qemu-iotests/254                        |   2 +-
tests/qemu-iotests/254.out                    |   9 +-
tests/qemu-iotests/257.out                    | 378 ++++++------------
tests/qemu-iotests/260                        |   5 +-
tests/qemu-iotests/273.out                    |  15 +-
.../tests/migrate-bitmaps-postcopy-test       |   6 +-
tests/qtest/migration-test.c                  |  48 ---
tests/qtest/numa-test.c                       |   6 +-
tests/qtest/qmp-test.c                        |   6 +-
tests/qtest/test-hmp.c                        |   6 +-
tests/qtest/test-x86-cpuid-compat.c           |   4 +-
tests/qtest/vhost-user-test.c                 |   8 +-
ui/vnc.c                                      |  38 --
64 files changed, 369 insertions(+), 1801 deletions(-)
[PULL v2 00/13] misc patches removing deprecated features
Posted by Daniel P. Berrangé 3 years, 1 month ago
The following changes since commit 571d413b5da6bc6f1c2aaca8484717642255ddb0:

  Merge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios-20210316' into staging (2021-03-17 21:02:37 +0000)

are available in the Git repository at:

  https://gitlab.com/berrange/qemu tags/dep-many-pull-request

for you to fetch changes up to 8d17adf34f501ded65a106572740760f0a75577c:

  block: remove support for using "file" driver with block/char devices (2021-03-18 09:22:55 +0000)

----------------------------------------------------------------
Remove many old deprecated features

The following features have been deprecated for well over the 2
release cycle we promise

  ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
  ``-vnc acl`` (since 4.0.0)
  ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
  ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
  ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
  ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
  ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
  ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
  ``query-cpus`` (since 2.12.0)
  ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
  ``query-events`` (since 4.0)
  chardev client socket with ``wait`` option (since 4.0)
  ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
  ``ide-drive`` (since 4.2)
  ``scsi-disk`` (since 4.2)

----------------------------------------------------------------

Daniel P. Berrangé (13):
  ui, monitor: remove deprecated VNC ACL option and HMP commands
  monitor: raise error when 'pretty' option is used with HMP
  monitor: remove 'query-events' QMP command
  migrate: remove QMP/HMP commands for speed, downtime and cache size
  machine: remove 'query-cpus' QMP command
  machine: remove 'arch' field from 'query-cpus-fast' QMP command
  chardev: reject use of 'wait' flag for socket client chardevs
  hw/ide: remove 'ide-drive' device
  hw/scsi: remove 'scsi-disk' device
  block: remove 'encryption_key_missing' flag from QAPI
  block: remove dirty bitmaps 'status' field
  block: remove 'dirty-bitmaps' field from 'BlockInfo' struct
  block: remove support for using "file" driver with block/char devices

 block/dirty-bitmap.c                          |  38 --
 block/file-posix.c                            |  17 +-
 block/qapi.c                                  |   6 -
 chardev/char-socket.c                         |  12 +-
 docs/devel/migration.rst                      |   2 +-
 docs/qdev-device-use.txt                      |   2 +-
 docs/rdma.txt                                 |   2 +-
 docs/system/deprecated.rst                    | 108 -----
 docs/system/removed-features.rst              | 108 +++++
 docs/xbzrle.txt                               |   5 -
 hmp-commands-info.hx                          |  13 -
 hmp-commands.hx                               | 121 ------
 hw/core/machine-hmp-cmds.c                    |   8 +-
 hw/core/machine-qmp-cmds.c                    | 120 ------
 hw/i386/pc.c                                  |   2 -
 hw/ide/qdev.c                                 |  38 --
 hw/ppc/mac_newworld.c                         |  13 -
 hw/ppc/mac_oldworld.c                         |  13 -
 hw/scsi/scsi-disk.c                           |  62 ---
 hw/sparc64/sun4u.c                            |  15 -
 include/block/dirty-bitmap.h                  |   1 -
 include/monitor/hmp.h                         |   4 -
 migration/migration.c                         |  45 ---
 migration/ram.c                               |   2 +-
 monitor/hmp-cmds.c                            |  34 --
 monitor/misc.c                                | 187 ---------
 monitor/monitor.c                             |   4 +-
 monitor/qmp-cmds-control.c                    |  24 --
 qapi/block-core.json                          |  64 +--
 qapi/control.json                             |  45 ---
 qapi/machine.json                             | 181 +--------
 qapi/migration.json                           |  98 -----
 qemu-options.hx                               |   5 +-
 scripts/device-crash-test                     |   2 -
 softmmu/vl.c                                  |   1 -
 tests/acceptance/pc_cpu_hotplug_props.py      |   2 +-
 tests/acceptance/x86_cpu_model_versions.py    |   2 +-
 tests/migration/guestperf/engine.py           |  18 +-
 tests/qemu-iotests/051                        |   4 -
 tests/qemu-iotests/051.pc.out                 |  20 -
 tests/qemu-iotests/124                        |   4 -
 tests/qemu-iotests/181                        |   2 +-
 tests/qemu-iotests/184.out                    |   6 +-
 tests/qemu-iotests/191.out                    |  48 +--
 tests/qemu-iotests/194                        |   4 +-
 tests/qemu-iotests/194.out                    |   4 +-
 tests/qemu-iotests/226.out                    |  10 +-
 tests/qemu-iotests/236                        |   2 +-
 tests/qemu-iotests/236.out                    |  42 +-
 tests/qemu-iotests/246                        |   3 +-
 tests/qemu-iotests/246.out                    |  66 +--
 tests/qemu-iotests/254                        |   2 +-
 tests/qemu-iotests/254.out                    |   9 +-
 tests/qemu-iotests/257.out                    | 378 ++++++------------
 tests/qemu-iotests/260                        |   5 +-
 tests/qemu-iotests/273.out                    |  15 +-
 .../tests/migrate-bitmaps-postcopy-test       |   6 +-
 tests/qtest/migration-test.c                  |  48 ---
 tests/qtest/numa-test.c                       |   6 +-
 tests/qtest/qmp-test.c                        |   6 +-
 tests/qtest/test-hmp.c                        |   6 +-
 tests/qtest/test-x86-cpuid-compat.c           |   4 +-
 tests/qtest/vhost-user-test.c                 |   8 +-
 ui/vnc.c                                      |  38 --
 64 files changed, 369 insertions(+), 1801 deletions(-)

-- 
2.30.2


Re: [PULL v2 00/13] misc patches removing deprecated features
Posted by Peter Maydell 3 years, 1 month ago
On Thu, 18 Mar 2021 at 09:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> The following changes since commit 571d413b5da6bc6f1c2aaca8484717642255ddb0:
>
>   Merge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios-20210316' into staging (2021-03-17 21:02:37 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/berrange/qemu tags/dep-many-pull-request
>
> for you to fetch changes up to 8d17adf34f501ded65a106572740760f0a75577c:
>
>   block: remove support for using "file" driver with block/char devices (2021-03-18 09:22:55 +0000)
>
> ----------------------------------------------------------------
> Remove many old deprecated features
>
> The following features have been deprecated for well over the 2
> release cycle we promise
>
>   ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
>   ``-vnc acl`` (since 4.0.0)
>   ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
>   ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
>   ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
>   ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
>   ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
>   ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
>   ``query-cpus`` (since 2.12.0)
>   ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
>   ``query-events`` (since 4.0)
>   chardev client socket with ``wait`` option (since 4.0)
>   ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
>   ``ide-drive`` (since 4.2)
>   ``scsi-disk`` (since 4.2)
>
> ----------------------------------------------------------------


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/6.0
for any user-visible changes.

-- PMM


Re: [PULL v2 00/13] misc patches removing deprecated features
Posted by Peter Maydell 3 years, 1 month ago
On Thu, 18 Mar 2021 at 09:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
> ----------------------------------------------------------------
> Remove many old deprecated features
>
> The following features have been deprecated for well over the 2
> release cycle we promise
>
>   ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
>   ``-vnc acl`` (since 4.0.0)
>   ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
>   ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
>   ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
>   ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
>   ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
>   ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
>   ``query-cpus`` (since 2.12.0)
>   ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
>   ``query-events`` (since 4.0)
>   chardev client socket with ``wait`` option (since 4.0)
>   ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
>   ``ide-drive`` (since 4.2)
>   ``scsi-disk`` (since 4.2)

Just FYI, it turns out that removing 'query-cpus' has broken Xen,
whose libxl control library is still using it:

https://lists.xenproject.org/archives/html/xen-devel/2021-03/msg01178.html

This suggests we should probably try to coordinate a bit better
with Xen as one of our downstream QMP users.

Am I right in thinking that QEMU now has an option that Xen
could turn on in their CI for "make attempts to use deprecated
features fail" so they could catch them earlier in the
deprecate-and-delete cycle?

thanks
-- PMM

Re: [PULL v2 00/13] misc patches removing deprecated features
Posted by Ian Jackson 3 years, 1 month ago
Peter Maydell writes ("Re: [PULL v2 00/13] misc patches removing deprecated features"):
> On Thu, 18 Mar 2021 at 09:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > ----------------------------------------------------------------
> > Remove many old deprecated features
> >
> > The following features have been deprecated for well over the 2
> > release cycle we promise
> >
> >   ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
> >   ``-vnc acl`` (since 4.0.0)
> >   ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
> >   ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
> >   ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
> >   ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
> >   ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
> >   ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
> >   ``query-cpus`` (since 2.12.0)
> >   ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
> >   ``query-events`` (since 4.0)
> >   chardev client socket with ``wait`` option (since 4.0)
> >   ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
> >   ``ide-drive`` (since 4.2)
> >   ``scsi-disk`` (since 4.2)
> 
> Just FYI, it turns out that removing 'query-cpus' has broken Xen,
> whose libxl control library is still using it:
> 
> https://lists.xenproject.org/archives/html/xen-devel/2021-03/msg01178.html
> 
> This suggests we should probably try to coordinate a bit better
> with Xen as one of our downstream QMP users.
> 
> Am I right in thinking that QEMU now has an option that Xen
> could turn on in their CI for "make attempts to use deprecated
> features fail" so they could catch them earlier in the
> deprecate-and-delete cycle?

That would be really great for us.

Thanks,
Ian.

Re: [PULL v2 00/13] misc patches removing deprecated features
Posted by Daniel P. Berrangé 3 years, 1 month ago
On Mon, Mar 22, 2021 at 10:52:02AM +0000, Peter Maydell wrote:
> On Thu, 18 Mar 2021 at 09:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > ----------------------------------------------------------------
> > Remove many old deprecated features
> >
> > The following features have been deprecated for well over the 2
> > release cycle we promise
> >
> >   ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
> >   ``-vnc acl`` (since 4.0.0)
> >   ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
> >   ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
> >   ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
> >   ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
> >   ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
> >   ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
> >   ``query-cpus`` (since 2.12.0)
> >   ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
> >   ``query-events`` (since 4.0)
> >   chardev client socket with ``wait`` option (since 4.0)
> >   ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
> >   ``ide-drive`` (since 4.2)
> >   ``scsi-disk`` (since 4.2)
> 
> Just FYI, it turns out that removing 'query-cpus' has broken Xen,
> whose libxl control library is still using it:
> 
> https://lists.xenproject.org/archives/html/xen-devel/2021-03/msg01178.html

I don't know what Xen's expected min QEMU version is, but if it
can assume QEMU >= 2.12, then there's no need to check for existance
of query-cpus-fast, and can just apply

diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c
index 5d4ec90711..5fdcc182c0 100644
--- a/tools/libs/light/libxl_domain.c
+++ b/tools/libs/light/libxl_domain.c
@@ -1749,14 +1749,14 @@ static int qmp_parse_query_cpus(libxl__gc *gc,
     const libxl__json_object *cpu;
 
     libxl_bitmap_set_none(map);
-    /* Parse response to QMP command "query-cpus":
-     * [ { 'CPU': 'int',...} ]
+    /* Parse response to QMP command "query-cpus-fast":
+     * [ { 'cpu-index': 'int',...} ]
      */
     for (i = 0; (cpu = libxl__json_array_get(response, i)); i++) {
         unsigned int cpu_index;
         const libxl__json_object *o;
 
-        o = libxl__json_map_get("CPU", cpu, JSON_INTEGER);
+        o = libxl__json_map_get("cpu-index", cpu, JSON_INTEGER);
         if (!o) {
             LOGD(ERROR, domid, "Failed to retrieve CPU index.");
             return ERROR_QEMU_API;
@@ -1841,7 +1841,7 @@ int libxl_set_vcpuonline(libxl_ctx *ctx, uint32_t domid,
                                              LIBXL_QMP_CMD_TIMEOUT * 1000);
             if (rc) goto out;
             qmp->callback = set_vcpuonline_qmp_cpus_queried;
-            rc = libxl__ev_qmp_send(egc, qmp, "query-cpus", NULL);
+            rc = libxl__ev_qmp_send(egc, qmp, "query-cpus-fast", NULL);
             if (rc) goto out;
             return AO_INPROGRESS;
         default:
@@ -2199,7 +2199,7 @@ static void retrieve_domain_configuration_lock_acquired(
         libxl_bitmap_alloc(CTX, &rdcs->qemuu_cpus,
                            d_config->b_info.max_vcpus);
         rdcs->qmp.callback = retrieve_domain_configuration_cpu_queried;
-        rc = libxl__ev_qmp_send(egc, &rdcs->qmp, "query-cpus", NULL);
+        rc = libxl__ev_qmp_send(egc, &rdcs->qmp, "query-cpus-fast", NULL);
         if (rc) goto out;
         has_callback = true;
     }


If Xen needs to support QEMU < 2.12, then it will need to
probe for existance of query-cpus-fast using "query-commands"
as libvirt does...

> This suggests we should probably try to coordinate a bit better
> with Xen as one of our downstream QMP users.

They could be added to the MAINTAINERS file for the deprecated features
doc, so that paches automatically CC xen-devel (or another suitable
list or person). 



> Am I right in thinking that QEMU now has an option that Xen
> could turn on in their CI for "make attempts to use deprecated
> features fail" so they could catch them earlier in the
> deprecate-and-delete cycle?

Yes, you just merged it from Markus:

  https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg06803.html

it won't capture all deprecations, but it will catch QMP related ones
and thus I believe would have caught the query-cpus deprecaton.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|