[RFC v2 0/9] error: auto propagated local_err

Vladimir Sementsov-Ogievskiy posted 9 patches 4 years, 7 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
include/hw/pci-host/spapr.h                   |   2 +
include/monitor/hmp.h                         |   2 +-
include/qapi/error.h                          |  61 ++++-
target/ppc/kvm_ppc.h                          |   3 +
target/s390x/cpu_models.h                     |   3 +
ui/vnc.h                                      |   2 +-
audio/audio.c                                 |  12 +-
authz/pamacct.c                               |   1 +
backends/cryptodev-vhost-user.c               |   9 +-
backends/cryptodev.c                          |  24 +-
backends/hostmem-file.c                       |  19 +-
backends/hostmem-memfd.c                      |  17 +-
backends/hostmem.c                            |  38 ++-
backends/rng.c                                |   6 +-
block.c                                       | 208 +++++++----------
block/blkdebug.c                              |  33 +--
block/blklogwrites.c                          |  21 +-
block/blkreplay.c                             |   6 +-
block/blkverify.c                             |  16 +-
block/block-backend.c                         |  17 +-
block/commit.c                                |   6 +-
block/crypto.c                                |  12 +-
block/curl.c                                  |   6 +-
block/file-posix.c                            |  71 +++---
block/file-win32.c                            |  27 +--
block/gluster.c                               |  64 +++--
block/io.c                                    |  11 +-
block/iscsi.c                                 |  33 +--
block/mirror.c                                |  17 +-
block/nbd.c                                   |  44 ++--
block/nfs.c                                   |   6 +-
block/nvme.c                                  |  17 +-
block/parallels.c                             |  28 +--
block/qapi.c                                  |  23 +-
block/qcow.c                                  |  15 +-
block/qcow2-bitmap.c                          |   6 +-
block/qcow2.c                                 |  90 +++----
block/qed.c                                   |  16 +-
block/quorum.c                                |  22 +-
block/raw-format.c                            |   6 +-
block/rbd.c                                   |  25 +-
block/replication.c                           |  36 ++-
block/sheepdog.c                              |  66 +++---
block/snapshot.c                              |  14 +-
block/ssh.c                                   |  11 +-
block/throttle-groups.c                       |  22 +-
block/throttle.c                              |   6 +-
block/vdi.c                                   |  12 +-
block/vhdx.c                                  |  20 +-
block/vmdk.c                                  |  36 +--
block/vpc.c                                   |  25 +-
block/vvfat.c                                 |  11 +-
block/vxhs.c                                  |  22 +-
blockdev.c                                    | 220 +++++++-----------
blockjob.c                                    |   7 +-
bootdevice.c                                  |  29 +--
chardev/char-socket.c                         |   6 +-
chardev/char.c                                |  18 +-
crypto/block-luks.c                           |  31 +--
crypto/secret.c                               |  16 +-
crypto/tlssession.c                           |   6 +-
dump/dump.c                                   | 142 +++++------
dump/win_dump.c                               |  27 +--
exec.c                                        |  12 +-
hw/9pfs/9p-local.c                            |   7 +-
hw/acpi/core.c                                |  17 +-
hw/acpi/ich9.c                                |  27 +--
hw/acpi/memory_hotplug.c                      |   6 +-
hw/arm/allwinner-a10.c                        |  26 +--
hw/arm/armv7m.c                               |  51 ++--
hw/arm/bcm2835_peripherals.c                  |  84 +++----
hw/arm/bcm2836.c                              |  40 ++--
hw/arm/digic.c                                |  21 +-
hw/arm/fsl-imx25.c                            |  61 ++---
hw/arm/fsl-imx31.c                            |  56 ++---
hw/arm/fsl-imx6.c                             |  80 +++----
hw/arm/integratorcp.c                         |   6 +-
hw/arm/msf2-soc.c                             |  21 +-
hw/arm/nrf51_soc.c                            |  46 ++--
hw/arm/smmu-common.c                          |   6 +-
hw/arm/smmuv3.c                               |   6 +-
hw/arm/stm32f205_soc.c                        |  38 ++-
hw/arm/tosa.c                                 |   1 +
hw/arm/xlnx-versal-virt.c                     |   6 +-
hw/arm/xlnx-zynqmp.c                          |  84 +++----
hw/audio/intel-hda.c                          |  12 +-
hw/block/dataplane/xen-block.c                |  16 +-
hw/block/fdc.c                                |  17 +-
hw/block/onenand.c                            |   6 +-
hw/block/pflash_cfi01.c                       |   6 +-
hw/block/pflash_cfi02.c                       |   6 +-
hw/block/vhost-user-blk.c                     |   5 +-
hw/block/virtio-blk.c                         |   6 +-
hw/block/xen-block.c                          | 114 ++++-----
hw/char/debugcon.c                            |   6 +-
hw/char/serial-pci-multi.c                    |   6 +-
hw/char/serial-pci.c                          |   6 +-
hw/char/virtio-serial-bus.c                   |   6 +-
hw/core/bus.c                                 |  14 +-
hw/core/loader-fit.c                          |   2 +-
hw/core/machine.c                             |  18 +-
hw/core/numa.c                                |  48 ++--
hw/core/qdev-properties-system.c              |  24 +-
hw/core/qdev-properties.c                     |  78 +++----
hw/core/qdev.c                                |  36 ++-
hw/core/sysbus.c                              |   1 +
hw/cpu/a15mpcore.c                            |   6 +-
hw/cpu/a9mpcore.c                             |  26 +--
hw/cpu/arm11mpcore.c                          |  21 +-
hw/cpu/core.c                                 |  12 +-
hw/cpu/realview_mpcore.c                      |  11 +-
hw/display/bcm2835_fb.c                       |   5 +-
hw/display/qxl.c                              |   6 +-
hw/display/virtio-gpu-base.c                  |   6 +-
hw/display/virtio-gpu-pci.c                   |   6 +-
hw/display/virtio-vga.c                       |   6 +-
hw/dma/bcm2835_dma.c                          |   5 +-
hw/dma/xilinx_axidma.c                        |  21 +-
hw/gpio/aspeed_gpio.c                         |   6 +-
hw/gpio/bcm2835_gpio.c                        |   9 +-
hw/i386/kvm/apic.c                            |   2 +
hw/i386/pc.c                                  | 109 ++++-----
hw/ide/qdev.c                                 |  15 +-
hw/input/virtio-input.c                       |  12 +-
hw/intc/apic_common.c                         |   6 +-
hw/intc/arm_gic.c                             |   6 +-
hw/intc/arm_gic_kvm.c                         |  11 +-
hw/intc/arm_gicv3.c                           |  11 +-
hw/intc/arm_gicv3_its_kvm.c                   |   6 +-
hw/intc/arm_gicv3_kvm.c                       |  16 +-
hw/intc/armv7m_nvic.c                         |  11 +-
hw/intc/nios2_iic.c                           |   5 +-
hw/intc/pnv_xive.c                            |  14 +-
hw/intc/realview_gic.c                        |   6 +-
hw/intc/s390_flic_kvm.c                       |  10 +-
hw/intc/spapr_xive.c                          |  11 +-
hw/intc/spapr_xive_kvm.c                      |  49 ++--
hw/intc/xics.c                                |  31 +--
hw/intc/xics_kvm.c                            |  28 +--
hw/intc/xics_pnv.c                            |   6 +-
hw/intc/xive.c                                |  23 +-
hw/ipack/ipack.c                              |   4 +-
hw/isa/pc87312.c                              |   6 +-
hw/mem/memory-device.c                        |  19 +-
hw/mem/nvdimm.c                               |  23 +-
hw/mem/pc-dimm.c                              |  21 +-
hw/microblaze/xlnx-zynqmp-pmu.c               |  11 +-
hw/mips/cps.c                                 |  45 ++--
hw/misc/arm11scu.c                            |   2 +
hw/misc/bcm2835_mbox.c                        |   5 +-
hw/misc/bcm2835_property.c                    |   9 +-
hw/misc/ivshmem.c                             |  33 +--
hw/misc/macio/macio.c                         |  65 ++----
hw/misc/mps2-scc.c                            |   2 +
hw/misc/tmp105.c                              |   6 +-
hw/misc/tmp421.c                              |   6 +-
hw/net/dp8393x.c                              |   6 +-
hw/net/eepro100.c                             |   6 +-
hw/net/ne2000-isa.c                           |  16 +-
hw/net/xilinx_axienet.c                       |  21 +-
hw/nvram/fw_cfg.c                             |  12 +-
hw/nvram/nrf51_nvm.c                          |   6 +-
hw/pci-bridge/dec.c                           |   2 +
hw/pci-bridge/gen_pcie_root_port.c            |   6 +-
hw/pci-bridge/pci_bridge_dev.c                |  12 +-
hw/pci-bridge/pci_expander_bridge.c           |   6 +-
hw/pci-bridge/pcie_pci_bridge.c               |   7 +-
hw/pci-host/piix.c                            |   6 +-
hw/pci/pci.c                                  |  17 +-
hw/pci/pcie.c                                 |   6 +-
hw/pci/shpc.c                                 |  12 +-
hw/ppc/e500.c                                 |   6 +-
hw/ppc/pnv.c                                  |  92 +++-----
hw/ppc/pnv_core.c                             |  21 +-
hw/ppc/pnv_lpc.c                              |  22 +-
hw/ppc/pnv_occ.c                              |   4 +-
hw/ppc/pnv_psi.c                              |  21 +-
hw/ppc/spapr.c                                | 124 ++++------
hw/ppc/spapr_caps.c                           |  50 ++--
hw/ppc/spapr_cpu_core.c                       |  33 ++-
hw/ppc/spapr_drc.c                            |  44 ++--
hw/ppc/spapr_irq.c                            |  95 +++-----
hw/ppc/spapr_pci.c                            |  88 +++----
hw/ppc/spapr_vio.c                            |  11 +-
hw/riscv/riscv_hart.c                         |   7 +-
hw/riscv/sifive_e.c                           |   6 +-
hw/riscv/sifive_u.c                           |  10 +-
hw/s390x/3270-ccw.c                           |  12 +-
hw/s390x/css-bridge.c                         |   6 +-
hw/s390x/css.c                                |   6 +-
hw/s390x/ipl.c                                |  23 +-
hw/s390x/s390-ccw.c                           |  17 +-
hw/s390x/s390-pci-bus.c                       |  34 ++-
hw/s390x/s390-skeys.c                         |   6 +-
hw/s390x/s390-virtio-ccw.c                    |  10 +-
hw/s390x/sclp.c                               |  14 +-
hw/s390x/tod-kvm.c                            |  13 +-
hw/s390x/virtio-ccw-crypto.c                  |   6 +-
hw/s390x/virtio-ccw-rng.c                     |   6 +-
hw/s390x/virtio-ccw.c                         |  12 +-
hw/scsi/esp-pci.c                             |   6 +-
hw/scsi/megasas.c                             |  10 +-
hw/scsi/mptsas.c                              |  12 +-
hw/scsi/scsi-bus.c                            |  22 +-
hw/scsi/scsi-disk.c                           |   6 +-
hw/scsi/vhost-scsi.c                          |  11 +-
hw/scsi/vhost-user-scsi.c                     |   6 +-
hw/scsi/virtio-scsi.c                         |   6 +-
hw/sd/milkymist-memcard.c                     |  10 +-
hw/sd/sdhci-pci.c                             |   6 +-
hw/sd/sdhci.c                                 |  20 +-
hw/sd/ssi-sd.c                                |  13 +-
hw/smbios/smbios.c                            |  41 ++--
hw/sparc/sun4m.c                              |  18 +-
hw/sparc64/sun4u.c                            |   6 +-
hw/timer/aspeed_timer.c                       |   6 +-
hw/tpm/tpm_util.c                             |   6 +-
hw/usb/bus.c                                  |  33 +--
hw/usb/dev-serial.c                           |   6 +-
hw/usb/dev-smartcard-reader.c                 |  12 +-
hw/usb/dev-storage.c                          |  16 +-
hw/usb/hcd-ohci-pci.c                         |   6 +-
hw/usb/hcd-ohci.c                             |  12 +-
hw/usb/hcd-uhci.c                             |   6 +-
hw/usb/hcd-xhci.c                             |  12 +-
hw/vfio/ap.c                                  |   9 +-
hw/vfio/ccw.c                                 |  23 +-
hw/vfio/pci-quirks.c                          |   6 +-
hw/vfio/pci.c                                 |  36 ++-
hw/virtio/virtio-balloon.c                    |  33 ++-
hw/virtio/virtio-bus.c                        |  16 +-
hw/virtio/virtio-rng-pci.c                    |   6 +-
hw/virtio/virtio-rng.c                        |   6 +-
hw/virtio/virtio.c                            |  17 +-
hw/watchdog/wdt_aspeed.c                      |   4 +-
hw/xen/xen-backend.c                          |   6 +-
hw/xen/xen-bus.c                              |  85 +++----
hw/xen/xen-host-pci-device.c                  |  26 +--
hw/xen/xen_pt.c                               |  24 +-
hw/xen/xen_pt_config_init.c                   |  19 +-
io/dns-resolver.c                             |   6 +-
io/net-listener.c                             |   6 +-
iothread.c                                    |  25 +-
job.c                                         |   6 +-
memory.c                                      |  54 ++---
memory_mapping.c                              |   6 +-
migration/colo.c                              |  33 +--
migration/migration.c                         |  35 ++-
migration/ram.c                               |  12 +-
migration/rdma.c                              |  11 +-
migration/socket.c                            |  16 +-
monitor/hmp-cmds.c                            |   8 +-
monitor/misc.c                                |   8 +-
monitor/qmp-cmds.c                            |   6 +-
net/can/can_host.c                            |   6 +-
net/dump.c                                    |  14 +-
net/filter-buffer.c                           |  14 +-
net/filter.c                                  |   6 +-
net/net.c                                     |  44 ++--
net/netmap.c                                  |   6 +-
net/slirp.c                                   |   6 +-
net/tap-bsd.c                                 |   1 +
net/tap-solaris.c                             |   1 +
net/tap-stub.c                                |   1 +
net/tap.c                                     |  44 ++--
qapi/opts-visitor.c                           |   1 +
qapi/qapi-dealloc-visitor.c                   |   8 +
qapi/qapi-visit-core.c                        |  53 ++---
qapi/qmp-dispatch.c                           |   6 +-
qapi/string-input-visitor.c                   |   6 +-
qdev-monitor.c                                |  37 ++-
qga/commands-posix.c                          | 197 ++++++----------
qga/commands-win32.c                          | 127 ++++------
qom/object.c                                  | 187 ++++++---------
qom/object_interfaces.c                       |  26 +--
qom/qom-qobject.c                             |   6 +-
replication.c                                 |  24 +-
scsi/pr-manager-helper.c                      |   6 +-
stubs/xen-hvm.c                               |   3 +
target/alpha/cpu.c                            |   6 +-
target/arm/cpu.c                              |   6 +-
target/arm/cpu64.c                            |  10 +-
target/cris/cpu.c                             |   6 +-
target/hppa/cpu.c                             |   6 +-
target/i386/cpu.c                             | 107 ++++-----
target/lm32/cpu.c                             |   6 +-
target/m68k/cpu.c                             |   6 +-
target/microblaze/cpu.c                       |   6 +-
target/mips/cpu.c                             |   6 +-
target/moxie/cpu.c                            |   6 +-
target/nios2/cpu.c                            |   6 +-
target/openrisc/cpu.c                         |   6 +-
target/ppc/compat.c                           |  18 +-
target/ppc/kvm.c                              |   6 +-
target/ppc/translate_init.inc.c               |  23 +-
target/riscv/cpu.c                            |   6 +-
target/s390x/cpu.c                            |  25 +-
target/s390x/kvm-stub.c                       |   1 +
target/sh4/cpu.c                              |   6 +-
target/sparc/cpu.c                            |  12 +-
target/tilegx/cpu.c                           |   6 +-
target/tricore/cpu.c                          |   6 +-
target/unicore32/cpu.c                        |   6 +-
target/xtensa/cpu.c                           |   6 +-
tests/test-image-locking.c                    |   6 +-
tests/test-qmp-cmds.c                         |   7 +
tpm.c                                         |   6 +-
trace/qmp.c                                   |  12 +-
ui/input-barrier.c                            |   6 +-
ui/input.c                                    |  12 +-
ui/vnc.c                                      |  27 +--
util/error.c                                  |   8 +-
util/main-loop.c                              |   4 +-
util/oslib-posix.c                            |   5 +-
util/qemu-config.c                            |  27 +--
util/qemu-option.c                            |  50 ++--
util/qemu-sockets.c                           |  25 +-
vl.c                                          |  13 +-
scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
319 files changed, 2729 insertions(+), 4245 deletions(-)
create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
[RFC v2 0/9] error: auto propagated local_err
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
Hi all!

Here is a proposal of auto propagation for local_err, to not call
error_propagate on every exit point, when we deal with local_err.

It also fixes two issues:
1. Fix issue with error_fatal & error_append_hint: user can't see these
hints, because exit() happens in error_setg earlier than hint is
appended. [Reported by Greg Kurz]

2. Fix issue with error_abort & error_propagate: when we wrap
error_abort by local_err+error_propagate, resulting coredump will
refer to error_propagate and not to the place where error happened.
(the macro itself don't fix the issue, but it allows to [3.] drop all
local_err+error_propagate pattern, which will definitely fix the issue)
[Reported by Kevin Wolf]

It's still an RFC, due to the following reasons:

1. I'm new to coccinella, so I failed to do the following pattern:

 <...
- goto out;
+ return;
 ...>
- out:
- error_propagate(errp, local_err)

So, here is compilation fix 08.. Who can help with it? If nobody, 08 is
to be merged to 07 by hand.

2. Question about using new macro in empty stub functions - see 09

3. What to do with huge auto-generated commit 07? Should I split it
per-maintainer or per-subsystem, or leave it as-is?

4. Also, checkpatch has some complains about 07 patch:
  - using tabs.. (hmm exactly stubs functions..)
  - empty ifs
  Again, I don't see any ways to fix it other that by hand and merge to
  07..

==================

Also, if we decide, that this all is too huge, here is plan B:

1. apply 01
2. fix only functions that don't use local_err and use
error_append_hint, by just invocation of new macro at function start -
it will substitute Greg's series with no pain.
3[optional]. Do full update for some subsystems, for example, only for
block* and nbd*

Vladimir Sementsov-Ogievskiy (9):
  error: auto propagated local_err
  qapi/error: add (Error **errp) cleaning APIs
  errp: rename errp to errp_in where it is IN-argument
  hw/core/loader-fit: fix freeing errp in fit_load_fdt
  net/net: fix local variable shadowing in net_client_init
  scripts: add coccinelle script to use auto propagated errp
  Use auto-propagated errp
  fix-compilation: empty goto
  fix-compilation: includes

 include/hw/pci-host/spapr.h                   |   2 +
 include/monitor/hmp.h                         |   2 +-
 include/qapi/error.h                          |  61 ++++-
 target/ppc/kvm_ppc.h                          |   3 +
 target/s390x/cpu_models.h                     |   3 +
 ui/vnc.h                                      |   2 +-
 audio/audio.c                                 |  12 +-
 authz/pamacct.c                               |   1 +
 backends/cryptodev-vhost-user.c               |   9 +-
 backends/cryptodev.c                          |  24 +-
 backends/hostmem-file.c                       |  19 +-
 backends/hostmem-memfd.c                      |  17 +-
 backends/hostmem.c                            |  38 ++-
 backends/rng.c                                |   6 +-
 block.c                                       | 208 +++++++----------
 block/blkdebug.c                              |  33 +--
 block/blklogwrites.c                          |  21 +-
 block/blkreplay.c                             |   6 +-
 block/blkverify.c                             |  16 +-
 block/block-backend.c                         |  17 +-
 block/commit.c                                |   6 +-
 block/crypto.c                                |  12 +-
 block/curl.c                                  |   6 +-
 block/file-posix.c                            |  71 +++---
 block/file-win32.c                            |  27 +--
 block/gluster.c                               |  64 +++--
 block/io.c                                    |  11 +-
 block/iscsi.c                                 |  33 +--
 block/mirror.c                                |  17 +-
 block/nbd.c                                   |  44 ++--
 block/nfs.c                                   |   6 +-
 block/nvme.c                                  |  17 +-
 block/parallels.c                             |  28 +--
 block/qapi.c                                  |  23 +-
 block/qcow.c                                  |  15 +-
 block/qcow2-bitmap.c                          |   6 +-
 block/qcow2.c                                 |  90 +++----
 block/qed.c                                   |  16 +-
 block/quorum.c                                |  22 +-
 block/raw-format.c                            |   6 +-
 block/rbd.c                                   |  25 +-
 block/replication.c                           |  36 ++-
 block/sheepdog.c                              |  66 +++---
 block/snapshot.c                              |  14 +-
 block/ssh.c                                   |  11 +-
 block/throttle-groups.c                       |  22 +-
 block/throttle.c                              |   6 +-
 block/vdi.c                                   |  12 +-
 block/vhdx.c                                  |  20 +-
 block/vmdk.c                                  |  36 +--
 block/vpc.c                                   |  25 +-
 block/vvfat.c                                 |  11 +-
 block/vxhs.c                                  |  22 +-
 blockdev.c                                    | 220 +++++++-----------
 blockjob.c                                    |   7 +-
 bootdevice.c                                  |  29 +--
 chardev/char-socket.c                         |   6 +-
 chardev/char.c                                |  18 +-
 crypto/block-luks.c                           |  31 +--
 crypto/secret.c                               |  16 +-
 crypto/tlssession.c                           |   6 +-
 dump/dump.c                                   | 142 +++++------
 dump/win_dump.c                               |  27 +--
 exec.c                                        |  12 +-
 hw/9pfs/9p-local.c                            |   7 +-
 hw/acpi/core.c                                |  17 +-
 hw/acpi/ich9.c                                |  27 +--
 hw/acpi/memory_hotplug.c                      |   6 +-
 hw/arm/allwinner-a10.c                        |  26 +--
 hw/arm/armv7m.c                               |  51 ++--
 hw/arm/bcm2835_peripherals.c                  |  84 +++----
 hw/arm/bcm2836.c                              |  40 ++--
 hw/arm/digic.c                                |  21 +-
 hw/arm/fsl-imx25.c                            |  61 ++---
 hw/arm/fsl-imx31.c                            |  56 ++---
 hw/arm/fsl-imx6.c                             |  80 +++----
 hw/arm/integratorcp.c                         |   6 +-
 hw/arm/msf2-soc.c                             |  21 +-
 hw/arm/nrf51_soc.c                            |  46 ++--
 hw/arm/smmu-common.c                          |   6 +-
 hw/arm/smmuv3.c                               |   6 +-
 hw/arm/stm32f205_soc.c                        |  38 ++-
 hw/arm/tosa.c                                 |   1 +
 hw/arm/xlnx-versal-virt.c                     |   6 +-
 hw/arm/xlnx-zynqmp.c                          |  84 +++----
 hw/audio/intel-hda.c                          |  12 +-
 hw/block/dataplane/xen-block.c                |  16 +-
 hw/block/fdc.c                                |  17 +-
 hw/block/onenand.c                            |   6 +-
 hw/block/pflash_cfi01.c                       |   6 +-
 hw/block/pflash_cfi02.c                       |   6 +-
 hw/block/vhost-user-blk.c                     |   5 +-
 hw/block/virtio-blk.c                         |   6 +-
 hw/block/xen-block.c                          | 114 ++++-----
 hw/char/debugcon.c                            |   6 +-
 hw/char/serial-pci-multi.c                    |   6 +-
 hw/char/serial-pci.c                          |   6 +-
 hw/char/virtio-serial-bus.c                   |   6 +-
 hw/core/bus.c                                 |  14 +-
 hw/core/loader-fit.c                          |   2 +-
 hw/core/machine.c                             |  18 +-
 hw/core/numa.c                                |  48 ++--
 hw/core/qdev-properties-system.c              |  24 +-
 hw/core/qdev-properties.c                     |  78 +++----
 hw/core/qdev.c                                |  36 ++-
 hw/core/sysbus.c                              |   1 +
 hw/cpu/a15mpcore.c                            |   6 +-
 hw/cpu/a9mpcore.c                             |  26 +--
 hw/cpu/arm11mpcore.c                          |  21 +-
 hw/cpu/core.c                                 |  12 +-
 hw/cpu/realview_mpcore.c                      |  11 +-
 hw/display/bcm2835_fb.c                       |   5 +-
 hw/display/qxl.c                              |   6 +-
 hw/display/virtio-gpu-base.c                  |   6 +-
 hw/display/virtio-gpu-pci.c                   |   6 +-
 hw/display/virtio-vga.c                       |   6 +-
 hw/dma/bcm2835_dma.c                          |   5 +-
 hw/dma/xilinx_axidma.c                        |  21 +-
 hw/gpio/aspeed_gpio.c                         |   6 +-
 hw/gpio/bcm2835_gpio.c                        |   9 +-
 hw/i386/kvm/apic.c                            |   2 +
 hw/i386/pc.c                                  | 109 ++++-----
 hw/ide/qdev.c                                 |  15 +-
 hw/input/virtio-input.c                       |  12 +-
 hw/intc/apic_common.c                         |   6 +-
 hw/intc/arm_gic.c                             |   6 +-
 hw/intc/arm_gic_kvm.c                         |  11 +-
 hw/intc/arm_gicv3.c                           |  11 +-
 hw/intc/arm_gicv3_its_kvm.c                   |   6 +-
 hw/intc/arm_gicv3_kvm.c                       |  16 +-
 hw/intc/armv7m_nvic.c                         |  11 +-
 hw/intc/nios2_iic.c                           |   5 +-
 hw/intc/pnv_xive.c                            |  14 +-
 hw/intc/realview_gic.c                        |   6 +-
 hw/intc/s390_flic_kvm.c                       |  10 +-
 hw/intc/spapr_xive.c                          |  11 +-
 hw/intc/spapr_xive_kvm.c                      |  49 ++--
 hw/intc/xics.c                                |  31 +--
 hw/intc/xics_kvm.c                            |  28 +--
 hw/intc/xics_pnv.c                            |   6 +-
 hw/intc/xive.c                                |  23 +-
 hw/ipack/ipack.c                              |   4 +-
 hw/isa/pc87312.c                              |   6 +-
 hw/mem/memory-device.c                        |  19 +-
 hw/mem/nvdimm.c                               |  23 +-
 hw/mem/pc-dimm.c                              |  21 +-
 hw/microblaze/xlnx-zynqmp-pmu.c               |  11 +-
 hw/mips/cps.c                                 |  45 ++--
 hw/misc/arm11scu.c                            |   2 +
 hw/misc/bcm2835_mbox.c                        |   5 +-
 hw/misc/bcm2835_property.c                    |   9 +-
 hw/misc/ivshmem.c                             |  33 +--
 hw/misc/macio/macio.c                         |  65 ++----
 hw/misc/mps2-scc.c                            |   2 +
 hw/misc/tmp105.c                              |   6 +-
 hw/misc/tmp421.c                              |   6 +-
 hw/net/dp8393x.c                              |   6 +-
 hw/net/eepro100.c                             |   6 +-
 hw/net/ne2000-isa.c                           |  16 +-
 hw/net/xilinx_axienet.c                       |  21 +-
 hw/nvram/fw_cfg.c                             |  12 +-
 hw/nvram/nrf51_nvm.c                          |   6 +-
 hw/pci-bridge/dec.c                           |   2 +
 hw/pci-bridge/gen_pcie_root_port.c            |   6 +-
 hw/pci-bridge/pci_bridge_dev.c                |  12 +-
 hw/pci-bridge/pci_expander_bridge.c           |   6 +-
 hw/pci-bridge/pcie_pci_bridge.c               |   7 +-
 hw/pci-host/piix.c                            |   6 +-
 hw/pci/pci.c                                  |  17 +-
 hw/pci/pcie.c                                 |   6 +-
 hw/pci/shpc.c                                 |  12 +-
 hw/ppc/e500.c                                 |   6 +-
 hw/ppc/pnv.c                                  |  92 +++-----
 hw/ppc/pnv_core.c                             |  21 +-
 hw/ppc/pnv_lpc.c                              |  22 +-
 hw/ppc/pnv_occ.c                              |   4 +-
 hw/ppc/pnv_psi.c                              |  21 +-
 hw/ppc/spapr.c                                | 124 ++++------
 hw/ppc/spapr_caps.c                           |  50 ++--
 hw/ppc/spapr_cpu_core.c                       |  33 ++-
 hw/ppc/spapr_drc.c                            |  44 ++--
 hw/ppc/spapr_irq.c                            |  95 +++-----
 hw/ppc/spapr_pci.c                            |  88 +++----
 hw/ppc/spapr_vio.c                            |  11 +-
 hw/riscv/riscv_hart.c                         |   7 +-
 hw/riscv/sifive_e.c                           |   6 +-
 hw/riscv/sifive_u.c                           |  10 +-
 hw/s390x/3270-ccw.c                           |  12 +-
 hw/s390x/css-bridge.c                         |   6 +-
 hw/s390x/css.c                                |   6 +-
 hw/s390x/ipl.c                                |  23 +-
 hw/s390x/s390-ccw.c                           |  17 +-
 hw/s390x/s390-pci-bus.c                       |  34 ++-
 hw/s390x/s390-skeys.c                         |   6 +-
 hw/s390x/s390-virtio-ccw.c                    |  10 +-
 hw/s390x/sclp.c                               |  14 +-
 hw/s390x/tod-kvm.c                            |  13 +-
 hw/s390x/virtio-ccw-crypto.c                  |   6 +-
 hw/s390x/virtio-ccw-rng.c                     |   6 +-
 hw/s390x/virtio-ccw.c                         |  12 +-
 hw/scsi/esp-pci.c                             |   6 +-
 hw/scsi/megasas.c                             |  10 +-
 hw/scsi/mptsas.c                              |  12 +-
 hw/scsi/scsi-bus.c                            |  22 +-
 hw/scsi/scsi-disk.c                           |   6 +-
 hw/scsi/vhost-scsi.c                          |  11 +-
 hw/scsi/vhost-user-scsi.c                     |   6 +-
 hw/scsi/virtio-scsi.c                         |   6 +-
 hw/sd/milkymist-memcard.c                     |  10 +-
 hw/sd/sdhci-pci.c                             |   6 +-
 hw/sd/sdhci.c                                 |  20 +-
 hw/sd/ssi-sd.c                                |  13 +-
 hw/smbios/smbios.c                            |  41 ++--
 hw/sparc/sun4m.c                              |  18 +-
 hw/sparc64/sun4u.c                            |   6 +-
 hw/timer/aspeed_timer.c                       |   6 +-
 hw/tpm/tpm_util.c                             |   6 +-
 hw/usb/bus.c                                  |  33 +--
 hw/usb/dev-serial.c                           |   6 +-
 hw/usb/dev-smartcard-reader.c                 |  12 +-
 hw/usb/dev-storage.c                          |  16 +-
 hw/usb/hcd-ohci-pci.c                         |   6 +-
 hw/usb/hcd-ohci.c                             |  12 +-
 hw/usb/hcd-uhci.c                             |   6 +-
 hw/usb/hcd-xhci.c                             |  12 +-
 hw/vfio/ap.c                                  |   9 +-
 hw/vfio/ccw.c                                 |  23 +-
 hw/vfio/pci-quirks.c                          |   6 +-
 hw/vfio/pci.c                                 |  36 ++-
 hw/virtio/virtio-balloon.c                    |  33 ++-
 hw/virtio/virtio-bus.c                        |  16 +-
 hw/virtio/virtio-rng-pci.c                    |   6 +-
 hw/virtio/virtio-rng.c                        |   6 +-
 hw/virtio/virtio.c                            |  17 +-
 hw/watchdog/wdt_aspeed.c                      |   4 +-
 hw/xen/xen-backend.c                          |   6 +-
 hw/xen/xen-bus.c                              |  85 +++----
 hw/xen/xen-host-pci-device.c                  |  26 +--
 hw/xen/xen_pt.c                               |  24 +-
 hw/xen/xen_pt_config_init.c                   |  19 +-
 io/dns-resolver.c                             |   6 +-
 io/net-listener.c                             |   6 +-
 iothread.c                                    |  25 +-
 job.c                                         |   6 +-
 memory.c                                      |  54 ++---
 memory_mapping.c                              |   6 +-
 migration/colo.c                              |  33 +--
 migration/migration.c                         |  35 ++-
 migration/ram.c                               |  12 +-
 migration/rdma.c                              |  11 +-
 migration/socket.c                            |  16 +-
 monitor/hmp-cmds.c                            |   8 +-
 monitor/misc.c                                |   8 +-
 monitor/qmp-cmds.c                            |   6 +-
 net/can/can_host.c                            |   6 +-
 net/dump.c                                    |  14 +-
 net/filter-buffer.c                           |  14 +-
 net/filter.c                                  |   6 +-
 net/net.c                                     |  44 ++--
 net/netmap.c                                  |   6 +-
 net/slirp.c                                   |   6 +-
 net/tap-bsd.c                                 |   1 +
 net/tap-solaris.c                             |   1 +
 net/tap-stub.c                                |   1 +
 net/tap.c                                     |  44 ++--
 qapi/opts-visitor.c                           |   1 +
 qapi/qapi-dealloc-visitor.c                   |   8 +
 qapi/qapi-visit-core.c                        |  53 ++---
 qapi/qmp-dispatch.c                           |   6 +-
 qapi/string-input-visitor.c                   |   6 +-
 qdev-monitor.c                                |  37 ++-
 qga/commands-posix.c                          | 197 ++++++----------
 qga/commands-win32.c                          | 127 ++++------
 qom/object.c                                  | 187 ++++++---------
 qom/object_interfaces.c                       |  26 +--
 qom/qom-qobject.c                             |   6 +-
 replication.c                                 |  24 +-
 scsi/pr-manager-helper.c                      |   6 +-
 stubs/xen-hvm.c                               |   3 +
 target/alpha/cpu.c                            |   6 +-
 target/arm/cpu.c                              |   6 +-
 target/arm/cpu64.c                            |  10 +-
 target/cris/cpu.c                             |   6 +-
 target/hppa/cpu.c                             |   6 +-
 target/i386/cpu.c                             | 107 ++++-----
 target/lm32/cpu.c                             |   6 +-
 target/m68k/cpu.c                             |   6 +-
 target/microblaze/cpu.c                       |   6 +-
 target/mips/cpu.c                             |   6 +-
 target/moxie/cpu.c                            |   6 +-
 target/nios2/cpu.c                            |   6 +-
 target/openrisc/cpu.c                         |   6 +-
 target/ppc/compat.c                           |  18 +-
 target/ppc/kvm.c                              |   6 +-
 target/ppc/translate_init.inc.c               |  23 +-
 target/riscv/cpu.c                            |   6 +-
 target/s390x/cpu.c                            |  25 +-
 target/s390x/kvm-stub.c                       |   1 +
 target/sh4/cpu.c                              |   6 +-
 target/sparc/cpu.c                            |  12 +-
 target/tilegx/cpu.c                           |   6 +-
 target/tricore/cpu.c                          |   6 +-
 target/unicore32/cpu.c                        |   6 +-
 target/xtensa/cpu.c                           |   6 +-
 tests/test-image-locking.c                    |   6 +-
 tests/test-qmp-cmds.c                         |   7 +
 tpm.c                                         |   6 +-
 trace/qmp.c                                   |  12 +-
 ui/input-barrier.c                            |   6 +-
 ui/input.c                                    |  12 +-
 ui/vnc.c                                      |  27 +--
 util/error.c                                  |   8 +-
 util/main-loop.c                              |   4 +-
 util/oslib-posix.c                            |   5 +-
 util/qemu-config.c                            |  27 +--
 util/qemu-option.c                            |  50 ++--
 util/qemu-sockets.c                           |  25 +-
 vl.c                                          |  13 +-
 scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
 319 files changed, 2729 insertions(+), 4245 deletions(-)
 create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

-- 
2.21.0


Re: [RFC v2 0/9] error: auto propagated local_err
Posted by Eric Blake 4 years, 7 months ago
On 9/23/19 11:12 AM, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> Here is a proposal of auto propagation for local_err, to not call
> error_propagate on every exit point, when we deal with local_err.
> 
> It also fixes two issues:
> 1. Fix issue with error_fatal & error_append_hint: user can't see these
> hints, because exit() happens in error_setg earlier than hint is
> appended. [Reported by Greg Kurz]
> 
> 2. Fix issue with error_abort & error_propagate: when we wrap
> error_abort by local_err+error_propagate, resulting coredump will
> refer to error_propagate and not to the place where error happened.
> (the macro itself don't fix the issue, but it allows to [3.] drop all

doesn't

> local_err+error_propagate pattern, which will definitely fix the issue)
> [Reported by Kevin Wolf]
> 
> It's still an RFC, due to the following reasons:
> 
> 1. I'm new to coccinella, so I failed to do the following pattern:
> 
>  <...
> - goto out;
> + return;
>  ...>
> - out:
> - error_propagate(errp, local_err)
> 
> So, here is compilation fix 08.. Who can help with it? If nobody, 08 is
> to be merged to 07 by hand.

I'm not sure either; but I agree that if we can't figure out how to make
Coccinelle do quite what we want, that we are better off squashing in
compile fixes.

Also, while I like Coccinelle for automating the conversion, it's harder
to get everyone to run it; it would be nice if we could also figure out
a patch to scripts/checkpatch.pl that for any instance of 'Error
**errp)\n{\n' not followed by either } or the new macro, we flag that as
a checkpatch warning or error.

> 
> 2. Question about using new macro in empty stub functions - see 09

It would be nice if we could exempt empty functions - no need to use the
macro if there is no function body otherwise.  I'm not sure if
Coccinelle can do that filtering during the conversion, or if we clean
up by hand after the fact.

> 
> 3. What to do with huge auto-generated commit 07? Should I split it
> per-maintainer or per-subsystem, or leave it as-is?

It's big. I'd split it into multiple patches (and reduce the cc - except
for the cover letter, the rest of the patches can be limited to the
actual maintainer/subsystem affected rather than everyone involved
anywhere else in the series. With the current large cc, anyone that
replies gets several mail bounces about "too many recipients").  It may
be easier to split along directory boundaries than by maintainer
boundaries.  Markus has applied large tree-wide Coccinelle cleanups
before, maybe he has some advice.

> 
> 4. Also, checkpatch has some complains about 07 patch:
>   - using tabs.. (hmm exactly stubs functions..)
>   - empty ifs
>   Again, I don't see any ways to fix it other that by hand and merge to
>   07..

Hand cleanups for formatting or compilation fixes to Coccinelle's work
is not an uncommon issue after large patches; thankfully it's also not
very difficult (and surprisingly needed in very few places compared to
how much actually gets touched).

> 
> ==================
> 
> Also, if we decide, that this all is too huge, here is plan B:
> 
> 1. apply 01
> 2. fix only functions that don't use local_err and use
> error_append_hint, by just invocation of new macro at function start -
> it will substitute Greg's series with no pain.
> 3[optional]. Do full update for some subsystems, for example, only for
> block* and nbd*

Even if we go with plan B, it's still worth checking in a Coccinelle
script that we can periodically run to make sure we aren't missing out
on the use of the macro where it is needed.

> 
> Vladimir Sementsov-Ogievskiy (9):
>   error: auto propagated local_err
>   qapi/error: add (Error **errp) cleaning APIs
>   errp: rename errp to errp_in where it is IN-argument
>   hw/core/loader-fit: fix freeing errp in fit_load_fdt
>   net/net: fix local variable shadowing in net_client_init
>   scripts: add coccinelle script to use auto propagated errp
>   Use auto-propagated errp
>   fix-compilation: empty goto
>   fix-compilation: includes
> 
>  include/hw/pci-host/spapr.h                   |   2 +
>  include/monitor/hmp.h                         |   2 +-
>  include/qapi/error.h                          |  61 ++++-
>  target/ppc/kvm_ppc.h                          |   3 +
>  target/s390x/cpu_models.h                     |   3 +
>  ui/vnc.h                                      |   2 +-

>  vl.c                                          |  13 +-
>  scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
>  319 files changed, 2729 insertions(+), 4245 deletions(-)
>  create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

The diffstat is huge, but promising.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [Xen-devel] [RFC v2 0/9] error: auto propagated local_err
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
23.09.2019 22:47, Eric Blake wrote:
> On 9/23/19 11:12 AM, Vladimir Sementsov-Ogievskiy wrote:
>> Hi all!
>>
>> Here is a proposal of auto propagation for local_err, to not call
>> error_propagate on every exit point, when we deal with local_err.
>>
>> It also fixes two issues:
>> 1. Fix issue with error_fatal & error_append_hint: user can't see these
>> hints, because exit() happens in error_setg earlier than hint is
>> appended. [Reported by Greg Kurz]
>>
>> 2. Fix issue with error_abort & error_propagate: when we wrap
>> error_abort by local_err+error_propagate, resulting coredump will
>> refer to error_propagate and not to the place where error happened.
>> (the macro itself don't fix the issue, but it allows to [3.] drop all
> 
> doesn't
> 
>> local_err+error_propagate pattern, which will definitely fix the issue)
>> [Reported by Kevin Wolf]
>>
>> It's still an RFC, due to the following reasons:
>>
>> 1. I'm new to coccinella, so I failed to do the following pattern:
>>
>>   <...
>> - goto out;
>> + return;
>>   ...>
>> - out:
>> - error_propagate(errp, local_err)
>>
>> So, here is compilation fix 08.. Who can help with it? If nobody, 08 is
>> to be merged to 07 by hand.
> 
> I'm not sure either; but I agree that if we can't figure out how to make
> Coccinelle do quite what we want, that we are better off squashing in
> compile fixes.
> 
> Also, while I like Coccinelle for automating the conversion, it's harder
> to get everyone to run it; it would be nice if we could also figure out
> a patch to scripts/checkpatch.pl that for any instance of 'Error
> **errp)\n{\n' not followed by either } or the new macro, we flag that as
> a checkpatch warning or error.
> 
>>
>> 2. Question about using new macro in empty stub functions - see 09
> 
> It would be nice if we could exempt empty functions - no need to use the
> macro if there is no function body otherwise.  I'm not sure if
> Coccinelle can do that filtering during the conversion, or if we clean
> up by hand after the fact.
> 
>>
>> 3. What to do with huge auto-generated commit 07? Should I split it
>> per-maintainer or per-subsystem, or leave it as-is?
> 
> It's big. I'd split it into multiple patches (and reduce the cc - except
> for the cover letter, the rest of the patches can be limited to the
> actual maintainer/subsystem affected rather than everyone involved
> anywhere else in the series. With the current large cc, anyone that
> replies gets several mail bounces about "too many recipients").  It may
> be easier to split along directory boundaries than by maintainer
> boundaries.  Markus has applied large tree-wide Coccinelle cleanups
> before, maybe he has some advice.


If split by subsystem it would be 200+ patches:
git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | head -1; done | sort | uniq | wc -l
205


Try to look at larger subsystem:
git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | tail -2 | head -1; done | sort | uniq | wc -l
139

still too many.. Or is it OK?


> 
>>
>> 4. Also, checkpatch has some complains about 07 patch:
>>    - using tabs.. (hmm exactly stubs functions..)
>>    - empty ifs
>>    Again, I don't see any ways to fix it other that by hand and merge to
>>    07..
> 
> Hand cleanups for formatting or compilation fixes to Coccinelle's work
> is not an uncommon issue after large patches; thankfully it's also not
> very difficult (and surprisingly needed in very few places compared to
> how much actually gets touched).
> 
>>
>> ==================
>>
>> Also, if we decide, that this all is too huge, here is plan B:
>>
>> 1. apply 01
>> 2. fix only functions that don't use local_err and use
>> error_append_hint, by just invocation of new macro at function start -
>> it will substitute Greg's series with no pain.
>> 3[optional]. Do full update for some subsystems, for example, only for
>> block* and nbd*
> 
> Even if we go with plan B, it's still worth checking in a Coccinelle
> script that we can periodically run to make sure we aren't missing out
> on the use of the macro where it is needed.
> 
>>
>> Vladimir Sementsov-Ogievskiy (9):
>>    error: auto propagated local_err
>>    qapi/error: add (Error **errp) cleaning APIs
>>    errp: rename errp to errp_in where it is IN-argument
>>    hw/core/loader-fit: fix freeing errp in fit_load_fdt
>>    net/net: fix local variable shadowing in net_client_init
>>    scripts: add coccinelle script to use auto propagated errp
>>    Use auto-propagated errp
>>    fix-compilation: empty goto
>>    fix-compilation: includes
>>
>>   include/hw/pci-host/spapr.h                   |   2 +
>>   include/monitor/hmp.h                         |   2 +-
>>   include/qapi/error.h                          |  61 ++++-
>>   target/ppc/kvm_ppc.h                          |   3 +
>>   target/s390x/cpu_models.h                     |   3 +
>>   ui/vnc.h                                      |   2 +-
> 
>>   vl.c                                          |  13 +-
>>   scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
>>   319 files changed, 2729 insertions(+), 4245 deletions(-)
>>   create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
> 
> The diffstat is huge, but promising.
> 


-- 
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [RFC v2 0/9] error: auto propagated local_err
Posted by Eric Blake 4 years, 7 months ago
On 9/24/19 9:12 AM, Vladimir Sementsov-Ogievskiy wrote:

>>> 3. What to do with huge auto-generated commit 07? Should I split it
>>> per-maintainer or per-subsystem, or leave it as-is?
>>
>> It's big. I'd split it into multiple patches (and reduce the cc - except
>> for the cover letter, the rest of the patches can be limited to the
>> actual maintainer/subsystem affected rather than everyone involved
>> anywhere else in the series. With the current large cc, anyone that
>> replies gets several mail bounces about "too many recipients").  It may
>> be easier to split along directory boundaries than by maintainer
>> boundaries.  Markus has applied large tree-wide Coccinelle cleanups
>> before, maybe he has some advice.
> 
> 
> If split by subsystem it would be 200+ patches:
> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | head -1; done | sort | uniq | wc -l
> 205
> 
> 
> Try to look at larger subsystem:
> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | tail -2 | head -1; done | sort | uniq | wc -l
> 139
> 
> still too many.. Or is it OK?

Hmm - that becomes a tradeoff in length of the series (where individual
patches may be reviewed fast, but where the overall process may be
bogged down by sheer length), vs. length of individual emails (where the
email itself is daunting, but as the review is mechanical and done by
automation, it becomes a matter of spot-checking if we trust that the
automation was done correctly).  You can probably group it in fewer
patches, by joining smaller patches across a couple of subsystems.  It's
an art form, there's probably several ways to do it that would work, and
it comes down to a judgment call on how much work you want to do to try
and reduce other's work in reviewing it.  Maybe even an off-hand split
of gathering files until you reach about 500 or so lines per diff.  I
wish I had easier advice on how to tackle this sort of project in the
way that will get the fastest response time.


>>>   vl.c                                          |  13 +-
>>>   scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
>>>   319 files changed, 2729 insertions(+), 4245 deletions(-)
>>>   create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>
>> The diffstat is huge, but promising.

We also learned in reviews of 7/9 that the diffstat here is misleading,
the number of insertions will definitely be increasing once the
Coccinelle script is fixed to insert the macro in more functions, but
hopefully it's still a net reduction in overall lines.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [RFC v2 0/9] error: auto propagated local_err
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
24.09.2019 18:28, Eric Blake wrote:
> On 9/24/19 9:12 AM, Vladimir Sementsov-Ogievskiy wrote:
> 
>>>> 3. What to do with huge auto-generated commit 07? Should I split it
>>>> per-maintainer or per-subsystem, or leave it as-is?
>>>
>>> It's big. I'd split it into multiple patches (and reduce the cc - except
>>> for the cover letter, the rest of the patches can be limited to the
>>> actual maintainer/subsystem affected rather than everyone involved
>>> anywhere else in the series. With the current large cc, anyone that
>>> replies gets several mail bounces about "too many recipients").  It may
>>> be easier to split along directory boundaries than by maintainer
>>> boundaries.  Markus has applied large tree-wide Coccinelle cleanups
>>> before, maybe he has some advice.
>>
>>
>> If split by subsystem it would be 200+ patches:
>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | head -1; done | sort | uniq | wc -l
>> 205
>>
>>
>> Try to look at larger subsystem:
>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | tail -2 | head -1; done | sort | uniq | wc -l
>> 139
>>
>> still too many.. Or is it OK?
> 
> Hmm - that becomes a tradeoff in length of the series (where individual
> patches may be reviewed fast, but where the overall process may be
> bogged down by sheer length), vs. length of individual emails (where the
> email itself is daunting, but as the review is mechanical and done by
> automation, it becomes a matter of spot-checking if we trust that the
> automation was done correctly).  You can probably group it in fewer
> patches, by joining smaller patches across a couple of subsystems.  It's
> an art form, there's probably several ways to do it that would work, and
> it comes down to a judgment call on how much work you want to do to try
> and reduce other's work in reviewing it.  Maybe even an off-hand split
> of gathering files until you reach about 500 or so lines per diff.  I
> wish I had easier advice on how to tackle this sort of project in the
> way that will get the fastest response time.
> 
> 
>>>>    vl.c                                          |  13 +-
>>>>    scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
>>>>    319 files changed, 2729 insertions(+), 4245 deletions(-)
>>>>    create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>
>>> The diffstat is huge, but promising.
> 
> We also learned in reviews of 7/9 that the diffstat here is misleading,
> the number of insertions will definitely be increasing once the
> Coccinelle script is fixed to insert the macro in more functions, but
> hopefully it's still a net reduction in overall lines.
> 

No hope for us: with fixed script I now see

919 files changed, 6425 insertions(+), 4234 deletions(-)

-- 
Best regards,
Vladimir
Re: [RFC v2 0/9] error: auto propagated local_err
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
24.09.2019 18:44, Vladimir Sementsov-Ogievskiy wrote:
> 24.09.2019 18:28, Eric Blake wrote:
>> On 9/24/19 9:12 AM, Vladimir Sementsov-Ogievskiy wrote:
>>
>>>>> 3. What to do with huge auto-generated commit 07? Should I split it
>>>>> per-maintainer or per-subsystem, or leave it as-is?
>>>>
>>>> It's big. I'd split it into multiple patches (and reduce the cc - except
>>>> for the cover letter, the rest of the patches can be limited to the
>>>> actual maintainer/subsystem affected rather than everyone involved
>>>> anywhere else in the series. With the current large cc, anyone that
>>>> replies gets several mail bounces about "too many recipients").  It may
>>>> be easier to split along directory boundaries than by maintainer
>>>> boundaries.  Markus has applied large tree-wide Coccinelle cleanups
>>>> before, maybe he has some advice.
>>>
>>>
>>> If split by subsystem it would be 200+ patches:
>>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | head -1; done | sort | uniq | wc -l
>>> 205
>>>
>>>
>>> Try to look at larger subsystem:
>>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | tail -2 | head -1; done | sort | uniq | wc -l
>>> 139
>>>
>>> still too many.. Or is it OK?
>>
>> Hmm - that becomes a tradeoff in length of the series (where individual
>> patches may be reviewed fast, but where the overall process may be
>> bogged down by sheer length), vs. length of individual emails (where the
>> email itself is daunting, but as the review is mechanical and done by
>> automation, it becomes a matter of spot-checking if we trust that the
>> automation was done correctly).  You can probably group it in fewer
>> patches, by joining smaller patches across a couple of subsystems.  It's
>> an art form, there's probably several ways to do it that would work, and
>> it comes down to a judgment call on how much work you want to do to try
>> and reduce other's work in reviewing it.  Maybe even an off-hand split
>> of gathering files until you reach about 500 or so lines per diff.  I
>> wish I had easier advice on how to tackle this sort of project in the
>> way that will get the fastest response time.
>>
>>
>>>>>    vl.c                                          |  13 +-
>>>>>    scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
>>>>>    319 files changed, 2729 insertions(+), 4245 deletions(-)
>>>>>    create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>
>>>> The diffstat is huge, but promising.
>>
>> We also learned in reviews of 7/9 that the diffstat here is misleading,
>> the number of insertions will definitely be increasing once the
>> Coccinelle script is fixed to insert the macro in more functions, but
>> hopefully it's still a net reduction in overall lines.
>>
> 
> No hope for us: with fixed script I now see
> 
> 919 files changed, 6425 insertions(+), 4234 deletions(-)
> 

Also, I have add include "qapi/error.h" to files, where errp only passed
to called functions (or for functions, which are not simple stubs):

# git diff | grep '+#include' | wc -l
253


-- 
Best regards,
Vladimir
Re: [RFC v2 0/9] error: auto propagated local_err
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
24.09.2019 18:46, Vladimir Sementsov-Ogievskiy wrote:
> 24.09.2019 18:44, Vladimir Sementsov-Ogievskiy wrote:
>> 24.09.2019 18:28, Eric Blake wrote:
>>> On 9/24/19 9:12 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>
>>>>>> 3. What to do with huge auto-generated commit 07? Should I split it
>>>>>> per-maintainer or per-subsystem, or leave it as-is?
>>>>>
>>>>> It's big. I'd split it into multiple patches (and reduce the cc - except
>>>>> for the cover letter, the rest of the patches can be limited to the
>>>>> actual maintainer/subsystem affected rather than everyone involved
>>>>> anywhere else in the series. With the current large cc, anyone that
>>>>> replies gets several mail bounces about "too many recipients").  It may
>>>>> be easier to split along directory boundaries than by maintainer
>>>>> boundaries.  Markus has applied large tree-wide Coccinelle cleanups
>>>>> before, maybe he has some advice.
>>>>
>>>>
>>>> If split by subsystem it would be 200+ patches:
>>>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | head -1; done | sort | uniq | wc -l
>>>> 205
>>>>
>>>>
>>>> Try to look at larger subsystem:
>>>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | tail -2 | head -1; done | sort | uniq | wc -l
>>>> 139
>>>>
>>>> still too many.. Or is it OK?
>>>
>>> Hmm - that becomes a tradeoff in length of the series (where individual
>>> patches may be reviewed fast, but where the overall process may be
>>> bogged down by sheer length), vs. length of individual emails (where the
>>> email itself is daunting, but as the review is mechanical and done by
>>> automation, it becomes a matter of spot-checking if we trust that the
>>> automation was done correctly).  You can probably group it in fewer
>>> patches, by joining smaller patches across a couple of subsystems.  It's
>>> an art form, there's probably several ways to do it that would work, and
>>> it comes down to a judgment call on how much work you want to do to try
>>> and reduce other's work in reviewing it.  Maybe even an off-hand split
>>> of gathering files until you reach about 500 or so lines per diff.  I
>>> wish I had easier advice on how to tackle this sort of project in the
>>> way that will get the fastest response time.

git diff | wc -l
48941

so, by 500 lines it would be 97 patches.

Seems, we'll never be able to review this thing :(

Any ideas?

May be, separate big patch, which just add ERRP_FUNCTION_BEGIN() to all errp functions?

>>>
>>>
>>>>>>    vl.c                                          |  13 +-
>>>>>>    scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
>>>>>>    319 files changed, 2729 insertions(+), 4245 deletions(-)
>>>>>>    create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>
>>>>> The diffstat is huge, but promising.
>>>
>>> We also learned in reviews of 7/9 that the diffstat here is misleading,
>>> the number of insertions will definitely be increasing once the
>>> Coccinelle script is fixed to insert the macro in more functions, but
>>> hopefully it's still a net reduction in overall lines.
>>>
>>
>> No hope for us: with fixed script I now see
>>
>> 919 files changed, 6425 insertions(+), 4234 deletions(-)
>>
> 
> Also, I have add include "qapi/error.h" to files, where errp only passed
> to called functions (or for functions, which are not simple stubs):
> 
> # git diff | grep '+#include' | wc -l
> 253
> 
> 


-- 
Best regards,
Vladimir
Re: [RFC v2 0/9] error: auto propagated local_err
Posted by Vladimir Sementsov-Ogievskiy 4 years, 7 months ago
24.09.2019 18:49, Vladimir Sementsov-Ogievskiy wrote:
> 24.09.2019 18:46, Vladimir Sementsov-Ogievskiy wrote:
>> 24.09.2019 18:44, Vladimir Sementsov-Ogievskiy wrote:
>>> 24.09.2019 18:28, Eric Blake wrote:
>>>> On 9/24/19 9:12 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>>
>>>>>>> 3. What to do with huge auto-generated commit 07? Should I split it
>>>>>>> per-maintainer or per-subsystem, or leave it as-is?
>>>>>>
>>>>>> It's big. I'd split it into multiple patches (and reduce the cc - except
>>>>>> for the cover letter, the rest of the patches can be limited to the
>>>>>> actual maintainer/subsystem affected rather than everyone involved
>>>>>> anywhere else in the series. With the current large cc, anyone that
>>>>>> replies gets several mail bounces about "too many recipients").  It may
>>>>>> be easier to split along directory boundaries than by maintainer
>>>>>> boundaries.  Markus has applied large tree-wide Coccinelle cleanups
>>>>>> before, maybe he has some advice.
>>>>>
>>>>>
>>>>> If split by subsystem it would be 200+ patches:
>>>>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | head -1; done | sort | uniq | wc -l
>>>>> 205
>>>>>
>>>>>
>>>>> Try to look at larger subsystem:
>>>>> git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | tail -2 | head -1; done | sort | uniq | wc -l
>>>>> 139
>>>>>
>>>>> still too many.. Or is it OK?
>>>>
>>>> Hmm - that becomes a tradeoff in length of the series (where individual
>>>> patches may be reviewed fast, but where the overall process may be
>>>> bogged down by sheer length), vs. length of individual emails (where the
>>>> email itself is daunting, but as the review is mechanical and done by
>>>> automation, it becomes a matter of spot-checking if we trust that the
>>>> automation was done correctly).  You can probably group it in fewer
>>>> patches, by joining smaller patches across a couple of subsystems.  It's
>>>> an art form, there's probably several ways to do it that would work, and
>>>> it comes down to a judgment call on how much work you want to do to try
>>>> and reduce other's work in reviewing it.  Maybe even an off-hand split
>>>> of gathering files until you reach about 500 or so lines per diff.  I
>>>> wish I had easier advice on how to tackle this sort of project in the
>>>> way that will get the fastest response time.
> 
> git diff | wc -l
> 48941
> 
> so, by 500 lines it would be 97 patches.
> 
> Seems, we'll never be able to review this thing :(
> 
> Any ideas?
> 
> May be, separate big patch, which just add ERRP_FUNCTION_BEGIN() to all errp functions?

checked: for remaining improvements:
git diff | wc -l
20218

git diff --name-only | while read f; do scripts/get_maintainer.pl -f $f --subsystem --no-rolestats 2>/dev/null | grep -v @ | tail -2 | head -1; done | sort | uniq | wc -l
90

Still, too much..

I think we should switch to plan B, at least as something that may be merged to 4.2


> 
>>>>
>>>>
>>>>>>>    vl.c                                          |  13 +-
>>>>>>>    scripts/coccinelle/auto-propagated-errp.cocci |  82 +++++++
>>>>>>>    319 files changed, 2729 insertions(+), 4245 deletions(-)
>>>>>>>    create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>>
>>>>>> The diffstat is huge, but promising.
>>>>
>>>> We also learned in reviews of 7/9 that the diffstat here is misleading,
>>>> the number of insertions will definitely be increasing once the
>>>> Coccinelle script is fixed to insert the macro in more functions, but
>>>> hopefully it's still a net reduction in overall lines.
>>>>
>>>
>>> No hope for us: with fixed script I now see
>>>
>>> 919 files changed, 6425 insertions(+), 4234 deletions(-)
>>>
>>
>> Also, I have add include "qapi/error.h" to files, where errp only passed
>> to called functions (or for functions, which are not simple stubs):
>>
>> # git diff | grep '+#include' | wc -l
>> 253
>>
>>
> 
> 


-- 
Best regards,
Vladimir