[PATCH v6 00/21] Add virtual device fuzzing support

Oleinik, Alexander posted 21 patches 32 weeks ago
Failed in applying to current master (apply log)
Test asan passed
Test checkpatch passed
Test FreeBSD passed
Test docker-mingw@fedora passed
Test docker-clang@ubuntu passed
Test docker-quick@centos7 passed
Makefile                      |  16 ++-
Makefile.objs                 |   4 +
Makefile.target               |  18 ++-
configure                     |  39 ++++++
docs/devel/fuzzing.txt        | 119 +++++++++++++++++
exec.c                        |  12 +-
include/qemu/module.h         |   4 +-
include/sysemu/qtest.h        |   4 +
include/sysemu/sysemu.h       |   4 +
main.c                        |  53 ++++++++
qtest.c                       |  31 ++++-
tests/Makefile.include        |  75 +++++------
tests/fuzz/Makefile.include   |  12 ++
tests/fuzz/fork_fuzz.c        |  55 ++++++++
tests/fuzz/fork_fuzz.h        |  23 ++++
tests/fuzz/fork_fuzz.ld       |  37 ++++++
tests/fuzz/fuzz.c             | 179 ++++++++++++++++++++++++++
tests/fuzz/fuzz.h             |  94 ++++++++++++++
tests/fuzz/i440fx_fuzz.c      | 176 ++++++++++++++++++++++++++
tests/fuzz/qos_fuzz.c         | 232 ++++++++++++++++++++++++++++++++++
tests/fuzz/qos_fuzz.h         |  33 +++++
tests/fuzz/virtio_net_fuzz.c  | 105 +++++++++++++++
tests/fuzz/virtio_scsi_fuzz.c | 200 +++++++++++++++++++++++++++++
tests/libqos/i2c.c            |  10 +-
tests/libqos/i2c.h            |   4 +-
tests/libqos/qos_external.c   | 168 ++++++++++++++++++++++++
tests/libqos/qos_external.h   |  28 ++++
tests/libqtest.c              | 108 ++++++++++++++--
tests/libqtest.h              |   4 +
tests/pca9552-test.c          |  10 +-
tests/qos-test.c              | 140 +-------------------
util/module.c                 |   7 +
vl.c                          |  38 ++----
33 files changed, 1813 insertions(+), 229 deletions(-)
create mode 100644 docs/devel/fuzzing.txt
create mode 100644 main.c
create mode 100644 tests/fuzz/Makefile.include
create mode 100644 tests/fuzz/fork_fuzz.c
create mode 100644 tests/fuzz/fork_fuzz.h
create mode 100644 tests/fuzz/fork_fuzz.ld
create mode 100644 tests/fuzz/fuzz.c
create mode 100644 tests/fuzz/fuzz.h
create mode 100644 tests/fuzz/i440fx_fuzz.c
create mode 100644 tests/fuzz/qos_fuzz.c
create mode 100644 tests/fuzz/qos_fuzz.h
create mode 100644 tests/fuzz/virtio_net_fuzz.c
create mode 100644 tests/fuzz/virtio_scsi_fuzz.c
create mode 100644 tests/libqos/qos_external.c
create mode 100644 tests/libqos/qos_external.h

[PATCH v6 00/21] Add virtual device fuzzing support

Posted by Oleinik, Alexander 32 weeks ago
This series adds a framework for coverage-guided fuzzing of
virtual-devices. Fuzzing targets are based on qtest and can make use of
the libqos abstractions.

In this version, I added a virtio-scsi fuzzer. The actual fuzzing code
is very similar to the the virtio-net fuzzer. I experimented with using
a single fuzzer with device-specific initialization for each virtio
device, but it did not come out as cleanly as I hoped, since I could not
find an easy way to override the qos drivers for devices that have
more-complete qos support (such as virtio-net), so these changes have
not made it into v6.

V5/V6: (V5 did not have review tags)
 * added virtio-scsi fuzzer
 * add support for using fork-based fuzzers with multiple libfuzzer
   workers
 * misc fixes addressing V4 comments
 * cleanup in-process handlers/globals in libqtest.c
 * small fixes to fork-based fuzzing and support for multiple workers
 * changes to the virtio-net fuzzer to kick after each vq add

V4:
 * add/transfer license headers to new files
 * restructure the added QTestClientTransportOps struct
 * restructure the FuzzTarget struct and fuzzer skeleton
 * fork-based fuzzer now directly mmaps shm over the coverage bitmaps
 * fixes to i440 and virtio-net fuzz targets
 * undo the changes to qtest_memwrite
 * possible to build /fuzz and /all in the same build-dir
 * misc fixes to address V3 comments

V3:
 * rebased onto v4.1.0+
 * add the fuzzer as a new build-target type in the build-system
 * add indirection to qtest client/server communication functions
 * remove ramfile and snapshot-based fuzzing support
 * add i440fx fuzz-target as a reference for developers.
 * add linker-script to assist with fork-based fuzzer

V2:
 * split off changes to qos virtio-net and qtest server to other patches
 * move vl:main initialization into new func: qemu_init
 * moved useful functions from qos-test.c to a separate object
 * use struct of function pointers for add_fuzz_target(), instead of
   arguments
 * move ramfile to migration/qemu-file
 * rewrite fork-based fuzzer pending patch to libfuzzer
 * pass check-patch

Alexander Bulekov (21):
  softmmu: split off vl.c:main() into main.c
  libqos: Rename i2c_send and i2c_recv
  fuzz: Add FUZZ_TARGET module type
  qtest: add qtest_server_send abstraction
  libqtest: Add a layer of abstraciton to send/recv
  module: check module wasn't already initialized
  qtest: add in-process incoming command handler
  tests: provide test variables to other targets
  libqos: split qos-test and libqos makefile vars
  libqos: move useful qos-test funcs to qos_external
  libqtest: make bufwrite rely on the TransportOps
  libqtest: add in-process qtest.c tx/rx handlers
  fuzz: add configure flag --enable-fuzzing
  fuzz: Add target/fuzz makefile rules
  fuzz: add fuzzer skeleton
  fuzz: add support for fork-based fuzzing.
  fuzz: add support for qos-assisted fuzz targets
  fuzz: add i440fx fuzz targets
  fuzz: add virtio-net fuzz target
  fuzz: add virtio-scsi fuzz target
  fuzz: add documentation to docs/devel/

 Makefile                      |  16 ++-
 Makefile.objs                 |   4 +
 Makefile.target               |  18 ++-
 configure                     |  39 ++++++
 docs/devel/fuzzing.txt        | 119 +++++++++++++++++
 exec.c                        |  12 +-
 include/qemu/module.h         |   4 +-
 include/sysemu/qtest.h        |   4 +
 include/sysemu/sysemu.h       |   4 +
 main.c                        |  53 ++++++++
 qtest.c                       |  31 ++++-
 tests/Makefile.include        |  75 +++++------
 tests/fuzz/Makefile.include   |  12 ++
 tests/fuzz/fork_fuzz.c        |  55 ++++++++
 tests/fuzz/fork_fuzz.h        |  23 ++++
 tests/fuzz/fork_fuzz.ld       |  37 ++++++
 tests/fuzz/fuzz.c             | 179 ++++++++++++++++++++++++++
 tests/fuzz/fuzz.h             |  94 ++++++++++++++
 tests/fuzz/i440fx_fuzz.c      | 176 ++++++++++++++++++++++++++
 tests/fuzz/qos_fuzz.c         | 232 ++++++++++++++++++++++++++++++++++
 tests/fuzz/qos_fuzz.h         |  33 +++++
 tests/fuzz/virtio_net_fuzz.c  | 105 +++++++++++++++
 tests/fuzz/virtio_scsi_fuzz.c | 200 +++++++++++++++++++++++++++++
 tests/libqos/i2c.c            |  10 +-
 tests/libqos/i2c.h            |   4 +-
 tests/libqos/qos_external.c   | 168 ++++++++++++++++++++++++
 tests/libqos/qos_external.h   |  28 ++++
 tests/libqtest.c              | 108 ++++++++++++++--
 tests/libqtest.h              |   4 +
 tests/pca9552-test.c          |  10 +-
 tests/qos-test.c              | 140 +-------------------
 util/module.c                 |   7 +
 vl.c                          |  38 ++----
 33 files changed, 1813 insertions(+), 229 deletions(-)
 create mode 100644 docs/devel/fuzzing.txt
 create mode 100644 main.c
 create mode 100644 tests/fuzz/Makefile.include
 create mode 100644 tests/fuzz/fork_fuzz.c
 create mode 100644 tests/fuzz/fork_fuzz.h
 create mode 100644 tests/fuzz/fork_fuzz.ld
 create mode 100644 tests/fuzz/fuzz.c
 create mode 100644 tests/fuzz/fuzz.h
 create mode 100644 tests/fuzz/i440fx_fuzz.c
 create mode 100644 tests/fuzz/qos_fuzz.c
 create mode 100644 tests/fuzz/qos_fuzz.h
 create mode 100644 tests/fuzz/virtio_net_fuzz.c
 create mode 100644 tests/fuzz/virtio_scsi_fuzz.c
 create mode 100644 tests/libqos/qos_external.c
 create mode 100644 tests/libqos/qos_external.h

-- 
2.23.0


Re: [PATCH v6 00/21] Add virtual device fuzzing support

Posted by Stefan Hajnoczi 27 weeks ago
On Fri, Nov 29, 2019 at 09:34:35PM +0000, Oleinik, Alexander wrote:
> This series adds a framework for coverage-guided fuzzing of
> virtual-devices. Fuzzing targets are based on qtest and can make use of
> the libqos abstractions.
> 
> In this version, I added a virtio-scsi fuzzer. The actual fuzzing code
> is very similar to the the virtio-net fuzzer. I experimented with using
> a single fuzzer with device-specific initialization for each virtio
> device, but it did not come out as cleanly as I hoped, since I could not
> find an easy way to override the qos drivers for devices that have
> more-complete qos support (such as virtio-net), so these changes have
> not made it into v6.
> 
> V5/V6: (V5 did not have review tags)
>  * added virtio-scsi fuzzer
>  * add support for using fork-based fuzzers with multiple libfuzzer
>    workers
>  * misc fixes addressing V4 comments
>  * cleanup in-process handlers/globals in libqtest.c
>  * small fixes to fork-based fuzzing and support for multiple workers
>  * changes to the virtio-net fuzzer to kick after each vq add
> 
> V4:
>  * add/transfer license headers to new files
>  * restructure the added QTestClientTransportOps struct
>  * restructure the FuzzTarget struct and fuzzer skeleton
>  * fork-based fuzzer now directly mmaps shm over the coverage bitmaps
>  * fixes to i440 and virtio-net fuzz targets
>  * undo the changes to qtest_memwrite
>  * possible to build /fuzz and /all in the same build-dir
>  * misc fixes to address V3 comments
> 
> V3:
>  * rebased onto v4.1.0+
>  * add the fuzzer as a new build-target type in the build-system
>  * add indirection to qtest client/server communication functions
>  * remove ramfile and snapshot-based fuzzing support
>  * add i440fx fuzz-target as a reference for developers.
>  * add linker-script to assist with fork-based fuzzer
> 
> V2:
>  * split off changes to qos virtio-net and qtest server to other patches
>  * move vl:main initialization into new func: qemu_init
>  * moved useful functions from qos-test.c to a separate object
>  * use struct of function pointers for add_fuzz_target(), instead of
>    arguments
>  * move ramfile to migration/qemu-file
>  * rewrite fork-based fuzzer pending patch to libfuzzer
>  * pass check-patch
> 
> Alexander Bulekov (21):
>   softmmu: split off vl.c:main() into main.c
>   libqos: Rename i2c_send and i2c_recv
>   fuzz: Add FUZZ_TARGET module type
>   qtest: add qtest_server_send abstraction
>   libqtest: Add a layer of abstraciton to send/recv
>   module: check module wasn't already initialized
>   qtest: add in-process incoming command handler
>   tests: provide test variables to other targets
>   libqos: split qos-test and libqos makefile vars
>   libqos: move useful qos-test funcs to qos_external
>   libqtest: make bufwrite rely on the TransportOps
>   libqtest: add in-process qtest.c tx/rx handlers
>   fuzz: add configure flag --enable-fuzzing
>   fuzz: Add target/fuzz makefile rules
>   fuzz: add fuzzer skeleton
>   fuzz: add support for fork-based fuzzing.
>   fuzz: add support for qos-assisted fuzz targets
>   fuzz: add i440fx fuzz targets
>   fuzz: add virtio-net fuzz target
>   fuzz: add virtio-scsi fuzz target
>   fuzz: add documentation to docs/devel/
> 
>  Makefile                      |  16 ++-
>  Makefile.objs                 |   4 +
>  Makefile.target               |  18 ++-
>  configure                     |  39 ++++++
>  docs/devel/fuzzing.txt        | 119 +++++++++++++++++
>  exec.c                        |  12 +-
>  include/qemu/module.h         |   4 +-
>  include/sysemu/qtest.h        |   4 +
>  include/sysemu/sysemu.h       |   4 +
>  main.c                        |  53 ++++++++
>  qtest.c                       |  31 ++++-
>  tests/Makefile.include        |  75 +++++------
>  tests/fuzz/Makefile.include   |  12 ++
>  tests/fuzz/fork_fuzz.c        |  55 ++++++++
>  tests/fuzz/fork_fuzz.h        |  23 ++++
>  tests/fuzz/fork_fuzz.ld       |  37 ++++++
>  tests/fuzz/fuzz.c             | 179 ++++++++++++++++++++++++++
>  tests/fuzz/fuzz.h             |  94 ++++++++++++++
>  tests/fuzz/i440fx_fuzz.c      | 176 ++++++++++++++++++++++++++
>  tests/fuzz/qos_fuzz.c         | 232 ++++++++++++++++++++++++++++++++++
>  tests/fuzz/qos_fuzz.h         |  33 +++++
>  tests/fuzz/virtio_net_fuzz.c  | 105 +++++++++++++++
>  tests/fuzz/virtio_scsi_fuzz.c | 200 +++++++++++++++++++++++++++++
>  tests/libqos/i2c.c            |  10 +-
>  tests/libqos/i2c.h            |   4 +-
>  tests/libqos/qos_external.c   | 168 ++++++++++++++++++++++++
>  tests/libqos/qos_external.h   |  28 ++++
>  tests/libqtest.c              | 108 ++++++++++++++--
>  tests/libqtest.h              |   4 +
>  tests/pca9552-test.c          |  10 +-
>  tests/qos-test.c              | 140 +-------------------
>  util/module.c                 |   7 +
>  vl.c                          |  38 ++----
>  33 files changed, 1813 insertions(+), 229 deletions(-)
>  create mode 100644 docs/devel/fuzzing.txt
>  create mode 100644 main.c
>  create mode 100644 tests/fuzz/Makefile.include
>  create mode 100644 tests/fuzz/fork_fuzz.c
>  create mode 100644 tests/fuzz/fork_fuzz.h
>  create mode 100644 tests/fuzz/fork_fuzz.ld
>  create mode 100644 tests/fuzz/fuzz.c
>  create mode 100644 tests/fuzz/fuzz.h
>  create mode 100644 tests/fuzz/i440fx_fuzz.c
>  create mode 100644 tests/fuzz/qos_fuzz.c
>  create mode 100644 tests/fuzz/qos_fuzz.h
>  create mode 100644 tests/fuzz/virtio_net_fuzz.c
>  create mode 100644 tests/fuzz/virtio_scsi_fuzz.c
>  create mode 100644 tests/libqos/qos_external.c
>  create mode 100644 tests/libqos/qos_external.h

Please use "git rebase -i origin/master" with "x make" after each commit
to verify that building succeeds.  This is important for git-bisect(1)
where we need the tree to always build successfully and for
cherry-picking patches without introducing breakage.

It's not enough for the tree to build at the end of the patch series.
It should build at each step along the way.

Re: [PATCH v6 00/21] Add virtual device fuzzing support

Posted by Stefan Hajnoczi 27 weeks ago
On Fri, Nov 29, 2019 at 09:34:35PM +0000, Oleinik, Alexander wrote:
> This series adds a framework for coverage-guided fuzzing of
> virtual-devices. Fuzzing targets are based on qtest and can make use of
> the libqos abstractions.
> 
> In this version, I added a virtio-scsi fuzzer. The actual fuzzing code
> is very similar to the the virtio-net fuzzer. I experimented with using
> a single fuzzer with device-specific initialization for each virtio
> device, but it did not come out as cleanly as I hoped, since I could not
> find an easy way to override the qos drivers for devices that have
> more-complete qos support (such as virtio-net), so these changes have
> not made it into v6.
> 
> V5/V6: (V5 did not have review tags)
>  * added virtio-scsi fuzzer
>  * add support for using fork-based fuzzers with multiple libfuzzer
>    workers
>  * misc fixes addressing V4 comments
>  * cleanup in-process handlers/globals in libqtest.c
>  * small fixes to fork-based fuzzing and support for multiple workers
>  * changes to the virtio-net fuzzer to kick after each vq add

Please run scripts/checkpatch.pl on all patches in case you haven't
already.

I have finished reviewing this revision and posted my Reviewed-by on
most patches.  Unless other feedback is raised the next revision could
be merged.