[PATCH v4 00/16] Add a General Virtual Device Fuzzer

Alexander Bulekov posted 16 patches 3 years, 6 months ago
Test checkpatch failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20201015134137.205958-1-alxndr@bu.edu
Maintainers: Stefan Hajnoczi <stefanha@redhat.com>, Laurent Vivier <lvivier@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Bandan Das <bsd@redhat.com>, Alexander Bulekov <alxndr@bu.edu>, Thomas Huth <thuth@redhat.com>
docs/devel/fuzzing.txt                        |  39 +
include/exec/memory.h                         |  21 +
include/exec/memory_ldst_cached.h.inc         |   3 +
memory_ldst.c.inc                             |   4 +
scripts/oss-fuzz/build.sh                     |   6 +
scripts/oss-fuzz/minimize_qtest_trace.py      | 157 +++
.../oss-fuzz/reorder_fuzzer_qtest_trace.py    | 103 ++
softmmu/memory.c                              |  23 +
softmmu/physmem.c                             |   2 +
tests/qtest/fuzz/fuzz.c                       |  13 +
tests/qtest/fuzz/fuzz.h                       |  28 +
tests/qtest/fuzz/general_fuzz.c               | 948 ++++++++++++++++++
tests/qtest/fuzz/general_fuzz_configs.c       | 140 +++
tests/qtest/fuzz/general_fuzz_configs.h       |  24 +
tests/qtest/fuzz/meson.build                  |   1 +
15 files changed, 1512 insertions(+)
create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py
create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py
create mode 100644 tests/qtest/fuzz/general_fuzz.c
create mode 100644 tests/qtest/fuzz/general_fuzz_configs.c
create mode 100644 tests/qtest/fuzz/general_fuzz_configs.h
[PATCH v4 00/16] Add a General Virtual Device Fuzzer
Posted by Alexander Bulekov 3 years, 6 months ago
v4:
    - Replace yaml + c template-based oss-fuzz configs, with C code to
      register a FuzzTarget for each config (as suggested by Paolo)
    - Replicate the functionality of address_space_write_rom to ensure
      matching behavior when QTEST_LOG is enabled
    - Improve code documentation/comments
    - Small formatting changes
v3:
	- Use flatviews to help select regions for fuzzing 
	- Meson-related changes
    - Add some documentation
	- Improve minimalization script to trim write{bwlq} commands
v2:
	- Remove QOS dependency.
	- Add a custom crossover function
	- Fix broken minimization scripts
	- Fixes to the IO region and DMA handling code

This is a general virtual-device fuzzer, designed to fuzz devices over Port IO,
MMIO, and DMA.

To get started with this:
 1. Build the fuzzers (see docs/devel/fuzzing.txt)
    Note: Build with --enable-sanitizers, or create a "dictionary file":
    echo kw1=\"FUZZ\" > dict
    and pass it as an argument to libFuzzer with -dict=./dict
    This magic value is a command separator that lets the fuzzer perform
    multiple IO actions with a single input.

 2. Pick the qemu arguments you wish to fuzz:
    export QEMU_FUZZ_ARGS="-M q35 -device virtio-balloon"

 3. Tell the fuzzer which QOM objects or MemoryRegion names to fuzz. I find the
 "info qom-tree", "info qtree" and "info mtree" commands useful for identifying
 these. Supports globbing. Here I will try to simultaneously fuzz(for no good
 reason) virtio-balloon and e1000e, which is included by default in the q35:
    export QEMU_FUZZ_OBJECTS='virtio* e1000*'
    You can also try to fuzz the whole machine:
    export QEMU_FUZZ_OBJECTS='*'

 4. Run the fuzzer for 0 inputs. The fuzzer should output a list of
 MemoryRegions/PCI Devices it will try to fuzz. Confirm that these match your
 expectations.
    ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz -runs=0

 5. Run the fuzzer:
    ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz 


Basically, at the core, this fuzzer is an interpreter that splits the input
into a series of commands, such as mmio_write, pio_write, etc. We structure
these commands to hit only MemoryRegions that are associated with the devices
specified in QEMU_FUZZ_OBJECTS. Additionally, these patches add "hooks" to
functions that are typically used by virtual-devices to read from RAM (DMA).
These hooks attempt to populate these DMA regions with fuzzed data, just in
time.

Some of the issues I have found or reproduced with this fuzzer:
https://bugs.launchpad.net/bugs/1525123
https://bugs.launchpad.net/bugs/1681439
https://bugs.launchpad.net/bugs/1777315
https://bugs.launchpad.net/bugs/1878034
https://bugs.launchpad.net/bugs/1878043
https://bugs.launchpad.net/bugs/1878054
https://bugs.launchpad.net/bugs/1878057
https://bugs.launchpad.net/bugs/1878067
https://bugs.launchpad.net/bugs/1878134
https://bugs.launchpad.net/bugs/1878136
https://bugs.launchpad.net/bugs/1878253
https://bugs.launchpad.net/bugs/1878255
https://bugs.launchpad.net/bugs/1878259
https://bugs.launchpad.net/bugs/1878263
https://bugs.launchpad.net/bugs/1878323
https://bugs.launchpad.net/bugs/1878641
https://bugs.launchpad.net/bugs/1878642
https://bugs.launchpad.net/bugs/1878645
https://bugs.launchpad.net/bugs/1878651
https://bugs.launchpad.net/bugs/1879223
https://bugs.launchpad.net/bugs/1879227
https://bugs.launchpad.net/bugs/1879531
https://bugs.launchpad.net/bugs/1880355
https://bugs.launchpad.net/bugs/1880539
https://bugs.launchpad.net/bugs/1884693
https://bugs.launchpad.net/bugs/1886362
https://bugs.launchpad.net/bugs/1887303
https://bugs.launchpad.net/bugs/1887309
https://bugs.launchpad.net/bugs/697510

Alexander Bulekov (16):
  memory: Add FlatView foreach function
  fuzz: Add general virtual-device fuzzer
  fuzz: Add PCI features to the general fuzzer
  fuzz: Add DMA support to the generic-fuzzer
  fuzz: Declare DMA Read callback function
  fuzz: Add fuzzer callbacks to DMA-read functions
  fuzz: Add support for custom crossover functions
  fuzz: add a DISABLE_PCI op to general-fuzzer
  fuzz: add a crossover function to generic-fuzzer
  scripts/oss-fuzz: Add script to reorder a general-fuzzer trace
  scripts/oss-fuzz: Add crash trace minimization script
  fuzz: Add instructions for using general-fuzz
  fuzz: add an "opaque" to the FuzzTarget struct
  fuzz: add general-fuzz configs for oss-fuzz
  fuzz: register predefined general-fuzz configs
  scripts/oss-fuzz: remove the general-fuzz target

 docs/devel/fuzzing.txt                        |  39 +
 include/exec/memory.h                         |  21 +
 include/exec/memory_ldst_cached.h.inc         |   3 +
 memory_ldst.c.inc                             |   4 +
 scripts/oss-fuzz/build.sh                     |   6 +
 scripts/oss-fuzz/minimize_qtest_trace.py      | 157 +++
 .../oss-fuzz/reorder_fuzzer_qtest_trace.py    | 103 ++
 softmmu/memory.c                              |  23 +
 softmmu/physmem.c                             |   2 +
 tests/qtest/fuzz/fuzz.c                       |  13 +
 tests/qtest/fuzz/fuzz.h                       |  28 +
 tests/qtest/fuzz/general_fuzz.c               | 948 ++++++++++++++++++
 tests/qtest/fuzz/general_fuzz_configs.c       | 140 +++
 tests/qtest/fuzz/general_fuzz_configs.h       |  24 +
 tests/qtest/fuzz/meson.build                  |   1 +
 15 files changed, 1512 insertions(+)
 create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py
 create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py
 create mode 100644 tests/qtest/fuzz/general_fuzz.c
 create mode 100644 tests/qtest/fuzz/general_fuzz_configs.c
 create mode 100644 tests/qtest/fuzz/general_fuzz_configs.h

-- 
2.28.0


Re: [PATCH v4 00/16] Add a General Virtual Device Fuzzer
Posted by no-reply@patchew.org 3 years, 6 months ago
Patchew URL: https://patchew.org/QEMU/20201015134137.205958-1-alxndr@bu.edu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20201015134137.205958-1-alxndr@bu.edu
Subject: [PATCH v4 00/16] Add a General Virtual Device Fuzzer

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]      patchew/1602767767-32713-1-git-send-email-mjrosato@linux.ibm.com -> patchew/1602767767-32713-1-git-send-email-mjrosato@linux.ibm.com
 * [new tag]         patchew/20201015134137.205958-1-alxndr@bu.edu -> patchew/20201015134137.205958-1-alxndr@bu.edu
Switched to a new branch 'test'
03323c2 scripts/oss-fuzz: remove the general-fuzz target
3c59bd5 fuzz: register predefined general-fuzz configs
0d80db3 fuzz: add general-fuzz configs for oss-fuzz
cea76d4 fuzz: add an "opaque" to the FuzzTarget struct
10cf373 fuzz: Add instructions for using general-fuzz
540fed2 scripts/oss-fuzz: Add crash trace minimization script
0550db0 scripts/oss-fuzz: Add script to reorder a general-fuzzer trace
106e830 fuzz: add a crossover function to generic-fuzzer
90837f5 fuzz: add a DISABLE_PCI op to general-fuzzer
adf044f fuzz: Add support for custom crossover functions
0851cb4 fuzz: Add fuzzer callbacks to DMA-read functions
4acf44b fuzz: Declare DMA Read callback function
86fec34 fuzz: Add DMA support to the generic-fuzzer
237e9d6 fuzz: Add PCI features to the general fuzzer
41e6d7b fuzz: Add general virtual-device fuzzer
ba68f35 memory: Add FlatView foreach function

=== OUTPUT BEGIN ===
1/16 Checking commit ba68f35ed358 (memory: Add FlatView foreach function)
2/16 Checking commit 41e6d7bb40d3 (fuzz: Add general virtual-device fuzzer)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#19: 
new file mode 100644

WARNING: line over 80 characters
#322: FILE: tests/qtest/fuzz/general_fuzz.c:299:
+ * Some commands can be variable-width, so we use a separator, SEPARATOR, to specify

WARNING: line over 80 characters
#330: FILE: tests/qtest/fuzz/general_fuzz.c:307:
+ *      Simply, by removing the first byte, we end up with a very different sequence:

total: 0 errors, 3 warnings, 516 lines checked

Patch 2/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/16 Checking commit 237e9d6e734d (fuzz: Add PCI features to the general fuzzer)
4/16 Checking commit 86fec343b37a (fuzz: Add DMA support to the generic-fuzzer)
ERROR: externs should be avoided in .c files
#84: FILE: tests/qtest/fuzz/general_fuzz.c:120:
+void fuzz_dma_read_cb(size_t addr, size_t len, MemoryRegion *mr, bool is_write);

WARNING: Block comments use a leading /* on a separate line
#109: FILE: tests/qtest/fuzz/general_fuzz.c:145:
+    /* Regions are assumed to support 1-4 byte accesses unless

WARNING: Block comments use * on subsequent lines
#110: FILE: tests/qtest/fuzz/general_fuzz.c:146:
+    /* Regions are assumed to support 1-4 byte accesses unless
+       otherwise specified.  */

WARNING: Block comments use a trailing */ on a separate line
#110: FILE: tests/qtest/fuzz/general_fuzz.c:146:
+       otherwise specified.  */

total: 1 errors, 3 warnings, 299 lines checked

Patch 4/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

5/16 Checking commit 4acf44ba5630 (fuzz: Declare DMA Read callback function)
6/16 Checking commit 0851cb4b787d (fuzz: Add fuzzer callbacks to DMA-read functions)
7/16 Checking commit adf044f2b9d2 (fuzz: Add support for custom crossover functions)
8/16 Checking commit 90837f5a8315 (fuzz: add a DISABLE_PCI op to general-fuzzer)
9/16 Checking commit 106e83015d43 (fuzz: add a crossover function to generic-fuzzer)
10/16 Checking commit 0550db09152f (scripts/oss-fuzz: Add script to reorder a general-fuzzer trace)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#22: 
new file mode 100755

WARNING: line over 80 characters
#117: FILE: scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py:91:
+                    sys.stderr.write("Warning: Likely double fetch on line {}.\n"

total: 0 errors, 2 warnings, 103 lines checked

Patch 10/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/16 Checking commit 540fed26dc40 (scripts/oss-fuzz: Add crash trace minimization script)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#17: 
new file mode 100755

total: 0 errors, 1 warnings, 157 lines checked

Patch 11/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/16 Checking commit 10cf3731768d (fuzz: Add instructions for using general-fuzz)
13/16 Checking commit cea76d423400 (fuzz: add an "opaque" to the FuzzTarget struct)
14/16 Checking commit 0d80db351b77 (fuzz: add general-fuzz configs for oss-fuzz)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#15: 
new file mode 100644

total: 0 errors, 1 warnings, 172 lines checked

Patch 14/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
15/16 Checking commit 3c59bd5b67df (fuzz: register predefined general-fuzz configs)
16/16 Checking commit 03323c23616b (scripts/oss-fuzz: remove the general-fuzz target)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20201015134137.205958-1-alxndr@bu.edu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v4 00/16] Add a General Virtual Device Fuzzer
Posted by Paolo Bonzini 3 years, 6 months ago
If you have to do a v5, the correct word is generic not general. :)

Otherwise looks great.

Paolo

On 15/10/20 15:41, Alexander Bulekov wrote:
> v4:
>     - Replace yaml + c template-based oss-fuzz configs, with C code to
>       register a FuzzTarget for each config (as suggested by Paolo)
>     - Replicate the functionality of address_space_write_rom to ensure
>       matching behavior when QTEST_LOG is enabled
>     - Improve code documentation/comments
>     - Small formatting changes
> v3:
> 	- Use flatviews to help select regions for fuzzing 
> 	- Meson-related changes
>     - Add some documentation
> 	- Improve minimalization script to trim write{bwlq} commands
> v2:
> 	- Remove QOS dependency.
> 	- Add a custom crossover function
> 	- Fix broken minimization scripts
> 	- Fixes to the IO region and DMA handling code
> 
> This is a general virtual-device fuzzer, designed to fuzz devices over Port IO,
> MMIO, and DMA.
> 
> To get started with this:
>  1. Build the fuzzers (see docs/devel/fuzzing.txt)
>     Note: Build with --enable-sanitizers, or create a "dictionary file":
>     echo kw1=\"FUZZ\" > dict
>     and pass it as an argument to libFuzzer with -dict=./dict
>     This magic value is a command separator that lets the fuzzer perform
>     multiple IO actions with a single input.
> 
>  2. Pick the qemu arguments you wish to fuzz:
>     export QEMU_FUZZ_ARGS="-M q35 -device virtio-balloon"
> 
>  3. Tell the fuzzer which QOM objects or MemoryRegion names to fuzz. I find the
>  "info qom-tree", "info qtree" and "info mtree" commands useful for identifying
>  these. Supports globbing. Here I will try to simultaneously fuzz(for no good
>  reason) virtio-balloon and e1000e, which is included by default in the q35:
>     export QEMU_FUZZ_OBJECTS='virtio* e1000*'
>     You can also try to fuzz the whole machine:
>     export QEMU_FUZZ_OBJECTS='*'
> 
>  4. Run the fuzzer for 0 inputs. The fuzzer should output a list of
>  MemoryRegions/PCI Devices it will try to fuzz. Confirm that these match your
>  expectations.
>     ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz -runs=0
> 
>  5. Run the fuzzer:
>     ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz 
> 
> 
> Basically, at the core, this fuzzer is an interpreter that splits the input
> into a series of commands, such as mmio_write, pio_write, etc. We structure
> these commands to hit only MemoryRegions that are associated with the devices
> specified in QEMU_FUZZ_OBJECTS. Additionally, these patches add "hooks" to
> functions that are typically used by virtual-devices to read from RAM (DMA).
> These hooks attempt to populate these DMA regions with fuzzed data, just in
> time.
> 
> Some of the issues I have found or reproduced with this fuzzer:
> https://bugs.launchpad.net/bugs/1525123
> https://bugs.launchpad.net/bugs/1681439
> https://bugs.launchpad.net/bugs/1777315
> https://bugs.launchpad.net/bugs/1878034
> https://bugs.launchpad.net/bugs/1878043
> https://bugs.launchpad.net/bugs/1878054
> https://bugs.launchpad.net/bugs/1878057
> https://bugs.launchpad.net/bugs/1878067
> https://bugs.launchpad.net/bugs/1878134
> https://bugs.launchpad.net/bugs/1878136
> https://bugs.launchpad.net/bugs/1878253
> https://bugs.launchpad.net/bugs/1878255
> https://bugs.launchpad.net/bugs/1878259
> https://bugs.launchpad.net/bugs/1878263
> https://bugs.launchpad.net/bugs/1878323
> https://bugs.launchpad.net/bugs/1878641
> https://bugs.launchpad.net/bugs/1878642
> https://bugs.launchpad.net/bugs/1878645
> https://bugs.launchpad.net/bugs/1878651
> https://bugs.launchpad.net/bugs/1879223
> https://bugs.launchpad.net/bugs/1879227
> https://bugs.launchpad.net/bugs/1879531
> https://bugs.launchpad.net/bugs/1880355
> https://bugs.launchpad.net/bugs/1880539
> https://bugs.launchpad.net/bugs/1884693
> https://bugs.launchpad.net/bugs/1886362
> https://bugs.launchpad.net/bugs/1887303
> https://bugs.launchpad.net/bugs/1887309
> https://bugs.launchpad.net/bugs/697510
> 
> Alexander Bulekov (16):
>   memory: Add FlatView foreach function
>   fuzz: Add general virtual-device fuzzer
>   fuzz: Add PCI features to the general fuzzer
>   fuzz: Add DMA support to the generic-fuzzer
>   fuzz: Declare DMA Read callback function
>   fuzz: Add fuzzer callbacks to DMA-read functions
>   fuzz: Add support for custom crossover functions
>   fuzz: add a DISABLE_PCI op to general-fuzzer
>   fuzz: add a crossover function to generic-fuzzer
>   scripts/oss-fuzz: Add script to reorder a general-fuzzer trace
>   scripts/oss-fuzz: Add crash trace minimization script
>   fuzz: Add instructions for using general-fuzz
>   fuzz: add an "opaque" to the FuzzTarget struct
>   fuzz: add general-fuzz configs for oss-fuzz
>   fuzz: register predefined general-fuzz configs
>   scripts/oss-fuzz: remove the general-fuzz target
> 
>  docs/devel/fuzzing.txt                        |  39 +
>  include/exec/memory.h                         |  21 +
>  include/exec/memory_ldst_cached.h.inc         |   3 +
>  memory_ldst.c.inc                             |   4 +
>  scripts/oss-fuzz/build.sh                     |   6 +
>  scripts/oss-fuzz/minimize_qtest_trace.py      | 157 +++
>  .../oss-fuzz/reorder_fuzzer_qtest_trace.py    | 103 ++
>  softmmu/memory.c                              |  23 +
>  softmmu/physmem.c                             |   2 +
>  tests/qtest/fuzz/fuzz.c                       |  13 +
>  tests/qtest/fuzz/fuzz.h                       |  28 +
>  tests/qtest/fuzz/general_fuzz.c               | 948 ++++++++++++++++++
>  tests/qtest/fuzz/general_fuzz_configs.c       | 140 +++
>  tests/qtest/fuzz/general_fuzz_configs.h       |  24 +
>  tests/qtest/fuzz/meson.build                  |   1 +
>  15 files changed, 1512 insertions(+)
>  create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py
>  create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py
>  create mode 100644 tests/qtest/fuzz/general_fuzz.c
>  create mode 100644 tests/qtest/fuzz/general_fuzz_configs.c
>  create mode 100644 tests/qtest/fuzz/general_fuzz_configs.h
>