[PULL 07/10] docs/fuzz: add instructions for generating a coverage report

Thomas Huth posted 10 patches 5 years, 3 months ago
Maintainers: Sagar Karandikar <sagark@eecs.berkeley.edu>, Stefan Hajnoczi <stefanha@redhat.com>, Wainer dos Santos Moschetta <wainersm@redhat.com>, "Alex Bennée" <alex.bennee@linaro.org>, Peter Maydell <peter.maydell@linaro.org>, Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, Alexander Bulekov <alxndr@bu.edu>, Bastian Koppelmann <kbastian@mail.uni-paderborn.de>, Palmer Dabbelt <palmer@dabbelt.com>, "Philippe Mathieu-Daudé" <philmd@redhat.com>, Subbaraya Sundeep <sundeep.lkml@gmail.com>, Alistair Francis <alistair@alistair23.me>, Laurent Vivier <lvivier@redhat.com>, Niek Linnenbank <nieklinnenbank@gmail.com>, "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>, Beniamino Galvani <b.galvani@gmail.com>, Thomas Huth <thuth@redhat.com>, Bandan Das <bsd@redhat.com>
[PULL 07/10] docs/fuzz: add instructions for generating a coverage report
Posted by Thomas Huth 5 years, 3 months ago
From: Alexander Bulekov <alxndr@bu.edu>

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20200706195534.14962-5-alxndr@bu.edu>
[thuth: Replaced --enable-sanitizers with --enable-fuzzing]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/devel/fuzzing.txt | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/docs/devel/fuzzing.txt b/docs/devel/fuzzing.txt
index 6d18115239..96d71c94d7 100644
--- a/docs/devel/fuzzing.txt
+++ b/docs/devel/fuzzing.txt
@@ -85,6 +85,25 @@ better coverage performance, depending on the target.
 Note that libFuzzer's exact behavior will depend on the version of
 clang and libFuzzer used to build the device fuzzers.
 
+== Generating Coverage Reports ==
+Code coverage is a crucial metric for evaluating a fuzzer's performance.
+libFuzzer's output provides a "cov: " column that provides a total number of
+unique blocks/edges covered. To examine coverage on a line-by-line basis we
+can use Clang coverage:
+
+ 1. Configure libFuzzer to store a corpus of all interesting inputs (see
+    CORPUS_DIR above)
+ 2. ./configure the QEMU build with:
+    --enable-fuzzing \
+    --extra-cflags="-fprofile-instr-generate -fcoverage-mapping"
+ 3. Re-run the fuzzer. Specify $CORPUS_DIR/* as an argument, telling libfuzzer
+    to execute all of the inputs in $CORPUS_DIR and exit. Once the process
+    exits, you should find a file, "default.profraw" in the working directory.
+ 4. Execute these commands to generate a detailed HTML coverage-report:
+ llvm-profdata merge -output=default.profdata default.profraw
+ llvm-cov show ./path/to/qemu-fuzz-i386 -instr-profile=default.profdata \
+ --format html -output-dir=/path/to/output/report
+
 == Adding a new fuzzer ==
 Coverage over virtual devices can be improved by adding additional fuzzers.
 Fuzzers are kept in tests/qtest/fuzz/ and should be added to
-- 
2.18.1