From nobody Sat Oct 5 21:26:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1725266331286835.3393195690672; Mon, 2 Sep 2024 01:38:51 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 44700B92; Mon, 2 Sep 2024 04:38:50 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id C6E7312B6; Mon, 2 Sep 2024 04:35:07 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 03C2A128F; Mon, 2 Sep 2024 04:35:04 -0400 (EDT) Received: from mail-oa1-f47.google.com (mail-oa1-f47.google.com [209.85.160.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 60F011283 for ; Mon, 2 Sep 2024 04:35:02 -0400 (EDT) Received: by mail-oa1-f47.google.com with SMTP id 586e51a60fabf-277e42a451dso30759fac.0 for ; Mon, 02 Sep 2024 01:35:02 -0700 (PDT) Received: from localhost.localdomain ([106.208.103.22]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-715e55769c6sm6420278b3a.33.2024.09.02.01.34.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Sep 2024 01:35:00 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725266101; x=1725870901; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=jswq7yn5akSZUg0Dk3OO1TeVW4k4MDynwK0khP1NpP4=; b=HJOT2znJXXify4S15HFJ8yW9p8xRqJ1xB08yb5FOiVZnel5GYFAtTXJ/DIgO4xj+m2 Y4fTTi+AhnOpo6Dx33mmkEtUDv3Z7fUsIbV0ts+tfgpm9ZY18S+hjnppsDAihtRpDTAK 4kTMZey6SFLRKJP9OpjowfA16QERudh/eEBbjU2Z4Nq5IJ+D6SD3CnxtCDKUeHF4QjqF bEr0xslD9ugtZb2F6Ejlp8ucT/D0uZcXkiJnNicbvtC4LnWif8bnX2jN2qa7TNyEGse5 AEhwZhZ7w/KMKn5K7bd3puN2z+ZJKumiWCNoaQ2CosxjMAzLPRFk3IzAzWjdr8ocaZsq DCKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725266101; x=1725870901; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jswq7yn5akSZUg0Dk3OO1TeVW4k4MDynwK0khP1NpP4=; b=SJ4Ve7UJAb0J61T78ksGy1I2A2wsgs8F1JUNT4D7MOIudZ6NaK/DGbOREpTU37rtKu VTlBljYpaCJIISGfPxOgb4oPxhd4q79cU0LVNWT85OMnH7MM99b+d2AcRzsYQNfwR8Qa MA0T1+1GDqMhDimsDJAZr/se3oSecL/X2j3OKUc9yMEYo72+Ju3zlQBTRLlIj6pqheT1 Ni1Zwbu2bTtWU4uDgqem0upGzNb6QNfd6iwMDLk63PGNFerjR3qe4PaIrBrK3cAXUcMd Gtw9IDoVg6o4ekUxwtzsYiCGE6zkG6aHF9/AWdJuaaQ/t6bozlTaa63cpMr5lGim9ipN MYzw== X-Gm-Message-State: AOJu0Yw7gwqQzNFVniiXShd2RKW8BKhjoJJkgx72fh4sRkeepElEKU7X QD22DKYcz4VDYWxmAzw7z2XofnuJ88QTgvVBZzrTjK5agOhOrWI/0RcIGw== X-Google-Smtp-Source: AGHT+IHbGHqDYLnr7J+5z/DZqyneHCcD2ImLCoRRCY4Nczvvm/DCO3U4Mkizg9S7tVG/rmo8yMNthg== X-Received: by 2002:a05:6870:4413:b0:25e:dce:491b with SMTP id 586e51a60fabf-277b0b47501mr5458531fac.1.1725266101092; Mon, 02 Sep 2024 01:35:01 -0700 (PDT) From: Abhiram Tilak To: devel@lists.libvirt.org Subject: [RFC PATCH] docs: add documentation for the AFL Fuzzing method Date: Mon, 2 Sep 2024 04:33:50 -0400 Message-Id: <20240902083349.146209-1-atp.exp@gmail.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: 54IFG3U4C6EOTYIWGJNEJP3RD75ECA6S X-Message-ID-Hash: 54IFG3U4C6EOTYIWGJNEJP3RD75ECA6S X-MailFrom: atp.exp@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Abhiram Tilak X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1725266332653116600 Content-Type: text/plain; charset="utf-8" I have worked on fuzzing libvirt with AFLPlusPlus over the summer and documented my fuzzing setup [1], to help replicate my setup for anyone who wants to take on fuzzing libvirt. There are few ideas on how to move forward: - Have the gitlab repo setup as it is, and if anyone is looking to take forward fuzzing libvirt, they will find my repo in a search-engine. - Add a brief documented summary of my setup in the docs section for libvirt repo. This would make it easier for anyone to find my repo. The following patch adds the above mentioned summary to the 'kbase/internals' section. Adds a basic documentation required to replicate the fuzzing setup and explains usage of various scripts. The contents of this page is a ReStructured version of the README found in the original setup repo [1]. [1]: https://gitlab.com/edupua/libvirt-aflfuzz Signed-off-by: Abhiram Tilak --- docs/kbase/index.rst | 3 + docs/kbase/internals/afl-fuzzing.rst | 275 +++++++++++++++++++++++++++ docs/kbase/internals/meson.build | 1 + 3 files changed, 279 insertions(+) create mode 100644 docs/kbase/internals/afl-fuzzing.rst diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index e51b35cbfc..1cbd2fc762 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -116,3 +116,6 @@ Internals =20 `QEMU monitor event handling `__ Brief outline how events emitted by qemu on the monitor are handlded. + +`XML Fuzzing using AFLPlusPlus `__ + Guide to use AFLPlusPlus for fuzzing libvirt's XML parsers. diff --git a/docs/kbase/internals/afl-fuzzing.rst b/docs/kbase/internals/af= l-fuzzing.rst new file mode 100644 index 0000000000..2c37847476 --- /dev/null +++ b/docs/kbase/internals/afl-fuzzing.rst @@ -0,0 +1,275 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D +AFL Fuzzing method +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D + +Setup +=3D=3D=3D=3D=3D + +Basic Setup +----------- + +Current setup uses a debian12 (bookworm) virtual machine. On a fresh +installation, you will need the following packages installed on your +system. Some really basic debian setup instructions are not mentioned +here like adding user to sudo group etc. + +The helper scripts and other required files mentioned here can be found in +[this](https://gitlab.com/edupua/libvirt-aflfuzz) repository. This +repo will be referred as 'helper repo' in the following guide. + +- Libvirt Dependencies + (`source `__) + +.. code:: sh + + sudo apt install \ + augeas-lenses augeas-tools bash-completion black ca-certificates \ + ccache clang codespell cpp diffutils dnsmasq dwarves ebtables \ + flake8 gcc gettext git grep iproute2 iptables kmod libacl1-dev \ + libapparmor-dev libattr1-dev libaudit-dev libblkid-dev libc6-dev \ + libcap-ng-dev libclang-rt-dev libcurl4-gnutls-dev libdevmapper-dev \ + libfuse-dev libglib2.0-dev libglusterfs-dev libgnutls28-dev libiscs= i-dev \ + libnbd-dev libnl-3-dev libnl-route-3-dev libnuma-dev libparted-dev \ + libpcap0.8-dev libpciaccess-dev librbd-dev libreadline-dev libsanlo= ck-dev \ + libsasl2-dev libselinux1-dev libssh-gcrypt-dev libssh2-1-dev libtir= pc-dev \ + libudev-dev libxen-dev libxml2-dev libxml2-utils libyajl-dev locale= s \ + lvm2 make meson nfs-common ninja-build numad open-iscsi perl-base \ + pkgconf policykit-1 python3 python3-docutils python3-pytest qemu-ut= ils \ + sed systemtap-sdt-dev wireshark-dev xsltproc apparmor + +- AFLPlusPlus Dependencies + (`source `__) + +.. code:: sh + + sudo apt-get install \ + build-essential python3-dev automake cmake git flex bison \ + libglib2.0-dev libpixman-1-dev python3-setuptools cargo \ + libgtk-3-dev lld llvm llvm-dev clang ninja-build + + sudo apt-get install \ + gcc-$(gcc --version|head -n1|sed 's/\..*//'|sed 's/.* //')-plugin-d= ev \ + libstdc++-$(gcc --version|head -n1|sed 's/\..*//'|sed 's/.* //')-dev + +(separated the commands because the gcc-plugin-dev command requires gcc +to be installed.) + +- By default the debian path doesn=E2=80=99t contain ``/sbin`` and + ``/usr/sbin``, its good to update the path to have them + +.. code:: sh + + export PATH=3D/sbin:/usr/sbin/:/usr/local/sbin:$PATH + +Building AFLPlusPlus +-------------------- + +Build all the modules using the ``distrib`` target. + +.. code:: sh + + git clone https://github.com/AFLplusplus/AFLplusplus + cd AFLplusplus + make PERFORMANCE=3D1 CODE_COVERAGE=3D1 PROFILING=3D1 INTROSPECTION=3D1 = LLVM_CONFIG=3Dllvm-config-14 distrib + sudo make install + +The following variables are the recommended one, but it is possible to hav= e a +more simpler build if one wants to. + +Bulding AFL-Cov +--------------- + +.. code:: sh + + git clone https://github.com/vanhauser-thc/afl-cov.git + cd afl-cov + sudo make install + +Install tmux +------------ + +.. code:: sh + + sudo apt install tmux + +Building Libvirt with AFL harness +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D + +Currently there is a fork of +`libvirt `__ which adds support for +compiling the required harnesses. + +[Note: This repo might not be upto date with upstream, please rebase] + +The directory structure is really important because all the helper +scripts rely on it. + +- Libvirt is expected to be cloned in the home directory. + + .. code:: sh + + cd ~/libvirt + git clone https://gitlab.com/libvirt/libvirt + +Alternatively there is a patch series with required commits. You can +find them in the ``patches`` directory in the helper repo. Again, this mig= ht +not be upto date with the current repo and might need conflict handling. + +Apply all the patches in order, except the cover-letter. + +.. code:: sh + + git am patches/0001-afl-fuzzing-add-meson-support-for-fuzzing.patch + git am patches/0002-afl-fuzzzing-add-harness-for-net-create-and-net-por= t.patch + git am patches/0003-afl-fuzzzing-add-harness-for-nwfilter.patch + git am patches/0004-afl-fuzzzing-add-harness-for-vol-create-and-pool-cr= e.patch + git am patches/0005-afl-fuzzzing-add-harness-for-domain-create.patch + +Build Script +------------ + +The first script to be used is the build script which can be found in the +helper repo. It can build individual builds with support for different +compilers and different sanitizers. It also has a cmplog build which +requires a corresponding default build to run. (Using cmplog requires +2 repos, the cmplog build and a non-cmplog AFL build). + +.. code:: sh + + scripts/build.sh --all + +The full build is about 15 Gigs, please make sure you have provision for +it... code:: sh + + cd ~/libvirt + git clone https://gitlab.com/libvirt/libvirt + +The default build contains ``undefined sanitizer`` and +``address sanitizer``, there are other options like ``tsan`` and +``AFL_HARDEN`` are also produced. + +Corpus generation +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +There is a python generator tool +```lxmlgen`` `__ which can be used to +generate a decent corpus useful for fuzzing most commands. The required +documentation is in the repo itself and consists of just running two +scripts ``run.sh`` and ``cmin.sh``. + +.. code:: sh + + git clone https://gitlab.com/edupua/lxmlgen + cd lxmlgen + ./run.sh + ./cmin.sh --all + +The cmin.sh expects the default build to exist (``build_default``), +which should be generated in the previous step. + +The fuzzing scripts expect the final corpus to be in the ``corpus`` +directory in the root of the current repo. + +Fuzzing Individual builds +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + +Up from this point, a casual fuzzer can start fuzzing runs. Use the +helper scripts ``base.sh`` or ``custom_mutator.sh`` that pass extra +arguments to ``afl-fuzz`` thus, there are +`plethora `__ of options to play +around with. + +.. code:: sh + + scripts/base.sh net-create build_default + +Custom Mutator +-------------- + +There is a simple `custom mutator `__ +write in python. To take full advantage of this mutator, please use it +singly with ``AFL_CUSTOM_MUTATOR_ONLY`` option. + +Once you clone the repo don=E2=80=99t forget to install the ``requirements= .txt`` +and you can use the script ``custom_mutator.sh`` just like ``base.sh``. + +This script expects the mutator to be cloned into a folder called +``custom_mutator``, with the python libraries installed. It also expects +the corresponding RelaxNG schema files to be placed in ``schemas`` dir, +which should be present by default. + +Generate a tmux command +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +If you have access to a large amount of computational power with +multiple cores it is recommended to run a parallel fuzz with multiple +instances running at once. Its meant to only run one afl subcommand at +once. + +The script ``gen_tmux.sh`` automatically detects all the build +directories in the libvirt directory and uses a relevant combination of +``afl-fuzz`` options with different weights for each option. + +It also has supports for ``--custom_mutator`` argument which uses +``custom_mutator.sh`` instead of ``base.sh``. + +.. code:: sh + + scripts/gen_tmux.sh net-create --custom_mutator + +Finally this script should generate a ``mega.sh`` which deploys multiple +instances. The script generates approximately 30 instances and each run +uses a core of CPU, and may require manually adding or removing +instances accordingly. + +Gather Coverage +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +There is again a script to generate coverage for a given subcommand. + +.. code:: sh + + scripts/cov.sh net-create + +Note: This command can take a really long time since it generates +coverage for each and every queue file in the corpus and does so for all +the master plus slave instances of a fuzzing run. + +You can use the argument ``--cover-corpus`` which avoids running each +file and is lot faster. + +The final coverage can be viewed in HTML format in the ``cov/web`` +directory inside the specified output folder. + +See Also +=3D=3D=3D=3D=3D=3D=3D=3D + +Setting up AFLPlusPlus in a new machine may require a tuning. Please use +the following resources: + +- Ultimate fuzzing using AFLPlusPlus guide - + https://aflplus.plus/docs/fuzzing_in_depth/ +- Performance Tips in AFLPlusPlus - + https://aflplus.plus/docs/perf_tips/ +- Parallel fuzzing - https://aflplus.plus/docs/parallel_fuzzing/ + +There is also detailed documentation that can be easily found in AFL=E2=80= =99s +github repo: - `Custom +mutators `__ +- Use this guide to help craft custom mutators for AFLPlusPlus, also +take a look at the reference links to find some related links. - +`Instrumentation `__ +- This directory contains a detailed list of popular instrumentation +methods used and some related examples. + +It is also really important to have a great understand of how AFL works +under the hood before using it to do something useful. Refer to `this +blog `__ + +Please consider looking at other efforts to fuzz libvirt as well: + +- AltLinux: `Fuzzing libvirt + RPC `__ +- Libfuzzer + Libprotobuf-mutator: `Fuzzing libvirt using grammar aware + mutator `__ + diff --git a/docs/kbase/internals/meson.build b/docs/kbase/internals/meson.= build index f1e9122f8f..b944c50e0b 100644 --- a/docs/kbase/internals/meson.build +++ b/docs/kbase/internals/meson.build @@ -9,6 +9,7 @@ docs_kbase_internals_files =3D [ 'qemu-migration', 'qemu-threads', 'rpc', + 'afl-fuzzing', ] =20 =20 --=20 2.39.2