From nobody Tue Nov 18 07:44:08 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass(p=none dis=none) header.from=outlook.com ARC-Seal: i=2; a=rsa-sha256; t=1609216940; cv=pass; d=zohomail.com; s=zohoarc; b=Nlo+D/9/m65Tm0O8ozveZmgi0jS8deMLPTGx2UHNLp6O+mCA1E22fqEPVhh4dvYMrpwsSDoY7/UdekkJAMosnJQhTjIsrq6QkmlwSg75FXhvmvpiwunEzo0BLvXhNhDz4E7U5xveoP3no9G2BxsmPzazumMJWwYllA8IKRRHGpA= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1609216940; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=asFEf41E7C/LI6HYvtDP5NrBaUVrKW4QghpxAocWPww=; b=H+0TxZZ0o9ZzwYREQG8h3SbZILCTXiDpKVfGvPuPcwKUGRWNfviyc+qD9qjBYlZWArRpFB5mbXkzYFf/mYn0r4wZzfpc6w0b3lLa4AQufvClgyeoMPJrv+ZHO/m52kQD7TwpKiBHeWuNMPazMCPGiMTwDPvfKNDcNTVUn6rvv44= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1609216940569837.820221700594; Mon, 28 Dec 2020 20:42:20 -0800 (PST) Received: from localhost ([::1]:58056 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ku6pv-0005ZO-8n for importer@patchew.org; Mon, 28 Dec 2020 23:42:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38472) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6or-0004ax-2z for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:13 -0500 Received: from mail-oln040092255107.outbound.protection.outlook.com ([40.92.255.107]:27280 helo=APC01-HK2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6oo-0004R0-Um for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:12 -0500 Received: from SG2APC01FT025.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::4c) by SG2APC01HT192.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::412) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27; Tue, 29 Dec 2020 04:41:01 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2a01:111:e400:7ebd::44) by SG2APC01FT025.mail.protection.outlook.com (2a01:111:e400:7ebd::187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:01 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e]) by ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e%8]) with mapi id 15.20.3700.031; Tue, 29 Dec 2020 04:41:00 +0000 Received: from pc (2001:250:fe01:130:6d26:2348:77a5:f7ed) by TYAPR01CA0057.jpnprd01.prod.outlook.com (2603:1096:404:2b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HH5osuqtduFnTDpO2Brq99GE+PQTwZHf11u/EEwh4G/nQ+5ttYHpsTYwttMTEKUrwAVpiKMYrSLbBSnCPaSwORmChXHfV8/E+3sFim7dp2YGb3zUY94paAl0lKPnwYL8ztEookOMQwKRHsP/PifvOYusbUBj8cTUyB5WeGak0Er1xpXq2BsBWYrBClH4/xeReP3KGCMJD8lhppGPoRSfCj4TY7NHJw7QPnCOEklHcGMcYo0mnWlbtsIUsc4Ee53dr5jKhZ3A/3ZTeaq/7U62Edk+rmmSw3xe2/x9c2QZz6lXjO6N+5DwPDyeBAo0UFNZEwNDPKSZrKtesOMChdMW1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=asFEf41E7C/LI6HYvtDP5NrBaUVrKW4QghpxAocWPww=; b=bCH29inEu9MdiYSTX+BnhaOpsIU93/SQIlsG92FXBnhLqVCianHp9vJrTR6ax25luPTB5BGLqZW7faFDNDkQ5tMhwc0N4MRhg6V+HBsmK2dR23iSbCw/TC/0TUd5wZJ317SleaPZA1S4oEy2djwMtcaD7KrLX59O4jsC/pfuBMeP60dhCq2BpAp0qSHZ76YMmQBRiQyuiDyHeUg4lbVWRnpfeJU+DgAZea+d9JN8knJ3kffiPYOI9T+771BNMyDtZthDtCp2hU5SKOPTXBUQzBzRaTICJ1XrEI95fG/15WLUZCyjz0YKRHRfJji9UtlllUjrP0ZAvWTuAomzs5Ma3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=asFEf41E7C/LI6HYvtDP5NrBaUVrKW4QghpxAocWPww=; b=E6SPiVkDUyI4wZpduGkUvKR4Lv//01cdlZ5BZUO9N75uiCJKbb2FRZlXhYa331wSHYX84CmDW2uTv4cwSb61etLk3OE3O0opd/Vlmx09kfajkcQ6WBW9seRHB5bfMjDTnyJATazhyoIW0/WzIXEAITO0/nepA0vijWaH3ErGfTyks/bLp8IS9m1NeC9MvJ3hRvv5vk0ubV4HIcABEzA02lPLGsAF1BWSeIUzTPgk4rErDJZE7/NehnO4wljV7S16WIaNzIZfffi6DsTPeLkrkpUDlZvM2eJ4+g38zHpEPDEIA5MAq1nSVcLWaTZ2q1EiQuOvN+kVhPLgwpW/TxyaQA== X-IncomingTopHeaderMarker: OriginalChecksum:D2D56D144EA6D73698207E9F8DEAEB473B947F2B79102BAE9CA82C7C8623C282; UpperCasedChecksum:9B4432929F5D4A1198E8F76D46BD775A7C722C04309497AB84A66CC840073734; SizeAsReceived:7649; Count:47 From: Qiuhao Li To: alxndr@bu.edu, qemu-devel@nongnu.org Subject: [PATCH v4 1/7] fuzz: accelerate non-crash detection Date: Tue, 29 Dec 2020 12:40:38 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-TMN: [FpNWG0zo7Thhc7YUudQoHoWXt65tcbg3igWpP9zhuFjLlCdmlGdeYGOYXlFerJMz] X-ClientProxiedBy: TYAPR01CA0057.jpnprd01.prod.outlook.com (2603:1096:404:2b::21) To ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:ac::12) X-Microsoft-Original-Message-ID: <20201229044044.3706227-1-Qiuhao.Li@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 2a357311-8c2c-4a49-78a2-08d8abb3f158 X-MS-Exchange-SLBlob-MailProps: pt8ZUcDWjMWIUuQLPiUjHYmN0Ah6oaNn166PGN/57hxfdzQIfXPyq26CV2avYg1/z/7/Bh6FrMDJazJTKLSCCqMO8AAJw66sBEwBGEsYsdZGCergt8w711My6CaE7uE0L7fyo2BixcwCIF3GcoElUdb3NCFJn2o3VvY+bzZVEiyW7Irk+8z4UcDS0DWRj0GEu1KN+6dSD4RCZ0xp9WcY2Jnw8sTILQnIVcsOM5VzZWBLCYd2yQY4F+28Udxqm8zF94YcXaKquX/lTTfnx6PPoC142FdC6q7zvPkQxkUezh2Q6I6R+IfzInUKGS2jTJ+bkdmUicGliOnHpjWdAj8c6rrKY7KX0Ba2akWc7w5brIr00ukvH/z0gFgVLuhTJqyYAKtRE+sfv6ZbHtCzCAKWpECBQOm2XwtJY1LfgY9BZhazZY+dG0ffw/Hf6qHmsDaxwQJHP1dvIydZkqu7M9KlGSoZRnBvbOn9ZLaIVp7+wSwmtgITA35YTgDPSnQW3x+W7rnjalyEKH6CnFIGjE411Gd+T/Fb35pKIBKmzWKGdK8EIWjfGqK5LvQgJBv7CeGy7TmE6W+TzRKA743ktrBjlJzTJk/y1mK8OxPDehZOAY1Ajb/2mEFXunFALstCGSFpG844jrz3IjwsRBy8mW0mK/SdboUs1oiu8hgmGL0OU1CJifIvHyln5h1cS5IkRINwfGv5zzTDiOymj5okXOwviiYBzQRTgd2zcbXfl8JSAzEZRWe5AwELGEJb3DCi7iM8KnifBBSOBJXXJc9qpOCzl7v025ZHADHTwNrIL7aTznf+t+jbxmdg+gQ4XaaduBnSs9X+2/c+B8PTIquOZ4nbAketC1bzAmBGzNNDSvtFYCT6JLnY3VeOc/9tfTuHCtWbMtSssalCZpJ7+IDw27uob+QYk0ih73SuLV9/WeEMVMoyYatsKJ4D7TzCnIEcqrp8 X-MS-TrafficTypeDiagnostic: SG2APC01HT192: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: t+Rt/VasACneowm2B4HOtXWT08WrGjC9g7TyfXBHpoMjjD6YU5yrxvOOaaMJKLdsBiNBU2nFURD8yiaKLn9vxb/xK81CuM83MCAuvghjAyOaG+FgwjE7WT3XN2YYEfegmOb5kzzO2h8Wd6B01kAheU3dS0KtmEX/YqcGVbyXk3S1kmb44kiZ9Yxs689cUcAQZj9z7SJ8h9lIOoTUgkGb2tUC9LAZoJmav8G52NofjvSXzXanw/TnD7hI5aBDk2U/37a+qZYzwv9sKQT4m0IbhVpD26vrdcTwKgQm+wZLjW0= X-MS-Exchange-AntiSpam-MessageData: WcfXWgyUMo2WHlxJT5w4AbWkZ40exVKdupwRy1/kGbbPqOcHT2UlcRzc4KcxVVBfZFNkvj7IYIbrXDsIhCc8+B+8IZxzaPEm4wtpa+LnWWpTKFBuw8I5kMYRJpOKVipufjypYUtaaDeOUQyTInlFCXqudaHZrFyLNvfivg0N1sp5WgY9a3O9NmCrmLHyJHImcz+u4jiO+7RWAvgIiDQTtA== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2020 04:41:00.8880 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-Network-Message-Id: 2a357311-8c2c-4a49-78a2-08d8abb3f158 X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT025.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT192 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.92.255.107; envelope-from=Qiuhao.Li@outlook.com; helo=APC01-HK2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, Qiuhao Li , darren.kenny@oracle.com, bsd@redhat.com, stefanha@redhat.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @outlook.com) Content-Type: text/plain; charset="utf-8" We spend much time waiting for the timeout program during the minimization process until it passes a time limit. This patch hacks the CLOSED (indicates the redirection file closed) notification in QTest's output if it doesn't crash. Test with quadrupled trace input at: https://bugs.launchpad.net/qemu/+bug/1890333/comments/1 Original version: real 1m37.246s user 0m13.069s sys 0m8.399s Refined version: real 0m45.904s user 0m16.874s sys 0m10.042s Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov --- scripts/oss-fuzz/minimize_qtest_trace.py | 41 ++++++++++++++++-------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/mi= nimize_qtest_trace.py index 5e405a0d5f..aa69c7963e 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -29,30 +29,46 @@ whether the crash occred. Optionally, manually set a st= ring that idenitifes the crash by setting CRASH_TOKEN=3D """.format((sys.argv[0]))) =20 +deduplication_note =3D """\n\ +Note: While trimming the input, sometimes the mutated trace triggers a dif= ferent +crash output but indicates the same bug. Under this situation, our minimiz= er is +incapable of recognizing and stopped from removing it. In the future, we m= ay +use a more sophisticated crash case deduplication method. +\n""" + def check_if_trace_crashes(trace, path): - global CRASH_TOKEN with open(path, "w") as tracefile: tracefile.write("".join(trace)) =20 - rc =3D subprocess.Popen("timeout -s 9 {timeout}s {qemu_path} {qemu_arg= s} 2>&1\ + proc =3D subprocess.Popen("timeout {timeout}s {qemu_path} {qemu_args} = 2>&1\ < {trace_path}".format(timeout=3DTIMEOUT, qemu_path=3DQEMU_PATH, qemu_args=3DQEMU_ARGS, trace_path=3Dpath), shell=3DTrue, stdin=3Dsubprocess.PIPE, - stdout=3Dsubprocess.PIPE) - stdo =3D rc.communicate()[0] - output =3D stdo.decode('unicode_escape') - if rc.returncode =3D=3D 137: # Timed Out - return False - if len(output.splitlines()) < 2: - return False - + stdout=3Dsubprocess.PIPE, + encoding=3D"utf-8") + global CRASH_TOKEN if CRASH_TOKEN is None: - CRASH_TOKEN =3D output.splitlines()[-2] + try: + outs, _ =3D proc.communicate(timeout=3D5) + CRASH_TOKEN =3D outs.splitlines()[-2] + except subprocess.TimeoutExpired: + print("subprocess.TimeoutExpired") + return False + print("Identifying Crashes by this string: {}".format(CRASH_TOKEN)) + global deduplication_note + print(deduplication_note) + return True =20 - return CRASH_TOKEN in output + for line in iter(proc.stdout.readline, b''): + if "CLOSED" in line: + return False + if CRASH_TOKEN in line: + return True + + return False =20 =20 def minimize_trace(inpath, outpath): @@ -66,7 +82,6 @@ def minimize_trace(inpath, outpath): print("Crashed in {} seconds".format(end-start)) TIMEOUT =3D (end-start)*5 print("Setting the timeout for {} seconds".format(TIMEOUT)) - print("Identifying Crashes by this string: {}".format(CRASH_TOKEN)) =20 i =3D 0 newtrace =3D trace[:] --=20 2.25.1 From nobody Tue Nov 18 07:44:08 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass(p=none dis=none) header.from=outlook.com ARC-Seal: i=2; a=rsa-sha256; t=1609216943; cv=pass; d=zohomail.com; s=zohoarc; b=ZuaEy0sdxHrdZFbOHc/1u2PsAnVjY3SS7aAxr3ICYuD5dfzpk6iq/jJ4oeGak53JLxpjHWlmwdonCkDmHJlRIeGeywjABnRuc+/kVCuz/kmtycNtYbjiIXmqct4rolmYN2w/hIhyb3Fu7vNMLiG692Ghr4LcqlRa9Fad0zSZGvg= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1609216943; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=sG1GmprGsMl3k4rRNoIj6GXzWL3roc1+FGD8fQ/FZ5E=; b=NRYPp9jvBgF6zkYst5DV/4Dvt5dcg+2fJS7kAjtHhgPcUrLQ62PDj44OKLG2GbzNL4lBjkVRn0qhoOXITxUy+8pNaUdgsDdSvgmmLEIftWVFmJEltdRdUImNfgGDnZa2BuZFei13GsCP1cbSt19iggyidgUaz4WCCS+b5K5hUAs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1609216943152368.6025042804944; Mon, 28 Dec 2020 20:42:23 -0800 (PST) Received: from localhost ([::1]:58242 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ku6py-0005dv-3R for importer@patchew.org; Mon, 28 Dec 2020 23:42:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38508) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6ou-0004dG-Cw for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:18 -0500 Received: from mail-oln040092255107.outbound.protection.outlook.com ([40.92.255.107]:27280 helo=APC01-HK2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6or-0004R0-Sg for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:16 -0500 Received: from SG2APC01FT025.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::4c) by SG2APC01HT192.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::412) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27; Tue, 29 Dec 2020 04:41:07 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2a01:111:e400:7ebd::44) by SG2APC01FT025.mail.protection.outlook.com (2a01:111:e400:7ebd::187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:07 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e]) by ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e%8]) with mapi id 15.20.3700.031; Tue, 29 Dec 2020 04:41:07 +0000 Received: from pc (2001:250:fe01:130:6d26:2348:77a5:f7ed) by TYAPR01CA0062.jpnprd01.prod.outlook.com (2603:1096:404:2b::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SxoU2Uv5cdAEUL0GhBN5k/Qd9RIsERt5OU/k2CNVZImno1CVnz1pKkakD3Ry7+WWVwy3rE0HdsVxncZZwKcd4uTNbiLsaQJ0aZA0xltlPnEA/G6sX9cq1PlmDjiC0qaIxiaCkWW16cXrl6qHim037/bRSDrkom+jwjduz52Vg5/pmwQhNreG+fiVE5UIynMqymy5t+gExCaG+4wYcMFwcmRBFmlnrRcZ7nvkExgSbGSAccRW5vHomZ4X05ulZ/mgHeboHxSnzH+DTN3EPO7gEavALk7WRHdMz7gVB6P8PUeDLLIj+Vt8Sa+4rjmSZqwZvYxqZyV2dGAlrNhai5ixlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sG1GmprGsMl3k4rRNoIj6GXzWL3roc1+FGD8fQ/FZ5E=; b=gfsAK1OfPItw6+lwA+469EeisLZG6ktKemx6/QxQ8WMuvpJIOb/MNG5Ptlio2dfw2dZMbuOwnUJHaLo8K5D1iDBrOosPcnaOzb38VX8j6lPO+GQ49M0gi+AqXrtgUMqim6turw8aQTzBp7HoNQFkJIJhFdPrDYQChpHEHWYiqhOfkuqN6j77FuqQ+Pj40g3CM3IIMuquiJStqO/xAV9BYWz8LzO/LpUwrCXuuKkdwjk9f570ABmh15jkPfUh2gp1Xj009huapIn6nWcLSnaF/0exeACR8y3LJxzO3LocnkbuVTdCorrbO1w+ZRCdi1X5Ve5Svi+8xT8npTwenWAuJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sG1GmprGsMl3k4rRNoIj6GXzWL3roc1+FGD8fQ/FZ5E=; b=GFIKOGbJl/ncaDyEzpdkJywBoI5polF2yQKIFd8oTrIl17EBbA49IDig3roh71Wg8ZxV0g3QDFYkCvOuD+5oxmAH+iW7H3t6AzaVaHmG/jKpMqtDtNPptsSMsH4H+HPV4hSjkMde57VfocZz0B9anE6SkEzbV+XuEFdlHIg2DucG1sr6ufWjpBsam8YUIUT6CAB/UVxlVUf6Y5ESmG3GDjo1IWdgb1/tV6q9HoykyYBO8qnBekHanckwNX9r0mPt+anF0scNshh+zDoRYn9Pw2DThYrdLbkF42I761Nqgblrhg9urPzFLsjyvhl9X11D90EI1Jz8nosUT7LF3iHMOg== X-IncomingTopHeaderMarker: OriginalChecksum:3446043F68947CD490118B5FAFAA73E187613CFD520AE690C050B58FE9B9E4FD; UpperCasedChecksum:70AA99C655FA50F8BEB802EE31C48E80781D6A685482623C8691848AFEF2748C; SizeAsReceived:7677; Count:47 From: Qiuhao Li To: alxndr@bu.edu, qemu-devel@nongnu.org Subject: [PATCH v4 2/7] fuzz: double the IOs to remove for every loop Date: Tue, 29 Dec 2020 12:40:39 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-TMN: [wJzSKXjFNAAFBU3D1PpuN4GpapTSscxoJj1msfLXoepMoAexyGPXFqytXxHbzP0p] X-ClientProxiedBy: TYAPR01CA0062.jpnprd01.prod.outlook.com (2603:1096:404:2b::26) To ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:ac::12) X-Microsoft-Original-Message-ID: <20201229044044.3706227-2-Qiuhao.Li@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 6be3f89a-4058-404e-a331-08d8abb3f52e X-MS-Exchange-SLBlob-MailProps: =?us-ascii?Q?KE5ywuOaN5gUKW4Ngaj7XpCefOhcv0ruVZnfG8mmZVeJim02NEm/Ja8/qxDQ?= =?us-ascii?Q?evFlw8W0w3tSjVSXlwlb7+tjEqNutwGEG9c2u6LBhpXC4LkMb4HOKW5gARYy?= =?us-ascii?Q?SrJZkt987mdYUCaqFB57/Q+UZxGFazNT+Ca3KVwwHu7kokZ30dGe2vlIrwn5?= =?us-ascii?Q?dvM1VZ6YR8MMByGdDBz9fAG4qBpvKcKfRNN9bkG2mRz38ks9LA5svdHjOAdN?= =?us-ascii?Q?oGPihvQKOe5LG00TQ5V8SBomPAsky4wPLUxyJg8dvR1UKS39vA7HBXJDin5h?= =?us-ascii?Q?zQC2kJrUqIksv8fytpIqO5MJYbi2YHfdfmm8cEaTpYDvWrmuSH4azzv0eyBF?= =?us-ascii?Q?ilT+K9npdMiOjcu1fobahBO5lgApXmM3ebR0Y7gWUDbtII/QfKOmVxwmm3YA?= =?us-ascii?Q?l1qv6fHzXLhBXOOllrAekOnqEHE+gXu8t/jnnfWM9YjTVf/yHnNM6uay4Moc?= =?us-ascii?Q?m8P9xgsNEeA0c6MDeLS9CyiYxCabXSuTckCn46/YhPBR5T1tVMSDyZlWKQ0j?= =?us-ascii?Q?6jNLO3tVZXAzfH75E2IU1TO5veD0jP9I3H1VkYrYIN8fjkT/5Tg8PLmFRbfF?= =?us-ascii?Q?cnF/eWN6B2a/BBPHj1wAcuuFOMlCUcN4WJz351+z2+bZjF530S7fMoIK94Qn?= =?us-ascii?Q?3NXuURGdd0VHsZx+sS3LpPrKs0w6QPELtbUbNwgCFZ3rBWjjkERHp8YDtn5m?= =?us-ascii?Q?8hFOown1KWSW7nPt6QND1KhpnpTk0PIXEpvJz/joxRiTN1f7BTD6WG44pWWB?= =?us-ascii?Q?b6d/r3AEuiAnC7rOy4kgJfjuY6LjoyDt/Li6Huais7iFh6qMzJgkDXzjk2NK?= =?us-ascii?Q?+fsMCYL+0GZBW9uzVQRIoi1m0yZYzb+timx6gYrGinUXUas5PSt7g0rXKdFz?= =?us-ascii?Q?vLfLtz+ngG/VT6GncD9eZEGMPWDahoVWTS2JGtEm/rqyAEDP2u9nXeffrs3R?= =?us-ascii?Q?Pb1q86ed4UtuvnzOub1iGUQDyv78Vy7Z?= X-MS-TrafficTypeDiagnostic: SG2APC01HT192: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PPkk8FkJ2Y4R1REDn+tFonXG97hfkoovIXjsSMEuvXI4SYZArrAuE6SjUYef+z6bXhYklvXpRSeSyEz3VY9iLPmCD66W/smPBN/1sCV/AimsLPUvKUuBaPiFHgunvMejzVDwREKIZ1weFiTjnTLN5kN2BPP+hUG1ZtaWi0mJSNjDa3CtGjYH+1LHecdkwpfBRRslKGUl6+8YRobZ+z18V8lF20gpDDmyOVEULXI2iiu9AeqxOJHT8+tlZmijk06u39bXyjno6t5uwZ52bfKV4p7lUFuBgHsj40Abhyovdss= X-MS-Exchange-AntiSpam-MessageData: GuEzI3nJtB/N8DlLQ6Jxbic7vWfH0Ft9gPNqgm+f4niOQmzYVC8mrCzrI4h64e2pYJRGKgDvUfVhfcIHp7wp1fbh4INx4G0ERGV5braE4RdFSMnFdfqcEF5NRgs73NEOW9vItf38jsn0m/TV0FHNbVQJTcRiIP2fV4/HITg/ysSjatEElbWpycEq+uqAab7Bnq1pHRb5r+dui3kouFnt6g== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2020 04:41:07.3467 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-Network-Message-Id: 6be3f89a-4058-404e-a331-08d8abb3f52e X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT025.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT192 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.92.255.107; envelope-from=Qiuhao.Li@outlook.com; helo=APC01-HK2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, Qiuhao Li , darren.kenny@oracle.com, bsd@redhat.com, stefanha@redhat.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @outlook.com) Content-Type: text/plain; charset="utf-8" Instead of removing IO instructions one by one, we can try deleting multiple instructions at once. According to the locality of reference, we double the number of instructions to remove for the next round and recover it to one once we fail. This patch is usually significant for large input. Test with quadrupled trace input at: https://bugs.launchpad.net/qemu/+bug/1890333/comments/1 Patched 1/6 version: real 0m45.904s user 0m16.874s sys 0m10.042s Refined version: real 0m11.412s user 0m6.888s sys 0m3.325s Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov --- scripts/oss-fuzz/minimize_qtest_trace.py | 33 +++++++++++++++--------- 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/mi= nimize_qtest_trace.py index aa69c7963e..0b665ae657 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -85,19 +85,28 @@ def minimize_trace(inpath, outpath): =20 i =3D 0 newtrace =3D trace[:] - # For each line + remove_step =3D 1 while i < len(newtrace): - # 1.) Try to remove it completely and reproduce the crash. If it w= orks, - # we're done. - prior =3D newtrace[i] - print("Trying to remove {}".format(newtrace[i])) - # Try to remove the line completely - newtrace[i] =3D "" + # 1.) Try to remove lines completely and reproduce the crash. + # If it works, we're done. + if (i+remove_step) >=3D len(newtrace): + remove_step =3D 1 + prior =3D newtrace[i:i+remove_step] + for j in range(i, i+remove_step): + newtrace[j] =3D "" + print("Removing {lines} ...".format(lines=3Dprior)) if check_if_trace_crashes(newtrace, outpath): - i +=3D 1 + i +=3D remove_step + # Double the number of lines to remove for next round + remove_step *=3D 2 continue - newtrace[i] =3D prior - + # Failed to remove multiple IOs, fast recovery + if remove_step > 1: + for j in range(i, i+remove_step): + newtrace[j] =3D prior[j-i] + remove_step =3D 1 + continue + newtrace[i] =3D prior[0] # remove_step =3D 1 # 2.) Try to replace write{bwlq} commands with a write addr, len # command. Since this can require swapping endianness, try both LE= and # BE options. We do this, so we can "trim" the writes in (3) @@ -118,7 +127,7 @@ def minimize_trace(inpath, outpath): if(check_if_trace_crashes(newtrace, outpath)): break else: - newtrace[i] =3D prior + newtrace[i] =3D prior[0] =20 # 3.) If it is a qtest write command: write addr len data, try to = split # it into two separate write commands. If splitting the write down= the @@ -151,7 +160,7 @@ def minimize_trace(inpath, outpath): if check_if_trace_crashes(newtrace, outpath): i -=3D 1 else: - newtrace[i] =3D prior + newtrace[i] =3D prior[0] del newtrace[i+1] i +=3D 1 check_if_trace_crashes(newtrace, outpath) --=20 2.25.1 From nobody Tue Nov 18 07:44:08 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass(p=none dis=none) header.from=outlook.com ARC-Seal: i=2; a=rsa-sha256; t=1609217074; cv=pass; d=zohomail.com; s=zohoarc; b=C4lhaewnHqdZZzTI8AA37LoB5POQPZKFyhWvxzk5ukKwukdm/6tT/r1ZCVtJmxkA9ceHG3aZ7FoZ2jEiSX8S5X+dk4WY2M71H2C/uhmBxPBdmLyH4FKLa6IExT165g5F0aG5hbSc5v4L73ul/LmxDYTOiSPjbMY8+RjYjvY9Q5Y= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1609217074; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=D7U3CWaqO/4BtDRpqyPH43l5mb6QPlr2k1azwmdY/YQ=; b=ZWilg/RDU9LVft+mpfFKXiKkwJCHJXC9EphSRQN/e55wGhyZDBFYBxmGyjk8T4qfjUq8oxbTIaySdt6wXe+L3JI/+hT95sz/8kAywUXfSOihncWEo1vDkvNugpn9ilCrkH02Asq5IooM5aQrxJ54jT+7YgxFIi2b4zno2WImAIs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1609217074078139.17417955576332; Mon, 28 Dec 2020 20:44:34 -0800 (PST) Received: from localhost ([::1]:36514 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ku6s4-0008Hj-SF for importer@patchew.org; Mon, 28 Dec 2020 23:44:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38556) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6p1-0004gp-Oe for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:23 -0500 Received: from mail-oln040092255030.outbound.protection.outlook.com ([40.92.255.30]:6184 helo=APC01-HK2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6oy-0004Tu-8x for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:23 -0500 Received: from SG2APC01FT025.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::4c) by SG2APC01HT192.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::412) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27; Tue, 29 Dec 2020 04:41:14 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2a01:111:e400:7ebd::44) by SG2APC01FT025.mail.protection.outlook.com (2a01:111:e400:7ebd::187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:13 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e]) by ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e%8]) with mapi id 15.20.3700.031; Tue, 29 Dec 2020 04:41:13 +0000 Received: from pc (2001:250:fe01:130:6d26:2348:77a5:f7ed) by TYAPR01CA0056.jpnprd01.prod.outlook.com (2603:1096:404:2b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M0eX++1TzYrlUN9f2+QJ9rxp9cG8NS3pB4muug/WPCqM5chISDQ8ZzZ3cIWyqkH72KLkNAAe2tiQq6riXJIMEVOK5Y7oHOTKLAucz0oPanKO2/7rNG9Y8x6CYGHPlVxkKXeGLia3sx6dPcDjDEFPcULWSQdVHLiesHPk7AbKogEs0gVY6g4aFYFJytP6iDjUZsNXvZIUKsMb2Ult/Od+dQWqJs6Z6Bd9R4cJ8VXIJ/4Tjv8fWe6xIOiglQVNmy+SP5hqIaudhAxYqQR6fB2FjDK/OXzGY98dhePMCGNN2llomTiCt8uXt9K5vPBlkzBidoBgMV4xD5JtLREV2ACE6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D7U3CWaqO/4BtDRpqyPH43l5mb6QPlr2k1azwmdY/YQ=; b=HK4P+LtR+ALh8uS5WhVwgt5TOWdUhv7CQizZESX/v9Nx7nIetMzyJ9XNsmnWeJQb/sEtyXoEX2oKCJHr5vBcUdgRTEh01gKOCudHRFUlEFae840gbablzsYEq8o+oFQesNeKTcIPKUstXb4qPg0Iz8cU5up1OD0/5CtYAdvoo44yyoJxHCcUEGT5eyYerfp7RHr9D6i7/8leGG2GPu6yvn8edvMl4+j0p9LmBu4a7WdKpz17MSHrU4G5KOvNK4ZZvMzMsK4Tu6WO0Bd0f8mvNdFheZH7awMUExADT4KB1wNJVkU3mf0xV9yGOlFIncxj8I7aAk8985/nyj1JMgAa4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D7U3CWaqO/4BtDRpqyPH43l5mb6QPlr2k1azwmdY/YQ=; b=VJsOzaBOQ7OhjYENjAZlPoyO34ESVr++YD258HaFrWfD2ulEBHdimxakMTGB54n4q2f22PEQVdpfwwPBkGKegcays6dBUGvz/hVi16DBSFnp47rRFTJ6qV95DV7XUIW7Z0aGKNgImDUBjJhFzEtUxUwwfSS9cX/MstWZXZdL8Q6v3VSW6IJ168tALK5G/O7GuKNf7RBBMOon0FzDY4aTZn3aR/sNwp+vxdcnDg+4ivkXTTKHOWkw/A/GMYfTAPoVRikCJ+K8zN3MbieRUUG2I+cTvpnLXLTxb08sR+YDaOJfjQHSbLOqsiAZ8F0fNDioGND2bI0AaRmQjbVCCBPxzQ== X-IncomingTopHeaderMarker: OriginalChecksum:BF9F7790F66757943C7B13BC89A77621CD311CA219F62A9C272302821F9AC615; UpperCasedChecksum:511A8F647AB433A9932D61DF2D47B8926BAACEF20D5A2256253B62F9D31FAA78; SizeAsReceived:7695; Count:47 From: Qiuhao Li To: alxndr@bu.edu, qemu-devel@nongnu.org Subject: [PATCH v4 3/7] fuzz: split write operand using binary approach Date: Tue, 29 Dec 2020 12:40:40 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-TMN: [5WJ3GKPgRY8x9wIWFnWzRuZUXzSns184af6qHWZ0+wOM2zVQ9AQ+eGpHLiDee4K7] X-ClientProxiedBy: TYAPR01CA0056.jpnprd01.prod.outlook.com (2603:1096:404:2b::20) To ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:ac::12) X-Microsoft-Original-Message-ID: <20201229044044.3706227-3-Qiuhao.Li@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 371997fe-960d-46d9-2c38-08d8abb3f912 X-MS-Exchange-SLBlob-MailProps: S/btQ8cKWiRaJ+ox6yOiFxhBa7ZFYMmiF1Yyh28+MKsrj98hRx6bocrdkNxlG3L97dNpRZFoRssMRndBTQuSigFuW9B8nw8W2n37G6dOrRKBcqZe8/uViNeZQPiWQ94jdQUfYuxWEZ4dPt4puzTxSr3Ng0yEEMduNZtz33XK0I5T/2ZIwheY81Ax99JRo7uY+7/nAwynRcQ6XN77xUN3mFQ2QO3i7SVgIxb5/H8/SFu5yxa/FLk3/JUaS7SkLbbDQ4OiMyb0I+DA1PAJcBEtpDAIdkow/Bo4xybE86ySIkPc+5vjfhOw+Mz7HNXLVjuoG/4bc1E8pxAx975rd+bK4Md07K5queDEILn0zrtitwkWvrY1MgkwLXdGVuWNjXhDE6vq3zDk3gkbr2GA3+Oy3DetfKc5RAwsFLTFhl3+Fu5GLvcdbPM0IiM1s+0TY9RWtuugllWNtIPpMNG5+EyiB+JRpb0UaSpzicOH3njqWbgSjDVyM0TPF0a5Jp/DF1S0adAb0/EUhi+EPZLhh/iYY4fAp9mIeeomyHWGWqzOMcemzpr0lqAbBfPdJcd34gbENRDS4bbMFLPnfzDqE0PN8XL+i42qW42PScnovqdF1ptxkR6efVDyDJK8RiNCKoEGCSJpi0XWsn0ToFAbcqr87g7kf7oaIPqGMGAu8ovASmS6G+58cMLTAvYuA1SSL6c8fpTooXu3btUU84NL9WjHmdeo/EVyWzS1VPinNiSK7H/NfBeoKPIjJfDzTZf4mJNnuYvl/g8O1GE= X-MS-TrafficTypeDiagnostic: SG2APC01HT192: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: u5XUQ4dtAxdrq9teX7pxo5v5/lpBUz93jZ2EcPKmrgzJvWC0qk0vRhbIkyNnCtN5IZwezaAt4TaopCM4To+4efO682vnpmtEBBsIkO/qolHBhZ8YMnDLCNtudwiQXdmAqikbAvJG/k4GSYdxoDFF1pD8NeJ4FWRbSi1KUnWbkgFOQc+1hatlA54sGL+tZ3ilWKR+slvTYXPjIbCfsDDvR0bDc9tX3Li2gqsKtsZ/A+EhBwT/xtbMrP3qbhjafFJl X-MS-Exchange-AntiSpam-MessageData: Vy0X4ACAO9YUFCcl7Faxxe4fpzNh3BgCX+BzF3XzQOAqpjqGcn2+vPlkYDeF5F8FkGh06Yvki2wJtPZld8T6ykzkM5iC4CpprFrrBKWfSGMpvsspgjadLPyC8wZb3/KoP0atScVps7Dlf7SSeo7+OBRDv6CL38NXXFJXEDlKm0qU0WJ2rnwB+/08iIdJEB2vTQl61pvGKv60CgpyHrbNgg== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2020 04:41:13.7724 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-Network-Message-Id: 371997fe-960d-46d9-2c38-08d8abb3f912 X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT025.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT192 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.92.255.30; envelope-from=Qiuhao.Li@outlook.com; helo=APC01-HK2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, Qiuhao Li , darren.kenny@oracle.com, bsd@redhat.com, stefanha@redhat.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @outlook.com) Content-Type: text/plain; charset="utf-8" Currently, we split the write commands' data from the middle. If it does not work, try to move the pivot left by one byte and retry until there is no space. But, this method has two flaws: 1. It may fail to trim all unnecessary bytes on the right side. For example, there is an IO write command: write addr uuxxxxuu u is the unnecessary byte for the crash. Unlike ram write commands, in most case, a split IO write won't trigger the same crash, So if we split from the middle, we will get: write addr uu (will be removed in next round) write addr xxxxuu For xxxxuu, since split it from the middle and retry to the leftmost byte won't get the same crash, we will be stopped from removing the last two bytes. 2. The algorithm complexity is O(n) since we move the pivot byte by byte. To solve the first issue, we can try a symmetrical position on the right if we fail on the left. As for the second issue, instead moving by one byte, we can approach the boundary exponentially, achieving O(log(n)). Give an example: xxxxuu len=3D6 + | + xxx,xuu 6/2=3D3 fail + +--------------+-------------+ | | + + xx,xxuu 6/2^2=3D1 fail xxxxu,u 6-1=3D5 success + + +------------------+----+ | | | +-------------+ u removed + + xx,xxu 5/2=3D2 fail xxxx,u 6-2=3D4 success + | +-----------+ u removed In some rare case, this algorithm will fail to trim all unnecessary bytes: xxxxxxxxxuxxxxxx xxxxxxxx-xuxxxxxx Fail xxxx-xxxxxuxxxxxx Fail xxxxxxxxxuxx-xxxx Fail ... I think the trade-off is worth it. Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov --- scripts/oss-fuzz/minimize_qtest_trace.py | 29 ++++++++++++++++-------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/mi= nimize_qtest_trace.py index 0b665ae657..1a26bf5b93 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -94,7 +94,7 @@ def minimize_trace(inpath, outpath): prior =3D newtrace[i:i+remove_step] for j in range(i, i+remove_step): newtrace[j] =3D "" - print("Removing {lines} ...".format(lines=3Dprior)) + print("Removing {lines} ...\n".format(lines=3Dprior)) if check_if_trace_crashes(newtrace, outpath): i +=3D remove_step # Double the number of lines to remove for next round @@ -107,9 +107,11 @@ def minimize_trace(inpath, outpath): remove_step =3D 1 continue newtrace[i] =3D prior[0] # remove_step =3D 1 + # 2.) Try to replace write{bwlq} commands with a write addr, len # command. Since this can require swapping endianness, try both LE= and # BE options. We do this, so we can "trim" the writes in (3) + if (newtrace[i].startswith("write") and not newtrace[i].startswith("write ")): suffix =3D newtrace[i].split()[0][-1] @@ -130,11 +132,15 @@ def minimize_trace(inpath, outpath): newtrace[i] =3D prior[0] =20 # 3.) If it is a qtest write command: write addr len data, try to = split - # it into two separate write commands. If splitting the write down= the - # middle does not work, try to move the pivot "left" and retry, un= til - # there is no space left. The idea is to prune unneccessary bytes = from - # long writes, while accommodating arbitrary MemoryRegion access s= izes - # and alignments. + # it into two separate write commands. If splitting the data opera= nd + # from length/2^n bytes to the left does not work, try to move the= pivot + # to the right side, then add one to n, until length/2^n =3D=3D 0.= The idea + # is to prune unneccessary bytes from long writes, while accommoda= ting + # arbitrary MemoryRegion access sizes and alignments. + + # This algorithm will fail under some rare situations. + # e.g., xxxxxxxxxuxxxxxx (u is the unnecessary byte) + if newtrace[i].startswith("write "): addr =3D int(newtrace[i].split()[1], 16) length =3D int(newtrace[i].split()[2], 16) @@ -143,6 +149,7 @@ def minimize_trace(inpath, outpath): leftlength =3D int(length/2) rightlength =3D length - leftlength newtrace.insert(i+1, "") + power =3D 1 while leftlength > 0: newtrace[i] =3D "write {addr} {size} 0x{data}\n".forma= t( addr=3Dhex(addr), @@ -154,9 +161,13 @@ def minimize_trace(inpath, outpath): data=3Ddata[leftlength*2:]) if check_if_trace_crashes(newtrace, outpath): break - else: - leftlength -=3D 1 - rightlength +=3D 1 + # move the pivot to right side + if leftlength < rightlength: + rightlength, leftlength =3D leftlength, rightlength + continue + power +=3D 1 + leftlength =3D int(length/pow(2, power)) + rightlength =3D length - leftlength if check_if_trace_crashes(newtrace, outpath): i -=3D 1 else: --=20 2.25.1 From nobody Tue Nov 18 07:44:08 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass(p=none dis=none) header.from=outlook.com ARC-Seal: i=2; a=rsa-sha256; t=1609217070; cv=pass; d=zohomail.com; s=zohoarc; b=LeIoxQLAljIdiZ8Ys1Fu/bc1npzOEhOXfUWgUi0H3SMEXYVyrKzcN24k702kL29MR56+agN1nBAHfO0Niq+O+z1f91jca+m0ADIWuAua0lJ7SCALzl8bwmPVkEMF1ryH8PWltaLrcMDYfOkKbtsGE1sJCevNb7G5vVPwrcj5D9o= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1609217070; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wK3n7Uk3ZhGJC8cCCDEWGObNFbGcVgU5QZ8HMxsoMdY=; b=L+sK/MgaaxzZjJ8SlZGAAAtG5YPpTP8AMIw7yictvQv3FweNxAwraez9g0mo+o2alIP+T2TT6/YNeixAGaJPzmXpifzTEQcnsjwgwfwmpkA270+t5S0apnS9OVDynEh1w/csKIDyQtEaSdOLilW95X7oI32QqntVRJBjVmZBJrs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1609217070018578.8633888451753; Mon, 28 Dec 2020 20:44:30 -0800 (PST) Received: from localhost ([::1]:36184 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ku6s0-00089k-Rr for importer@patchew.org; Mon, 28 Dec 2020 23:44:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38570) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6p4-0004if-Hb for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:27 -0500 Received: from mail-oln040092255030.outbound.protection.outlook.com ([40.92.255.30]:6184 helo=APC01-HK2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6p2-0004Tu-EL for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:26 -0500 Received: from SG2APC01FT025.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::4c) by SG2APC01HT192.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::412) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27; Tue, 29 Dec 2020 04:41:20 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2a01:111:e400:7ebd::44) by SG2APC01FT025.mail.protection.outlook.com (2a01:111:e400:7ebd::187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:20 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e]) by ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e%8]) with mapi id 15.20.3700.031; Tue, 29 Dec 2020 04:41:19 +0000 Received: from pc (2001:250:fe01:130:6d26:2348:77a5:f7ed) by TYAPR01CA0055.jpnprd01.prod.outlook.com (2603:1096:404:2b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:19 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kDHgLxL3jMoqXG/fAodFxfRvPz7bCqbxdJAqUMdshjEPt+Yi3N/jtsaFRrPJCbIK0PF529Dwx+tr69HWFD4sge0pMK19Dgbv4VVdzM0+wHkj6ueJqh+IMfeh96x54g0crBToiS0MJ8Ca1RP3G+ON9p3fal3CRRijayYbDgaSj2R2e4OgPe4cpoQ8f/+QR316S4GEkpuH+fuex+JaI2QfUnTdGUrODhVamowaJCaUuA+m/0DLvafWIXjke3kpMx8rfNvld0UGqcZMi/9u3/OWHZn9DwLd+8+dW+aM3THPy1CHdv602/q/Y9IBxT/xQWynE2hgoluyYPk6Up108gv3Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wK3n7Uk3ZhGJC8cCCDEWGObNFbGcVgU5QZ8HMxsoMdY=; b=AncWUKMVcKRmao77+CCfZK0QvqDxikJUyYpk08SYEfMT0MkVuUzSMl1RClCyiEa/D0Lf6PmojaROkHmwz6zeqo4z/n64CuiDRjwrPI0cDH/muk1F5rbpMeAlH5VKj0ikG8v6GL9oBuyz5+gIVg19ucmXp/IYGjj2JFTKhFimCZNuMrVxN4Tv79PvK+TWEq69U+Tq9gqwOrE/3TtxPZDxNJ55YcnTEA/Pcy5VMo4p8tckc0tTma79Qz0SpRN16qzJIu9f4Yhv9So4BU+tTA67BDPwpkr6ettwmYFefWprCaiL6XvOl82IKubXiFS0uvdr7qXF/4WIVGBj0wTS/PrMmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wK3n7Uk3ZhGJC8cCCDEWGObNFbGcVgU5QZ8HMxsoMdY=; b=GP0bi3IQQ5KveIEQcO5IRSpozYGmmXyyNWyEtPKRj0vlJfGUZ0tk8KcDOmJcts1t1MdB2SQoAvRPWiJkv3iebStJ431+seHVW9njOxV34cHIt3F2lZK5q7Glk4Pg4iL7nvU7KcQDx5ntkuvWgIV4iaUEjyf3ZXaxM22dVLyAf1/Z2Y757zF8A95Dst9mNO0gx2FieihRvSuDPfalzMCekaRySdarjRe0iRPKmu0cTYnoaQHh3+4RgID+FzS4XvAe4BqaZrFTAKqjngGpRZPsY+kYjU7EAH8v9v0H7+ZruA5VOBAQKP8+Ff4qOx06GozGbI4ag4Io2P7QTC3p9lcB0w== X-IncomingTopHeaderMarker: OriginalChecksum:57CC17EF41B0F21F7D88F38DE07C8098DF364D95F9724379764449ED6832E544; UpperCasedChecksum:520FC871A88D032A8A8C56736FEF2E3ECDD3644C5FA795694364AFEC09CFB131; SizeAsReceived:7660; Count:47 From: Qiuhao Li To: alxndr@bu.edu, qemu-devel@nongnu.org Subject: [PATCH v4 4/7] fuzz: loop the remove minimizer and refactoring Date: Tue, 29 Dec 2020 12:40:41 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-TMN: [8Q6U9TTSscVdXx6KBmXED5orJf2Huuc5HHP08t5gr6bHnSUSYxdEVqo8bagF+iYR] X-ClientProxiedBy: TYAPR01CA0055.jpnprd01.prod.outlook.com (2603:1096:404:2b::19) To ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:ac::12) X-Microsoft-Original-Message-ID: <20201229044044.3706227-4-Qiuhao.Li@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: bc5fe06d-83d0-493d-1a72-08d8abb3fcb2 X-MS-Exchange-SLBlob-MailProps: S/btQ8cKWiRaJ+ox6yOiFxhBa7ZFYMmiF1Yyh28+MKsrj98hRx6boQ9+6tvhCh+vkL+7D7u+8xSNToZcgEruP6l4r4EUBMeF0zZs12oeFhDJ4KLZ3ALeUapB0nNONbOnqNSBhmMTybpPAshXwpqRVMdi8dqDppW3bb+QoE9OpeN05Ws5h+mdfFUpBdSVsPYD+n4ypCxZgC8DyPiF0sMLbBNS6xIbsOLFQCeNwEGu28Kzg6rV96mSO2X9WHw6VkzeFLUvJFhJCLvGZmrnOLovpRbqHiiroycjvfi6LbnsO6QpUFklQhqDhPUwNlviRUN9lmJg79Gh0QsjtSzqGbHrc0qFQ1Lhz3BqD9AO0Y9bhle+tOs8Yicsd+J6CygXJE++3W9jNObAhrvJDgrzrSNt98rVWqCjdppnc+0EYReJFdxRI8i9Mbe+1gr6c3hlemZbzqC7HrF9bD00k4z/Ldsd7V8D4c8g4F1JYd2YZTRtmojQHcyY4KGCspQbzOitXeE61/ce4pqFDtf95iPKJaopU2Nb/PCqDYGztt4cwns61daw3x7qIj6e40p2DfMvcB52h1QqgZ0HwY4jw5Ai/vYFKxAFfwqgIG3dGo6AQSXNwyOQbJlUiFRCb3YZ6yWmdedr8SPvNrlXtt3d+7cwCTqPjP67amOYOCW9mTtx4/Iu1hiszOqhk8Ahg2PTda+kOnmaps4nboLgm/4vYS26ygXba+WalEfzBiK5yH2oPOzMyDGNRtpy8mui3oDJy68QtZ5xmgBiLBJ/e8E= X-MS-TrafficTypeDiagnostic: SG2APC01HT192: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ticQJBsA9zHiBevY73rEpfJiOBK9svsfvzqBJNfXlDNnuL9cvALHdJbxtsZHR0Fg0cNTA5KpByvfHJ3Y55bBo6ERrkBvtHzUaWe3++wyYrJJlAZQTe17TwGM1VTKb01vT4qrDfE+z1kMHL8i9cqaev+GWqFoPTqs6bUuGgUZr3drxY4W+KaO8IQSpVVNamxC8ShKBu2I/IW+tBguZDy6MjVxiks8V9/KxoQLk7aYG4gCh9BQm/gTEP3GKGeh2UaR X-MS-Exchange-AntiSpam-MessageData: E6ItLZdRXceOBRc9pupIJ9p437Vx7p/aNYfMc0sXIf2/3K/XkZEMgh/KKTt9avh+k+XuMz2bIBewM7GKqxOB8vzL5QY33zhP4AtdTrfWu2hCdJMsi3c/MlWXooSqKiAK/ngVsW7KzSv75YwTkWsTncENasVdOI4FQcW+9CE7O0HFHFemLgEuAr5Gz5A9+oHqc4GF8vxSTHF/MZXzx9sv4Q== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2020 04:41:19.9214 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-Network-Message-Id: bc5fe06d-83d0-493d-1a72-08d8abb3fcb2 X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT025.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT192 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.92.255.30; envelope-from=Qiuhao.Li@outlook.com; helo=APC01-HK2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, Qiuhao Li , darren.kenny@oracle.com, bsd@redhat.com, stefanha@redhat.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @outlook.com) Content-Type: text/plain; charset="utf-8" Now we use a one-time scan and remove strategy in the remval minimizer, which is not suitable for timing dependent instructions. For example, instruction A will indicate an address where the config chunk locates, and instruction B will make the configuration active. If we have the following instruction sequence: ... A1 B1 A2 B2 ... A2 and B2 are the actual instructions that trigger the bug. If we scan from top to bottom, after we remove A1, the behavior of B1 might be unknowable, including not to crash the program. But we will successfully remove B1 later cause A2 and B2 will crash the process anyway: ... A1 A2 B2 ... Now one more trimming will remove A1. In the perfect case, we would need to be able to remove A and B (or C!) at the same time. But for now, let's just add a loop around the minimizer. Since we only remove instructions, this iterative algorithm is converging. Tested with Bug 1908062. Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov --- scripts/oss-fuzz/minimize_qtest_trace.py | 41 +++++++++++++++--------- 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/mi= nimize_qtest_trace.py index 1a26bf5b93..378a7ccec6 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -71,21 +71,9 @@ def check_if_trace_crashes(trace, path): return False =20 =20 -def minimize_trace(inpath, outpath): - global TIMEOUT - with open(inpath) as f: - trace =3D f.readlines() - start =3D time.time() - if not check_if_trace_crashes(trace, outpath): - sys.exit("The input qtest trace didn't cause a crash...") - end =3D time.time() - print("Crashed in {} seconds".format(end-start)) - TIMEOUT =3D (end-start)*5 - print("Setting the timeout for {} seconds".format(TIMEOUT)) - - i =3D 0 - newtrace =3D trace[:] +def remove_minimizer(newtrace, outpath): remove_step =3D 1 + i =3D 0 while i < len(newtrace): # 1.) Try to remove lines completely and reproduce the crash. # If it works, we're done. @@ -174,7 +162,30 @@ def minimize_trace(inpath, outpath): newtrace[i] =3D prior[0] del newtrace[i+1] i +=3D 1 - check_if_trace_crashes(newtrace, outpath) + + +def minimize_trace(inpath, outpath): + global TIMEOUT + with open(inpath) as f: + trace =3D f.readlines() + start =3D time.time() + if not check_if_trace_crashes(trace, outpath): + sys.exit("The input qtest trace didn't cause a crash...") + end =3D time.time() + print("Crashed in {} seconds".format(end-start)) + TIMEOUT =3D (end-start)*5 + print("Setting the timeout for {} seconds".format(TIMEOUT)) + + newtrace =3D trace[:] + + # remove minimizer + old_len =3D len(newtrace) + 1 + while(old_len > len(newtrace)): + old_len =3D len(newtrace) + remove_minimizer(newtrace, outpath) + newtrace =3D list(filter(lambda s: s !=3D "", newtrace)) + + assert(check_if_trace_crashes(newtrace, outpath)) =20 =20 if __name__ =3D=3D '__main__': --=20 2.25.1 From nobody Tue Nov 18 07:44:08 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass(p=none dis=none) header.from=outlook.com ARC-Seal: i=2; a=rsa-sha256; t=1609217229; cv=pass; d=zohomail.com; s=zohoarc; b=PonL0oa7mZCCzJlEIFt6APKc5o4xisOIptf12cx7rZsfnO13xvztAn6N45Uyz/yux0Ffi14JnVI+cLsV8jAEoIc2rm2n3/W/IaYDfRm6pWOCIoA9k6MHhPrY9CDmpKZuTPc67wXB8ATPPU6f47/gs42OY2biQCL9a3w4WDSta8Q= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1609217229; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FR4fikGdSjjd1F3q5G7KKkCF1l0dJlVKYYIfV+Qb6x0=; b=m7JKR2JVwMnVkNQCfPN98r9aCYG+M5fchNixBAkvDB3bGzBclo4QlR8SBoY923od5Td/+vYhUoC7S5NgDXIUKBTN/afyt7NKa3pOuiRf0OEZ8qGAMo89UQLSXVOOmaUE+GPwlG4/nUIo3opj9Ow23uOh8D9tbptytexp+dcQmq8= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1609217229933601.8150914475099; Mon, 28 Dec 2020 20:47:09 -0800 (PST) Received: from localhost ([::1]:42464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ku6ua-0002Pt-J6 for importer@patchew.org; Mon, 28 Dec 2020 23:47:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38640) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6pM-00055K-Ii for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:44 -0500 Received: from mail-oln040092255075.outbound.protection.outlook.com ([40.92.255.75]:48375 helo=APC01-HK2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6pK-0004dn-Pd for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:44 -0500 Received: from SG2APC01FT025.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::52) by SG2APC01HT187.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::273) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27; Tue, 29 Dec 2020 04:41:26 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2a01:111:e400:7ebd::44) by SG2APC01FT025.mail.protection.outlook.com (2a01:111:e400:7ebd::187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:26 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e]) by ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e%8]) with mapi id 15.20.3700.031; Tue, 29 Dec 2020 04:41:26 +0000 Received: from pc (2001:250:fe01:130:6d26:2348:77a5:f7ed) by TYAPR01CA0064.jpnprd01.prod.outlook.com (2603:1096:404:2b::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:25 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N6nUw1Ztq/VkQwGY70Tl7MRUILzjmIcMHGsBblp22oz8n9exdAvPhq8kAzj7rcBk94zDMkg3taNYI76c4h0QBxSFDJkCRwDKtT19IH4vgY10qWiCebJ65kdRVrWarsya1zW0/6O2pkWc+whyB+VqwHKinzSQ3C1z70L9uiGl7cNyAf1haAmEZHj7XjijiygfE8VeXr4wA+d4w2Mk+iHQ4uOdwf1yDarpN0Prqwu0QatAcAYX7ADsb3ezIQePI/2zkyMPFRpKm5epgym2O9PWjE3EVr87RAsC/ejoHkfaLWlZSLdHvHx/GDfIvFGKOpPyrcZPdsrmN9ili7fFEHllfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FR4fikGdSjjd1F3q5G7KKkCF1l0dJlVKYYIfV+Qb6x0=; b=AKa1w6GlbtFvfVSr7bSko/WFpcuDO8OP2030jYcfhfkOt4pZfStF/fYSrriHcWZ+/4zofgXiIr0ebZ0PrTFlt/V8UijK5aJh1GdbsgFFufeE525lkfXqCCX3MFAB1S3wCgRNG5v5bXD6V+3+f8W2P8mM57L2RX+CyUM6XqNsOcMgf72EYbY4fi8W7nFB72OiVVXNv0bdSEo+BFYxrWS2bkUZHeI2LzNjm88Tc6S4kmjNXCq1KZTuw82oMWebgVVF77PzTUVE2u6Ky3xX9XB57b/pj8DE4lHHSkXobbkqo92gd1dzLA1LKWrbZTNr2udaqf7vRw8ZlxU1LdkYzLj1AQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FR4fikGdSjjd1F3q5G7KKkCF1l0dJlVKYYIfV+Qb6x0=; b=lt/zoaznzD2Au/7UQ0fH3F5kWm4oeiF+a8f9z8VEnxt6k6ah3oE7Ne4RU/dyT28w3zl3/m2PpyuwurYTBrunJxJcrX2zePFK/KhbmHnQ9UQG4bEeFey4/iPp/nfXZUYRwo2uru5FInaqXSW63GSsMlTcFJcYgv7UAXgcJIAliOjRqJR7HdTVpa4Bj0hZOkbsDEdeGfTJZWBDxQ1SdWs8Q8+rSlyYVrIxiBhTEs1Qt823z4535w87/e5Xm/htv80zJgmWkITtaStj8aYl0p3a2yA8+O3gJDG4ia0vfVQRzXQb5cNiBZwPZuICJkqzhLlGEerNY7bfbXjceE1quSMtDQ== X-IncomingTopHeaderMarker: OriginalChecksum:E0B42EFCD334BEC375BBEFDC4E0E23EA8AE852CB08D43F203D40EAB2CBECFAC3; UpperCasedChecksum:2CEB56DB018DD839F833D7550D94F06DBB1130E7E1CA4BFD17E01F7DAE0FD2C4; SizeAsReceived:7659; Count:47 From: Qiuhao Li To: alxndr@bu.edu, qemu-devel@nongnu.org Subject: [PATCH v4 5/7] fuzz: set bits in operand of write/out to zero Date: Tue, 29 Dec 2020 12:40:42 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-TMN: [0blPyXVQSS3Q1TGsf33DstTPPHH4DysVAlWagDI31I6l3RCZiHtbmpwJAba0BkCx] X-ClientProxiedBy: TYAPR01CA0064.jpnprd01.prod.outlook.com (2603:1096:404:2b::28) To ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:ac::12) X-Microsoft-Original-Message-ID: <20201229044044.3706227-5-Qiuhao.Li@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 1e54e6aa-c9e7-4037-204f-08d8abb400c4 X-MS-Exchange-SLBlob-MailProps: S/btQ8cKWiRaJ+ox6yOiFxhBa7ZFYMmilm+7+9bsz0UaWozymeyKgR2ktsuU+YRo/S4MpoiTTmwv1izw7aoQxydy+L49Gm2fNSEjCt2pL6I2na5MEYoXcz0tFJmcKs1uYlZZReR5dfMq+HGcb6XlGpAeZIuuclX7F7Ap6t8bxgXRsjLxuIpcBB4Bwv4uZZ702eZj9YloSzf4mCoMPLw02l3n001TZ6LBlmp3fdWL70FVJyDKVE2Wl9NLAVQfdvOYQ4KXcXFYJtMq2PDeIUedl3i7Dfx73zF/ESSZaw/+SfJcL5MrsIpRwzGX9RAAxcuRhqE6Sk4L5Z/ChTaQc7oYElCsYBTLPPJcbLprwCJl6XF35DQFTR06DR3wYhC07wBnFIcTlChHi07xjxOxvwdf74xxpehUavn4dknzySHfJlxiHGG0QfQyUudLpBG5w1WtBVw5kZUjJTJhZzmMy9wLd1sA9mexFSHfJW08HAAeDNURj6aUpmmuJYjYFIyoE3/pXbmFlLKhTIjaGr9PeNqzdVr8g6fQLy1lbfQSv+qKMW+zPkj0rEormNLAdTMnU5LM1G8rLtTVqvZ2B87P7y2JXnZ6bXCysNByeXtDtK29zNFsRSba2noUGqjZR8xiUFGTcCqAJPTEVM7fDLrPi4WpgIGknDTJePxyvDXa3w3U4AWRQuosRlums/j5Z/WU6QieJ7h0/RWzgMKEhc/xOjSB7SqWr2Kodkk0n4yIPA3gdE8QgZWJnWR978MYAhDa5IywhidCJ93GpEM= X-MS-TrafficTypeDiagnostic: SG2APC01HT187: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2vhi6XvKSrE25U3cKBVLx8AQXlXVP1QIJMxGeF95mQ8uSyU4ssoAp4AUVc5Bco3AaeYu1UXlsay+x73hT0KySyy4iJ1mcEbHX7LbXqN4pxD4ZbWNMhNTtigTfZBgDOGwdaHS/EuTzDltqqkG5qkchm2SYDF15GA7TxzVsgdE9jgjap/UN0ypqws4pWGqZ7nniNTjCpwPC2xyQCyZj1Ef5Nj2riLtVoyJEaASxC9dt3FtE5keRjara9HY/CX6xA8r X-MS-Exchange-AntiSpam-MessageData: JCaE2in04+0+yvfv24P2Lf+7LYXQAXYgu6hMiylpSkioDjdoCFIEU9eOPh+gDqY1krzyG/29moXRdS/pgoPgIOSROHjduDhnYs/i2lNpyOcyZQkyuEwBM0xyvYwpuw0glqx7Zc3V8IU/WMyS/Qe6LWTWHhDW5IIA6kpzbJICC52GZb7ntuFzzHicSidLr642Ne0NPz0LNmk+WpJCl/3Zeg== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2020 04:41:26.7265 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-Network-Message-Id: 1e54e6aa-c9e7-4037-204f-08d8abb400c4 X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT025.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT187 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.92.255.75; envelope-from=Qiuhao.Li@outlook.com; helo=APC01-HK2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, Qiuhao Li , darren.kenny@oracle.com, bsd@redhat.com, stefanha@redhat.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @outlook.com) Content-Type: text/plain; charset="utf-8" Simplifying the crash cases by opportunistically setting bits in operands of out/write to zero may help to debug, since usually bit one means turn on or trigger a function while zero is the default turn-off setting. Tested Bug 1908062. Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov --- scripts/oss-fuzz/minimize_qtest_trace.py | 39 ++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/mi= nimize_qtest_trace.py index 378a7ccec6..70ac0c5366 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -164,6 +164,42 @@ def remove_minimizer(newtrace, outpath): i +=3D 1 =20 =20 +def set_zero_minimizer(newtrace, outpath): + # try setting bits in operands of out/write to zero + i =3D 0 + while i < len(newtrace): + if (not newtrace[i].startswith("write ") and not + newtrace[i].startswith("out")): + i +=3D 1 + continue + # write ADDR SIZE DATA + # outx ADDR VALUE + print("\nzero setting bits: {}".format(newtrace[i])) + + prefix =3D " ".join(newtrace[i].split()[:-1]) + data =3D newtrace[i].split()[-1] + data_bin =3D bin(int(data, 16)) + data_bin_list =3D list(data_bin) + + for j in range(2, len(data_bin_list)): + prior =3D newtrace[i] + if (data_bin_list[j] =3D=3D '1'): + data_bin_list[j] =3D '0' + data_try =3D hex(int("".join(data_bin_list), 2)) + # It seems qtest only accepts padded hex-values. + if len(data_try) % 2 =3D=3D 1: + data_try =3D data_try[:2] + "0" + data_try[2:-1] + + newtrace[i] =3D "{prefix} {data_try}\n".format( + prefix=3Dprefix, + data_try=3Ddata_try) + + if not check_if_trace_crashes(newtrace, outpath): + data_bin_list[j] =3D '1' + newtrace[i] =3D prior + i +=3D 1 + + def minimize_trace(inpath, outpath): global TIMEOUT with open(inpath) as f: @@ -184,7 +220,10 @@ def minimize_trace(inpath, outpath): old_len =3D len(newtrace) remove_minimizer(newtrace, outpath) newtrace =3D list(filter(lambda s: s !=3D "", newtrace)) + assert(check_if_trace_crashes(newtrace, outpath)) =20 + # set zero minimizer + set_zero_minimizer(newtrace, outpath) assert(check_if_trace_crashes(newtrace, outpath)) =20 =20 --=20 2.25.1 From nobody Tue Nov 18 07:44:08 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass(p=none dis=none) header.from=outlook.com ARC-Seal: i=2; a=rsa-sha256; t=1609217287; cv=pass; d=zohomail.com; s=zohoarc; b=LwwuHbH8u/R1KS3Qel025bfAdwnqz1AK704lin1IllPiUVdSivTDlzzqFghBL49rhwgmZKZqkh4diOoggIJAbQNVbrVyrdY9JQnI5zmFMpsuC5lGrbFc0QUk94u9YuZmEd+YNroiR2sB9nrpf0K0EN4flg2smzmZyIaSFqZjnt4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1609217287; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NxTOO5W1fz4ZgM+M8sX7I4Vv0Pmt8JHxRpdNsForf2o=; b=IsWXsuJCZmGxcpLSQaZ2F4vz4c68VRDE8aBVpTKOhyN3MRZvZQgSsEf7n/ir/E5+ct73ypR8q0z0ZNwgOnbUs9kSjt+eaanRoEppDiNxF90BkaYalhSMt5aNaHRmQ6uafTaWw5pmeN9RkVoCWp7IIduj691spmQYDeW+DGGLnis= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1609217287657969.6454655623377; Mon, 28 Dec 2020 20:48:07 -0800 (PST) Received: from localhost ([::1]:44606 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ku6vW-0003Lw-Mi for importer@patchew.org; Mon, 28 Dec 2020 23:48:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6pP-0005Aa-Qy for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:47 -0500 Received: from mail-oln040092254070.outbound.protection.outlook.com ([40.92.254.70]:5440 helo=APC01-PU1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6pO-0004fk-1L for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:47 -0500 Received: from SG2APC01FT025.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::4e) by SG2APC01HT222.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::432) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27; Tue, 29 Dec 2020 04:41:33 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2a01:111:e400:7ebd::44) by SG2APC01FT025.mail.protection.outlook.com (2a01:111:e400:7ebd::187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:33 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e]) by ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e%8]) with mapi id 15.20.3700.031; Tue, 29 Dec 2020 04:41:33 +0000 Received: from pc (2001:250:fe01:130:6d26:2348:77a5:f7ed) by TYAPR01CA0063.jpnprd01.prod.outlook.com (2603:1096:404:2b::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UwyzLKVbQH6k+YZRTj9WUR61R11yz719TnJBvvuEKxZXtSz7R6JSpIOv3/Mj/+96ypyFfZHclF8hfrB5cD8oBKxH6qZgh9LDfY9xu06zJq+sd2MpZaushFpWdYspiQjuEB9D09wNH8TyzyRtZ0I0vfWOdY1vCeSQXSG3ASf0gg8IRrthGP8sfPIJKzr3h40TwtZNqsFiOwQR6Dakf0vOqCc5p3ZYE3peCTCMuY7iErYIcx/FqbuwZLvzljI8mMPbtcxxZA0fuK+979/bmRNX7ary/iP7fNDCqN7o9fzoP+E06l+BwJcjjvLS49YQ464zP7rF/clkovYd8XxWjM/FKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NxTOO5W1fz4ZgM+M8sX7I4Vv0Pmt8JHxRpdNsForf2o=; b=BwWxh/RIVFoEqKr+l0MyvaV/g1bsZCTW4wJalNsfMF2zpbZPI6MlrgvGGw4UzD4NmXcVyl9CXoWDPfVxz5IcOMM7XGb1pVkkC1UujScNBCjdlCIJrOMsPg3/mH4N2T0IC/0Bokfz05v396uFgkYlMbrkFFGflf/JAgrCgy8/AfS/NvOW681V57GCY7+nfaSTrOhN2Cp0uPFf0Kgg/qUJOtrSxhjB7i0oL11VKWYAOgxzT2L/y8ye8RKF7gbrZ5403JVEjjMHD6i/I6N0hXmzv7Pbbi7d4Pi1lfMiX4aN8soGXrEw7o8I7u5zTGNyysYMAQyT/dtVcK7aIhJy3jZZkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NxTOO5W1fz4ZgM+M8sX7I4Vv0Pmt8JHxRpdNsForf2o=; b=o4uhBVOi/3jdcnzrXwRYXtLCwHl9/buPu1J8qVS+ayX4V8fibr1GgcX/J6Snr2j00kCKTtdn7bGy/IGIN/uyXcr+duNU4WkWpLDGdlHjusmGX8gNH1ZyEN84yS1A2/lkDCLs0Aix9DBfwmiho3lDiwmqA8+FKhGy5RQ3pythN2Z4I48vP4Uht1SA5OWYvUfPX8GA/S9JD2BCmhzuxdJOgD0o4lLMUWVebUilinbPlPlBN/ks43N0y0eriip5BuKcbqfCMWPvvdWPUlUUbAln5IoG5aYmjRz9d2Q0KzTcFnm9gDEoqJ+O1vMnDkqPWCbMNn0s/1Qxz5ISXpfu5Yj07A== X-IncomingTopHeaderMarker: OriginalChecksum:2E38E0D612DAE27D869FACB455ADB821069B540FF8CBA68999FA92234AFAD1FE; UpperCasedChecksum:2AF128A5B12D8E8E5C56C1D4108CA6A4FFC1FE784040E80C54CE99F2BAF62636; SizeAsReceived:7634; Count:47 From: Qiuhao Li To: alxndr@bu.edu, qemu-devel@nongnu.org Subject: [PATCH v4 6/7] fuzz: add minimization options Date: Tue, 29 Dec 2020 12:40:43 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-TMN: [SwRVs+iBt2QY6IKTcH0L7ryAt5O7vRRc3yab6raQOZ2iuHvTot74ABrdxVChDqr4] X-ClientProxiedBy: TYAPR01CA0063.jpnprd01.prod.outlook.com (2603:1096:404:2b::27) To ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:ac::12) X-Microsoft-Original-Message-ID: <20201229044044.3706227-6-Qiuhao.Li@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 90dae100-6835-4d50-8d32-08d8abb404c2 X-MS-Exchange-SLBlob-MailProps: q+fD6XS3/UL0qSVcdtWiVVAkKoUvAbiis6b0itYI0jAVIvDJzTNll7JXAYGjrTskBo+22MjdKnPgmTfCi/am3XE3gqWXGAHfNOdQvs6iqGcp28uvGecIn5CKcmPLPEEjEQYRz6MlxaQGsVeAVHCXBXXabLW8unlT2hogjiqNJJq1+ZaSp9BBlm1LONv0T/hvKw7q5NHYCmqFJOja5pMo4i4LUZ1BuCUG118KES5XHOgg0Ge2yfMI2XRP7a0c+VSIaz47r+bFH/nn4utvNzrfbRwW8aGRbie8vsnlFI8E0fc0xzxUSHDuPH7R2Qbz6vKzug3UV0Q3roEtJEVSJf6zqT+INn3MdMwzPlxg3pziX4ZGNvHc1HP7omKTF8hj+jb3e44enKEMSZsvgwCHvkz1gN85L0+3hJxv6FRySbZPw32O0whnzRpIqo6niQiS21fClu6IgueM6N+NbS2I/RpYITbhzUt7HJ/0CEk2UqpQWHhrfHg16i5i80xotefJRy5F/g8zGGGAUMeZlIg6FRLv3OgpNz3uClG0KnyBp+7IiKBVAS/36ancqh2nTF4bDkXcHJrp79xIvWHbLByy7EXZxiROkY1YCDqBFZvWQOB7jlffS9h9+8fBSfhLJDfVmlwCznNweEFJQ3+Zz5NfuGFiKZgrWLmryqNQD/mus4LYLp0yytD1Y2bqQAuptedhAFo89KxmbPMuw79PE8m+YeC42A== X-MS-TrafficTypeDiagnostic: SG2APC01HT222: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FW+iyqX8Z4Cv35965WpYP5ooL1Mb/MHvq8VK+SZRB/4w6Os5G6maReAQPcIwV7wTy2SdassTbgh1ewK7gCJ8eevoGUR5nIgu5oxElif6dfZP2ADYeZfK75S3BgeOSsPNbWnxa52BFiophUcadcbfA343MPSKlAy9g2K/QytvN9f+jW/r3ERq06iuUf1nouVHsfz9in+aeKqXLCvdJxPBNc+hM1seiTATlHC8SWyiW4kLvE67AvXOcdAYdIbkqHmV X-MS-Exchange-AntiSpam-MessageData: U3kqlMxsud7+7cWryXGNqHuLslKBSfh/zuw1C3W9lOXtsAXX6Bvr+PpQfJej7CT7DcWFYV61i6LfAFPlGGb1DOkjGZF9SS0+DVoZFgQNRIJU9q5IoLSkXoSWkMIhBBa3gb0t6PCygWvRqKQWAJvREugx4/yjGvONvWxwJpmgBUAPVOOjx1FG8aEvLbmz22ufG+wwiRhrhRYyvbbDmqzfGA== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2020 04:41:33.5008 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-Network-Message-Id: 90dae100-6835-4d50-8d32-08d8abb404c2 X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT025.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT222 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.92.254.70; envelope-from=Qiuhao.Li@outlook.com; helo=APC01-PU1-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, Qiuhao Li , darren.kenny@oracle.com, bsd@redhat.com, stefanha@redhat.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @outlook.com) Content-Type: text/plain; charset="utf-8" -M1: loop around the remove minimizer -M2: try setting bits in operand of write/out to zero Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov --- scripts/oss-fuzz/minimize_qtest_trace.py | 32 +++++++++++++++++++----- 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/mi= nimize_qtest_trace.py index 70ac0c5366..a681984076 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -16,6 +16,10 @@ QEMU_PATH =3D None TIMEOUT =3D 5 CRASH_TOKEN =3D None =20 +# Minimization levels +M1 =3D False # loop around the remove minimizer +M2 =3D False # try setting bits in operand of write/out to zero + write_suffix_lookup =3D {"b": (1, "B"), "w": (2, "H"), "l": (4, "L"), @@ -23,10 +27,20 @@ write_suffix_lookup =3D {"b": (1, "B"), =20 def usage(): sys.exit("""\ -Usage: QEMU_PATH=3D"/path/to/qemu" QEMU_ARGS=3D"args" {} input_trace outpu= t_trace +Usage: + +QEMU_PATH=3D"/path/to/qemu" QEMU_ARGS=3D"args" {} [Options] input_trace ou= tput_trace + By default, will try to use the second-to-last line in the output to ident= ify whether the crash occred. Optionally, manually set a string that idenitife= s the crash by setting CRASH_TOKEN=3D + +Options: + +-M1: enable a loop around the remove minimizer, which may help decrease so= me + timing dependant instructions. Off by default. +-M2: try setting bits in operand of write/out to zero. Off by default. + """.format((sys.argv[0]))) =20 deduplication_note =3D """\n\ @@ -213,24 +227,30 @@ def minimize_trace(inpath, outpath): print("Setting the timeout for {} seconds".format(TIMEOUT)) =20 newtrace =3D trace[:] - + global M1, M2 # remove minimizer old_len =3D len(newtrace) + 1 while(old_len > len(newtrace)): old_len =3D len(newtrace) + print("trace lenth =3D ", old_len) remove_minimizer(newtrace, outpath) + if not M1 and not M2: + break newtrace =3D list(filter(lambda s: s !=3D "", newtrace)) assert(check_if_trace_crashes(newtrace, outpath)) =20 - # set zero minimizer - set_zero_minimizer(newtrace, outpath) + if M2: + set_zero_minimizer(newtrace, outpath) assert(check_if_trace_crashes(newtrace, outpath)) =20 =20 if __name__ =3D=3D '__main__': if len(sys.argv) < 3: usage() - + if "-M1" in sys.argv: + M1 =3D True + if "-M2" in sys.argv: + M2 =3D True QEMU_PATH =3D os.getenv("QEMU_PATH") QEMU_ARGS =3D os.getenv("QEMU_ARGS") if QEMU_PATH is None or QEMU_ARGS is None: @@ -239,4 +259,4 @@ if __name__ =3D=3D '__main__': # QEMU_ARGS +=3D " -accel qtest" CRASH_TOKEN =3D os.getenv("CRASH_TOKEN") QEMU_ARGS +=3D " -qtest stdio -monitor none -serial none " - minimize_trace(sys.argv[1], sys.argv[2]) + minimize_trace(sys.argv[-2], sys.argv[-1]) --=20 2.25.1 From nobody Tue Nov 18 07:44:08 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass(p=none dis=none) header.from=outlook.com ARC-Seal: i=2; a=rsa-sha256; t=1609217022; cv=pass; d=zohomail.com; s=zohoarc; b=UIhR11kvSUSciCzhC6yY/sUFksI5BJG2E9TIqybxURZpDbVytQH11ufmS1DX3mUA/y0phso/a4vf/k+a8h87oodZ9uHzyJWITnS4gOWmRF9bScP/3+sXsa2EbHCA1iGRx5TOnkibPQZ/1goJPpIM03TmEbKhdOhRIZSIhWnXhUc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1609217022; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8eam7v6KP/A9AX7ABhV9voi/p1k7ztgEpYwwuTaaksI=; b=W/9oX8q/o0liS1CePZE+98Bs8xx44wfpHXKT2laPhyxTo9sSvjET5yIuPISiO0sOC6LkcKK2JZMlDaS6Q842ru4YJXzFu6zJh9ynlVdt4VGFfiXqj01YC26vLyzLQ18YVUZ7GBXcj5/Mj5MWopyobOpr97f/y1xxRbcgp1lMmUs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=outlook.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1609217022324600.9522559442927; Mon, 28 Dec 2020 20:43:42 -0800 (PST) Received: from localhost ([::1]:60562 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ku6rF-0006aq-AU for importer@patchew.org; Mon, 28 Dec 2020 23:43:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38694) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6pW-0005LH-SU for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:54 -0500 Received: from mail-oln040092255064.outbound.protection.outlook.com ([40.92.255.64]:61472 helo=APC01-HK2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ku6pV-0004kD-2C for qemu-devel@nongnu.org; Mon, 28 Dec 2020 23:41:54 -0500 Received: from SG2APC01FT025.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::41) by SG2APC01HT138.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebd::436) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27; Tue, 29 Dec 2020 04:41:41 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2a01:111:e400:7ebd::44) by SG2APC01FT025.mail.protection.outlook.com (2a01:111:e400:7ebd::187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:41 +0000 Received: from ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e]) by ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM ([fe80::88cb:2262:60f7:7d1e%8]) with mapi id 15.20.3700.031; Tue, 29 Dec 2020 04:41:40 +0000 Received: from pc (2001:250:fe01:130:6d26:2348:77a5:f7ed) by TYAPR01CA0068.jpnprd01.prod.outlook.com (2603:1096:404:2b::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3700.27 via Frontend Transport; Tue, 29 Dec 2020 04:41:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ByszR5Eg/i2UkwyzKiICblDPmm6lUoesLosCa8W1/X/oG5RZ9oWsA3c59pyqceKOnTRZNVIWkn9xIS1TqBWDRBtyvTNl4rrFoCsrhHmcHvU0M/MIuK2feVZe6amF+cH9bgTWTOWnXQExUCYh7Rgr9DmH8Hfxcs4LGFREl3msaUCHFJXE4VDMSTJbVekjVAR3nAbmL/pnpX5kJ45HTPK2BsO1YcOK7Iz8+N3nL6n7G8kxagC7DvuQvKzF2JXpDN7YfgODZI7/bYSNRNRII3Eo4oiB0e1lfDNmNggSs8Ef83mMQ9I0ovEamUfiW8y8mRJfswWES5U19O2R/r08bVbprA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8eam7v6KP/A9AX7ABhV9voi/p1k7ztgEpYwwuTaaksI=; b=C+z1MTViMwcWzAvteyYmH8UQCegGUFkQNG7Ftq/2wQxDDtDapaFddhRJuIYNBJAkqRcRV7cWVI+H5d7Jrhbib6n0f94P9sbAMCv4RnMDmQLOH3hA66T0UMQ2hobc3+OGBMFE8Ewo8drmUSN6+cglJD4ILBwChLETAsMZ38c/u5bLrNXz2EsOl1Y19vzOsh/eLlDxtdo3zrsLe19LoSO0QAsc+yGyNi9HlpTk7OTNgIXOQeBgu0f7CPrCmHBW6wrk+6dBk2do5H08cX6wmIKBKib7W/ezlazzHa3R/ziXmoC3VDu6cLiEK+i72nowK5bxz6kf7rVZ8OwVxDYS7GwzZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8eam7v6KP/A9AX7ABhV9voi/p1k7ztgEpYwwuTaaksI=; b=fJAZ9dD/ruAumsi14fH7NK7PtqBiYB8737sMAjuSHL+Wa/ZtaCRj72A74M38pmTtCqju7BmswQNO5ZM/Hl1eIRo3/yegp09Ap9W8enO17WRlTdqCrdLYEyHzbIaa5hFEKtK00V3XqljjbGA+a41f+fPl/MUcNFA5E3aEiWwv8u98b86KifRiQc3jBVi8ppdB7wuTRf+97tXt513LaZVb/FLqM/UpHzrIy6BKmyCB4rm0JM0wMu2N6jKcI6gGGM9ZlDb7IZivKOcAekc3GZJcjPxyccn4XA0nCDUtkhfWV9Anwf46JGDoRgFTkj01jRTD9E+DlESxNIwG9kZEXv1Vaw== X-IncomingTopHeaderMarker: OriginalChecksum:DB7C933314428AE2670398319F3042D3D6523734E6941B8A7E0F05581CD62494; UpperCasedChecksum:2881717F600188A005C6F66E863A5F96D5A3206B58ABA76F45C5837968C4FCDF; SizeAsReceived:7668; Count:47 From: Qiuhao Li To: alxndr@bu.edu, qemu-devel@nongnu.org Subject: [PATCH v4 7/7] fuzz: heuristic split write based on past IOs Date: Tue, 29 Dec 2020 12:40:44 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-TMN: [nZ0CUZLx7DHSAsKDW6bkwbdwlvwh6/IFrDqEMp7D8dyMGkIKu8BDNjN4lFQzwYAS] X-ClientProxiedBy: TYAPR01CA0068.jpnprd01.prod.outlook.com (2603:1096:404:2b::32) To ME3P282MB1745.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:ac::12) X-Microsoft-Original-Message-ID: <20201229044044.3706227-7-Qiuhao.Li@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 47 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: b778307f-02cd-4ae7-c2fe-08d8abb4091a X-MS-Exchange-SLBlob-MailProps: S/btQ8cKWiRaJ+ox6yOiFxhBa7ZFYMmi4iQ5j3/PO1qZZR1+ERXa7c14KBiBQlvjDnb1ohQ3kuxqR8PbJE4mRYo0HxV2P18Ag6qJQ6DiR4lakRgX8oX1+wSTk1xItVVymBUzO36qDj26KL/yITKqKs5uHBy6yZCWizLnFt6l4UcsVeMZ4JBq5q+gUCD3FBS1Xql71kZ5getXpCTP8m8j8NQ9g/NMKI8qXhZ6HcKb+Dra+vZIG1ykx3B8fGelMtFRLfNtLlvqvxnWA4O9z7z2q2xC6B4+qQGmTc7F2dfElq/nwjinlVJlQ546Ts0twL1wK0mM86vnrBva5qUBDJ//1L5qG2dQBdOasX/BzqjrAbUZTRxLGM6dIbhYddvzD7+FFghckSN1CnT4TIopfd2mtIn3K+xv4dmAW0LAGIrUAaiDeJD+5gngfql2R0gdDq6oHf11bOV9NELHGP80lAMndsLOCK7BUAOaUoGKtfVxcmaqbQluHO5CTV0Ln23+v8PMem9hFh0sypsrmdgFI+g+SSywfVK+WMnWWkxasCSILGKSu7VerJ1LyDkxKZsMrRBRti4soaJPeczv7Qdj/YDJe9IoXtZ63UJaTPLpeZczjI7uCCBHoXZ+ztaW+VPCIH0kLjm5lcxrvsneWFmyUU2hP5d+z7f13f24jdLx+q+qgeD3oYdTK8uS84vciFG5iDWmNq9dmegImTL8VTuEnWJzrdn13+Nj6NqLWYT7mzx30EZPl8WywDuy3RoFFSC84RkISx8VTWBl6yA= X-MS-TrafficTypeDiagnostic: SG2APC01HT138: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8IrLlZwIseZUk8exP7HVTJZBCqVpr10YQOs+Uf7ebApyRvFEtxe3LDvTd+tKq6g4+84wfDHhlU9m0iIXmX8QXgElQpIHPOlxP/eT17+3sok3LBXvFx7yrVyLfPdultUTvO0JoBx44tt973fCHJzM5M9R5LdJ0lo8335jCpkQr24/zbJlf7kjOJl6AxDUkGji9FUB9o2mc/ZFco+sxf3atZ1pc4t7Jata6oHSUL+mntfonhhsuo1VWIL2LQ8ag4F8 X-MS-Exchange-AntiSpam-MessageData: UjK1t+dFmHqq6jgiMjHcBmbQ19pHxxRe8Wgne6BscVGSsXkHfHV6t7/SPveGfDuUQ6m+o/BzRRCEU2c9jkBuwQWMc4hHHiDuKHrMN//m1jpA46gmH3LLfBJSZU0WBjQ005K8AAj8Ht1db6TleBkGgmobU7Q/txEznW6xrnNkOs1S3tsho/p6aQ/k+Q5cBpgb/wnfy1prkDnizWom2jBhAw== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2020 04:41:40.7848 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-Network-Message-Id: b778307f-02cd-4ae7-c2fe-08d8abb4091a X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT025.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2APC01HT138 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.92.255.64; envelope-from=Qiuhao.Li@outlook.com; helo=APC01-HK2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, Qiuhao Li , darren.kenny@oracle.com, bsd@redhat.com, stefanha@redhat.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @outlook.com) Content-Type: text/plain; charset="utf-8" If previous write commands write the same length of data with the same step, we view it as a hint. Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov --- scripts/oss-fuzz/minimize_qtest_trace.py | 56 ++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/mi= nimize_qtest_trace.py index a681984076..6cbf2b0419 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -85,6 +85,43 @@ def check_if_trace_crashes(trace, path): return False =20 =20 +# If previous write commands write the same length of data at the same +# interval, we view it as a hint. +def split_write_hint(newtrace, i): + HINT_LEN =3D 3 # > 2 + if i <=3D(HINT_LEN-1): + return None + + #find previous continuous write traces + k =3D 0 + l =3D i-1 + writes =3D [] + while (k !=3D HINT_LEN and l >=3D 0): + if newtrace[l].startswith("write "): + writes.append(newtrace[l]) + k +=3D 1 + l -=3D 1 + elif newtrace[l] =3D=3D "": + l -=3D 1 + else: + return None + if k !=3D HINT_LEN: + return None + + length =3D int(writes[0].split()[2], 16) + for j in range(1, HINT_LEN): + if length !=3D int(writes[j].split()[2], 16): + return None + + step =3D int(writes[0].split()[1], 16) - int(writes[1].split()[1], 16) + for j in range(1, HINT_LEN-1): + if step !=3D int(writes[j].split()[1], 16) - \ + int(writes[j+1].split()[1], 16): + return None + + return (int(writes[0].split()[1], 16)+step, length) + + def remove_minimizer(newtrace, outpath): remove_step =3D 1 i =3D 0 @@ -148,6 +185,25 @@ def remove_minimizer(newtrace, outpath): length =3D int(newtrace[i].split()[2], 16) data =3D newtrace[i].split()[3][2:] if length > 1: + + # Can we get a hint from previous writes? + hint =3D split_write_hint(newtrace, i) + if hint is not None: + hint_addr =3D hint[0] + hint_len =3D hint[1] + if hint_addr >=3D addr and hint_addr+hint_len <=3D add= r+length: + newtrace[i] =3D "write {addr} {size} 0x{data}\n".f= ormat( + addr=3Dhex(hint_addr), + size=3Dhex(hint_len), + data=3Ddata[(hint_addr-addr)*2:\ + (hint_addr-addr)*2+hint_len*2]) + if check_if_trace_crashes(newtrace, outpath): + # next round + i +=3D 1 + continue + newtrace[i] =3D prior[0] + + # Try splitting it using a binary approach leftlength =3D int(length/2) rightlength =3D length - leftlength newtrace.insert(i+1, "") --=20 2.25.1