From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569973878; cv=none; d=zoho.com; s=zohoarc; b=IPpkIc8h2jFhHC4b6RCmUcMvZkpGtOc64YdvMoM3o5x8SEwR5dWGlwo6PIIDgOjSA+2skVnp2feSAPwXFak+6nEpEV7nKthrjZzLUoO9bKZH2bzbNKdQRhOHVRneAnzMDzF0c9gKZEfc4CHDxdCcCgCy8V17L1mI5XPQbqQ/sz0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569973878; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=L9hcuVJg+qcoGvjFhRCoZwYLAHvwJsgvRBubX3Lzb6g=; b=VGTzVTWrrd1foSX+vQqr5AtpJtKaG08cN3dPfo/UzheKeqpnBddYzUlemphZArjEfzRiFcZyo7FkHEzAI6uRuVWo9uv6hkcT5K8K6EMjvhMqtHp9bDulYOR1P+jTiNTM1BcT+2saR/rX/jI8gCVinaElKO8XtRjxTivgdxV3v28= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569973878596328.87596904024895; Tue, 1 Oct 2019 16:51:18 -0700 (PDT) Received: from localhost ([::1]:49402 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRv8-0003tA-Am for importer@patchew.org; Tue, 01 Oct 2019 19:51:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38885) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017K-2C for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001K1-Fn for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:28 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:42136) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRra-0001Hg-Tv; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPFg134046; Tue, 1 Oct 2019 19:47:25 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwber7p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from m0098404.ppops.net (m0098404.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlPJF133946; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwber75-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlmC032693; Tue, 1 Oct 2019 23:47:23 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma02dal.us.ibm.com with ESMTP id 2v9y57qb6b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:23 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlMVY54133144 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:22 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AE2142805A; Tue, 1 Oct 2019 23:47:22 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9293F28058; Tue, 1 Oct 2019 23:47:22 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:22 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 01/97] qcow2: Avoid COW during metadata preallocation Date: Tue, 1 Oct 2019 18:44:40 -0500 Message-Id: <20191001234616.7825-2-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Kevin Wolf Limiting the allocation to INT_MAX bytes isn't particularly clever because it means that the final cluster will be a partial cluster which will be completed through a COW operation. This results in unnecessary data read and write requests which lead to an unwanted non-sparse filesystem block for metadata preallocation. Align the maximum allocation size down to the cluster size to avoid this situation. Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake (cherry picked from commit f29fbf7c6b1c9a84f6931c1c222716fbe073e6e4) Signed-off-by: Michael Roth --- block/qcow2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index 3ace3b2209..dfac74c264 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2734,7 +2734,7 @@ static int coroutine_fn preallocate_co(BlockDriverSta= te *bs, uint64_t offset, bytes =3D new_length - offset; =20 while (bytes) { - cur_bytes =3D MIN(bytes, INT_MAX); + cur_bytes =3D MIN(bytes, QEMU_ALIGN_DOWN(INT_MAX, s->cluster_size)= ); ret =3D qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, &host_offset, &meta); if (ret < 0) { --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975507; cv=none; d=zoho.com; s=zohoarc; b=dp7CWVVK7lJLXIqCtbXxpTGqz/xPRM/LZGKpXTWC84WJTgviPqQQD2qkTPWxxd1ICw93OEesa1eeL5eBoEA5BNIOd9zV9NXS6KYvv+X5dKTchW0D8YrjDAG2fR1M2lNHvmVAmQpn2rMe7MhSC63WUdf/q/G0U27h4nnHQAEejBg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975507; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=AuxXTNvTZlvl6sPJXpAD2pZ/OZRduLIfz9LoSVcUvKM=; b=gxwKDTmz8t2ghmdxyY5tzO9PPt0Y+FTYOFps+Kr15jt+23IjWr1KWDOhVzjq/QbXZNFym6712z2+Iyg63xMMsXFZqZBIX1QVRjb7I38H+T2zy0/tNDzfEcQ+2Bl9aHGWG7iAA1ZgNPderoCKUJEBkOAOHHR+V1HT0EQYbjxVLSQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975507751277.60847873752516; Tue, 1 Oct 2019 17:18:27 -0700 (PDT) Received: from localhost ([::1]:49636 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSLa-0000yH-5o for importer@patchew.org; Tue, 01 Oct 2019 20:18:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39248) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrq-0001F3-Jf for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrn-0001YW-29 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:42 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:5626 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrm-0001Ob-LX; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlTdQ179519; Tue, 1 Oct 2019 19:47:29 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncr5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:29 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlROn179470; Tue, 1 Oct 2019 19:47:29 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncqv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:29 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njm5p023587; Tue, 1 Oct 2019 23:47:28 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577gxk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:28 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlSmv49742256 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:28 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 41B802805C; Tue, 1 Oct 2019 23:47:28 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 251DA28058; Tue, 1 Oct 2019 23:47:28 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:28 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 02/97] qcow2: Add errp to preallocate_co() Date: Tue, 1 Oct 2019 18:44:41 -0500 Message-Id: <20191001234616.7825-3-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Kevin Wolf We'll add a bdrv_co_truncate() call in the next patch which can return an Error that we don't want to discard. So add an errp parameter to preallocate_co(). Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake (cherry picked from commit 360bd07471dfd1830246e8403ffdc9ba9d82f9d4) Signed-off-by: Michael Roth --- block/qcow2.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index dfac74c264..b4f9f5a240 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2721,7 +2721,7 @@ static int qcow2_set_up_encryption(BlockDriverState *= bs, * Returns: 0 on success, -errno on failure. */ static int coroutine_fn preallocate_co(BlockDriverState *bs, uint64_t offs= et, - uint64_t new_length) + uint64_t new_length, Error **errp) { BDRVQcow2State *s =3D bs->opaque; uint64_t bytes; @@ -2738,6 +2738,7 @@ static int coroutine_fn preallocate_co(BlockDriverSta= te *bs, uint64_t offset, ret =3D qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, &host_offset, &meta); if (ret < 0) { + error_setg_errno(errp, -ret, "Allocating clusters failed"); return ret; } =20 @@ -2746,6 +2747,7 @@ static int coroutine_fn preallocate_co(BlockDriverSta= te *bs, uint64_t offset, =20 ret =3D qcow2_alloc_cluster_link_l2(bs, meta); if (ret < 0) { + error_setg_errno(errp, -ret, "Mapping clusters failed"); qcow2_free_any_clusters(bs, meta->alloc_offset, meta->nb_clusters, QCOW2_DISCARD_N= EVER); return ret; @@ -2775,6 +2777,7 @@ static int coroutine_fn preallocate_co(BlockDriverSta= te *bs, uint64_t offset, ret =3D bdrv_pwrite(s->data_file, (host_offset + cur_bytes) - 1, &data, 1); if (ret < 0) { + error_setg_errno(errp, -ret, "Writing to EOF failed"); return ret; } } @@ -3748,9 +3751,8 @@ static int coroutine_fn qcow2_co_truncate(BlockDriver= State *bs, int64_t offset, break; =20 case PREALLOC_MODE_METADATA: - ret =3D preallocate_co(bs, old_length, offset); + ret =3D preallocate_co(bs, old_length, offset, errp); if (ret < 0) { - error_setg_errno(errp, -ret, "Preallocation failed"); goto fail; } break; @@ -3766,9 +3768,8 @@ static int coroutine_fn qcow2_co_truncate(BlockDriver= State *bs, int64_t offset, /* With a data file, preallocation means just allocating the metad= ata * and forwarding the truncate request to the data file */ if (has_data_file(bs)) { - ret =3D preallocate_co(bs, old_length, offset); + ret =3D preallocate_co(bs, old_length, offset, errp); if (ret < 0) { - error_setg_errno(errp, -ret, "Preallocation failed"); goto fail; } break; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976170; cv=none; d=zoho.com; s=zohoarc; b=DX0C1Kqo/EYDfbXm9Um47sKS+nyNNiDKCS29fHzVzUW4mZbDytLGHXw6o0aSABDNODjEJg7Ke70Uhudf+eFyzLXBe6smNmS5Xrc+Dbm+QTO4H5odtVEp12FThXA6s3Cngux7ckxqRwcq5efgM2CMWHM95fzItQbfHLLgDOe7yWQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976170; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=za7k7+5XSq6Q3Hqd7x2WEsBMsPJrlOyTzpcqETjIWZ0=; b=gSDvzmcZU1C1WwAxEUm1quoW2fgReb7pg34f9JG4d5+SysrQontOtkMii+g9s6WDZa2+Y/VWOoH8KcHzUTbCQdPsdFRK+s3U9hCUXABYeyCrvbso3HiMAi3h9Z/ay2LeYHSa4sQTX4nyokZOKMIRI9aQvAB/uB9bntnTuyj53xQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976170755115.5265028771314; Tue, 1 Oct 2019 17:29:30 -0700 (PDT) Received: from localhost ([::1]:49728 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSWF-0001xW-5A for importer@patchew.org; Tue, 01 Oct 2019 20:29:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39758) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zg-Cx for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00024A-DU for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:35808) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs6-0001Ta-DD; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQK3086511; Tue, 1 Oct 2019 19:47:36 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat80t9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:35 -0400 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlZJn086765; Tue, 1 Oct 2019 19:47:35 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat80sm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:35 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njocu031648; Tue, 1 Oct 2019 23:47:34 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fbd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:34 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlXNQ4129440 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:33 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BDC712805A; Tue, 1 Oct 2019 23:47:33 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A22EB28058; Tue, 1 Oct 2019 23:47:33 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:33 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 03/97] qcow2: Fix full preallocation with external data file Date: Tue, 1 Oct 2019 18:44:42 -0500 Message-Id: <20191001234616.7825-4-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Kevin Wolf preallocate_co() already gave the data file the full size without forwarding the requested preallocation mode to the protocol. When bdrv_co_truncate() was called later with the preallocation mode, the file didn't actually grow any more, so the data file stayed unallocated even if full preallocation was requested. Pass the right preallocation mode to preallocate_co() and remove the second bdrv_co_truncate() to fix this. As a side effect, the ugly one-byte write in preallocate_co() is replaced with a truncate call, now leaving the last block unallocated on the protocol level as it should be. Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake (cherry picked from commit 718c0fce2f56755a8d8f737607779a98aa6e7cc4) Signed-off-by: Michael Roth --- block/qcow2.c | 41 +++++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index b4f9f5a240..7fbef97aab 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2721,11 +2721,13 @@ static int qcow2_set_up_encryption(BlockDriverState= *bs, * Returns: 0 on success, -errno on failure. */ static int coroutine_fn preallocate_co(BlockDriverState *bs, uint64_t offs= et, - uint64_t new_length, Error **errp) + uint64_t new_length, PreallocMode m= ode, + Error **errp) { BDRVQcow2State *s =3D bs->opaque; uint64_t bytes; uint64_t host_offset =3D 0; + int64_t file_length; unsigned int cur_bytes; int ret; QCowL2Meta *meta; @@ -2772,12 +2774,19 @@ static int coroutine_fn preallocate_co(BlockDriverS= tate *bs, uint64_t offset, * all of the allocated clusters (otherwise we get failing reads after * EOF). Extend the image to the last allocated sector. */ - if (host_offset !=3D 0) { - uint8_t data =3D 0; - ret =3D bdrv_pwrite(s->data_file, (host_offset + cur_bytes) - 1, - &data, 1); + file_length =3D bdrv_getlength(s->data_file->bs); + if (file_length < 0) { + error_setg_errno(errp, -file_length, "Could not get file size"); + return file_length; + } + + if (host_offset + cur_bytes > file_length) { + if (mode =3D=3D PREALLOC_MODE_METADATA) { + mode =3D PREALLOC_MODE_OFF; + } + ret =3D bdrv_co_truncate(s->data_file, host_offset + cur_bytes, mo= de, + errp); if (ret < 0) { - error_setg_errno(errp, -ret, "Writing to EOF failed"); return ret; } } @@ -3748,10 +3757,16 @@ static int coroutine_fn qcow2_co_truncate(BlockDriv= erState *bs, int64_t offset, =20 switch (prealloc) { case PREALLOC_MODE_OFF: + if (has_data_file(bs)) { + ret =3D bdrv_co_truncate(s->data_file, offset, prealloc, errp); + if (ret < 0) { + goto fail; + } + } break; =20 case PREALLOC_MODE_METADATA: - ret =3D preallocate_co(bs, old_length, offset, errp); + ret =3D preallocate_co(bs, old_length, offset, prealloc, errp); if (ret < 0) { goto fail; } @@ -3768,7 +3783,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriver= State *bs, int64_t offset, /* With a data file, preallocation means just allocating the metad= ata * and forwarding the truncate request to the data file */ if (has_data_file(bs)) { - ret =3D preallocate_co(bs, old_length, offset, errp); + ret =3D preallocate_co(bs, old_length, offset, prealloc, errp); if (ret < 0) { goto fail; } @@ -3883,16 +3898,6 @@ static int coroutine_fn qcow2_co_truncate(BlockDrive= rState *bs, int64_t offset, =20 bs->total_sectors =3D offset / BDRV_SECTOR_SIZE; =20 - if (has_data_file(bs)) { - if (prealloc =3D=3D PREALLOC_MODE_METADATA) { - prealloc =3D PREALLOC_MODE_OFF; - } - ret =3D bdrv_co_truncate(s->data_file, offset, prealloc, errp); - if (ret < 0) { - goto fail; - } - } - /* write updated header.size */ offset =3D cpu_to_be64(offset); ret =3D bdrv_pwrite_sync(bs->file, offsetof(QCowHeader, size), --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978311; cv=none; d=zoho.com; s=zohoarc; b=WzapbjHoMvOpfql3R2T+wWYnf68D1aHqOOvq8Y5BJZou634DIn0Kfj5Y0TUZmwE9NHxCxgqErQS/ODSqESK/oaTt8xBP5a1i/rMGXeh/0zhZu0GY1Gq7g49s2mDuymiwUtldwePd3oDEZygfcwxdDL9YkgMgijZIn1lgEeUt9es= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978311; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=NR7YB6xv9WhmMrBKRBOANC/b9sw/4GRE3OkNujKCXt0=; b=JiwVliOlIj8JCDDHzRqWoH3m9yNvh9s+RMvbd87kcnH4TPCQi7T2IJIZHgv49npwL6cEjtVU3dGtKqqLW3w5WUTp79MB6t/rGWmSukzDMBF+7aIvufytAM3Zc9EO5n8a8sDXnRXaWqgD5o52YaffQDqj/vgZuHnswlG7t+4cM6M= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997831173173.12914195480198; Tue, 1 Oct 2019 18:05:11 -0700 (PDT) Received: from localhost ([::1]:49996 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT4o-0008Nv-H7 for importer@patchew.org; Tue, 01 Oct 2019 21:05:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40415) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tO-B8 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsK-0002Ms-2X for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:23636) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001rg-6v; Tue, 01 Oct 2019 19:48:11 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlO5S133877; Tue, 1 Oct 2019 19:47:41 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwbere1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:41 -0400 Received: from m0098404.ppops.net (m0098404.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlf8X136416; Tue, 1 Oct 2019 19:47:41 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwberdh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:41 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkEI027150; Tue, 1 Oct 2019 23:47:40 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bh1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:40 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NldWT45285664 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:39 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 71F3D2805A; Tue, 1 Oct 2019 23:47:39 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 55E2628059; Tue, 1 Oct 2019 23:47:39 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:39 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 04/97] megasas: fix mapped frame size Date: Tue, 1 Oct 2019 18:44:43 -0500 Message-Id: <20191001234616.7825-5-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=937 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Peter Lieven , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Peter Lieven the current value of 1024 bytes (16 * MFI_FRAME_SIZE) we map is not enough = to hold the maximum number of scatter gather elements we advertise. We actually nee= d a maximum of 2048 bytes. This is 128 max sg elements * 16 bytes (sizeof (unio= n mfi_sgl)). Cc: qemu-stable@nongnu.org Signed-off-by: Peter Lieven Message-Id: <20190404121015.28634-1-pl@kamp.de> Reviewed-by: Hannes Reinecke Signed-off-by: Paolo Bonzini (cherry picked from commit 2e56fbc87f6ec3cd56c37b01d313abd502b80d61) Signed-off-by: Michael Roth --- hw/scsi/megasas.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/scsi/megasas.c b/hw/scsi/megasas.c index a56317e026..5ad762de23 100644 --- a/hw/scsi/megasas.c +++ b/hw/scsi/megasas.c @@ -477,7 +477,7 @@ static MegasasCmd *megasas_enqueue_frame(MegasasState *= s, { PCIDevice *pcid =3D PCI_DEVICE(s); MegasasCmd *cmd =3D NULL; - int frame_size =3D MFI_FRAME_SIZE * 16; + int frame_size =3D MEGASAS_MAX_SGE * sizeof(union mfi_sgl); hwaddr frame_size_p =3D frame_size; unsigned long index; =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977526; cv=none; d=zoho.com; s=zohoarc; b=gXOZad2YcBv3Oq0I8CC16fHkvVU+EkUtzrsqiMUwGWzVLs6Hou9AfA/fmyvocSj2KTG3koXtEAayWa7XTwIbC2RTOIPMTXOvG/kRUT6paf6vwwSFvP7vdaqxNYUwDDwubGiLlQUhIFIC9C+T+gwKKqhdUoT1pe5EA7NcdVbn5ek= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977526; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=L1vwi8aggPXHHhYqMIGKMQULJ1tEmAuX9LvISEjVPBY=; b=S54PWqQGwedF57rwY4dzEQ1lmoJe1ZcmUW2+3KWlkF8K19I37q2y9okKT8GWSGKmwXh9Kk/5/6IIEdCdfxrp9Vbu3n8omSJ7TwTopE5KC+WTRirS6rdy9Cy8EQRu8wao20L/Ahdui0gd3j6M2jYd2mLZQRO0tmUnkrK4Pu61vPg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977526766640.5568469773257; Tue, 1 Oct 2019 17:52:06 -0700 (PDT) Received: from localhost ([::1]:49884 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSs9-000540-2e for importer@patchew.org; Tue, 01 Oct 2019 20:52:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40242) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsL-0001nv-99 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsG-0002H9-MZ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:12 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:10548 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsG-0001mJ-Am; Tue, 01 Oct 2019 19:48:08 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOTR098881; Tue, 1 Oct 2019 19:47:46 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v22-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:46 -0400 Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NljeE099480; Tue, 1 Oct 2019 19:47:45 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v1w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:45 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkYK031624; Tue, 1 Oct 2019 23:47:45 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fca-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:45 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NljYs47513972 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:45 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E97902805A; Tue, 1 Oct 2019 23:47:44 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CDE4228059; Tue, 1 Oct 2019 23:47:44 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:44 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 05/97] qcow2: Fix qcow2_make_empty() with external data file Date: Tue, 1 Oct 2019 18:44:44 -0500 Message-Id: <20191001234616.7825-6-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Kevin Wolf make_completely_empty() is an optimisated path for bdrv_make_empty() where completely new metadata is created inside the image file instead of going through all clusters and discarding them. For an external data file, however, we actually need to do discard operations on the data file; just overwriting the qcow2 file doesn't get rid of the data. The necessary slow path with an explicit discard operation already exists for other cases. Use it for external data files, too. Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake (cherry picked from commit db04524f820582ebf1189223b6378de238511da1) Signed-off-by: Michael Roth --- block/qcow2.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 7fbef97aab..840f289a48 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4384,14 +4384,17 @@ static int qcow2_make_empty(BlockDriverState *bs) =20 if (s->qcow_version >=3D 3 && !s->snapshots && !s->nb_bitmaps && 3 + l1_clusters <=3D s->refcount_block_size && - s->crypt_method_header !=3D QCOW_CRYPT_LUKS) { + s->crypt_method_header !=3D QCOW_CRYPT_LUKS && + !has_data_file(bs)) { /* The following function only works for qcow2 v3 images (it * requires the dirty flag) and only as long as there are no * features that reserve extra clusters (such as snapshots, * LUKS header, or persistent bitmaps), because it completely * empties the image. Furthermore, the L1 table and three * additional clusters (image header, refcount table, one - * refcount block) have to fit inside one refcount block. */ + * refcount block) have to fit inside one refcount block. It + * only resets the image file, i.e. does not work with an + * external data file. */ return make_completely_empty(bs); } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569982336; cv=none; d=zoho.com; s=zohoarc; b=kqUVDTgl8wndNUtyhosFZs18r7ZPpDeJYtHMCJ5glxxBysAEBInzIw5R3lWI1SHJt4APmuGnHSjAj4xABXX01KeZswFRnyD6AGLLEFUABcB7N5QoukyL6pswAeKk6Q2ZEypQN9bNhP7V6r+88kYkEv1TUHcPJ1/hIoeRGjfZYHw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569982336; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=IkBZE2c7vtg+GqbM8799BWW4CthtrWGW0itrzuNLAJ8=; b=gCNxXyVE1Wr3eFFKzVxuFv4t3VI7/oRwivAaerjudNpjqMjQjM1HHQH68TLxPmOWH0LCvueD4l3DYF2ri4tymCYxwaUDArYDuvYF/nmi2AuCHT10jLhSErlB0RS5VSsvYw6i/rwCqDcdLOtcj5VmCOvb6xLysyaFZH7MkBrZdqc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569982336110559.1536235387568; Tue, 1 Oct 2019 19:12:16 -0700 (PDT) Received: from localhost ([::1]:50618 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFU7h-0000rp-KR for importer@patchew.org; Tue, 01 Oct 2019 22:12:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42029) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRtL-0002mE-Cf for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRtJ-0003hk-Dw for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:15 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:61154) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRtJ-0003Eh-4X; Tue, 01 Oct 2019 19:49:13 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NmmVE005032; Tue, 1 Oct 2019 19:48:52 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdfxweuf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:51 -0400 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NmpuQ005573; Tue, 1 Oct 2019 19:48:51 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdfxweh2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:51 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjmR2032160; Tue, 1 Oct 2019 23:47:52 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma01wdc.us.ibm.com with ESMTP id 2v9y57fgc8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:52 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlo9449742282 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:50 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8B38E28058; Tue, 1 Oct 2019 23:47:50 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6F45A2805C; Tue, 1 Oct 2019 23:47:50 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:50 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 06/97] block: Fix AioContext switch for bs->drv == NULL Date: Tue, 1 Oct 2019 18:44:45 -0500 Message-Id: <20191001234616.7825-7-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Kevin Wolf Even for block nodes with bs->drv =3D=3D NULL, we can't just ignore a bdrv_set_aio_context() call. Leaving the node in its old context can mean that it's still in an iothread context in bdrv_close_all() during shutdown, resulting in an attempted unlock of the AioContext lock which we don't hold. This is an example stack trace of a related crash: #0 0x00007ffff59da57f in raise () at /lib64/libc.so.6 #1 0x00007ffff59c4895 in abort () at /lib64/libc.so.6 #2 0x0000555555b97b1e in error_exit (err=3D, msg=3Dmsg@ent= ry=3D0x555555d386d0 <__func__.19059> "qemu_mutex_unlock_impl") at util/qemu= -thread-posix.c:36 #3 0x0000555555b97f7f in qemu_mutex_unlock_impl (mutex=3Dmutex@entry=3D0x= 5555568002f0, file=3Dfile@entry=3D0x555555d378df "util/async.c", line=3Dlin= e@entry=3D507) at util/qemu-thread-posix.c:97 #4 0x0000555555b92f55 in aio_context_release (ctx=3Dctx@entry=3D0x5555568= 00290) at util/async.c:507 #5 0x0000555555b05cf8 in bdrv_prwv_co (child=3Dchild@entry=3D0x7fffc80012= f0, offset=3Doffset@entry=3D131072, qiov=3Dqiov@entry=3D0x7fffffffd4f0, is_= write=3Dis_write@entry=3Dtrue, flags=3Dflags@entry=3D0) at block/io.c:833 #6 0x0000555555b060a9 in bdrv_pwritev (qiov=3D0x7fffffffd4f0, offset=3D13= 1072, child=3D0x7fffc80012f0) at block/io.c:990 #7 0x0000555555b060a9 in bdrv_pwrite (child=3D0x7fffc80012f0, offset=3D13= 1072, buf=3D, bytes=3D) at block/io.c:990 #8 0x0000555555ae172b in qcow2_cache_entry_flush (bs=3Dbs@entry=3D0x55555= 6810680, c=3Dc@entry=3D0x5555568cc740, i=3Di@entry=3D0) at block/qcow2-cach= e.c:51 #9 0x0000555555ae18dd in qcow2_cache_write (bs=3Dbs@entry=3D0x55555681068= 0, c=3D0x5555568cc740) at block/qcow2-cache.c:248 #10 0x0000555555ae15de in qcow2_cache_flush (bs=3D0x555556810680, c=3D) at block/qcow2-cache.c:259 #11 0x0000555555ae16b1 in qcow2_cache_flush_dependency (c=3D0x5555568a1700= , c=3D0x5555568a1700, bs=3D0x555556810680) at block/qcow2-cache.c:194 #12 0x0000555555ae16b1 in qcow2_cache_entry_flush (bs=3Dbs@entry=3D0x55555= 6810680, c=3Dc@entry=3D0x5555568a1700, i=3Di@entry=3D0) at block/qcow2-cach= e.c:194 #13 0x0000555555ae18dd in qcow2_cache_write (bs=3Dbs@entry=3D0x55555681068= 0, c=3D0x5555568a1700) at block/qcow2-cache.c:248 #14 0x0000555555ae15de in qcow2_cache_flush (bs=3Dbs@entry=3D0x55555681068= 0, c=3D) at block/qcow2-cache.c:259 #15 0x0000555555ad242c in qcow2_inactivate (bs=3Dbs@entry=3D0x555556810680= ) at block/qcow2.c:2124 #16 0x0000555555ad2590 in qcow2_close (bs=3D0x555556810680) at block/qcow2= .c:2153 #17 0x0000555555ab0c62 in bdrv_close (bs=3D0x555556810680) at block.c:3358 #18 0x0000555555ab0c62 in bdrv_delete (bs=3D0x555556810680) at block.c:3542 #19 0x0000555555ab0c62 in bdrv_unref (bs=3D0x555556810680) at block.c:4598 #20 0x0000555555af4d72 in blk_remove_bs (blk=3Dblk@entry=3D0x5555568103d0)= at block/block-backend.c:785 #21 0x0000555555af4dbb in blk_remove_all_bs () at block/block-backend.c:483 #22 0x0000555555aae02f in bdrv_close_all () at block.c:3412 #23 0x00005555557f9796 in main (argc=3D, argv=3D, envp=3D) at vl.c:4776 The reproducer I used is a qcow2 image on gluster volume, where the virtual disk size (4 GB) is larger than the gluster volume size (64M), so we can easily trigger an ENOSPC. This backend is assigned to a virtio-blk device using an iothread, and then from the guest a 'dd if=3D/dev/zero of=3D/dev/vda bs=3D1G count=3D1' causes the VM to stop because of an I/O error. qemu_gluster_co_flush_to_disk() sets bs->drv =3D NULL on error, so when virtio-blk stops the dataplane, the block nodes stay in the iothread AioContext. A 'quit' monitor command issued from this paused state crashes the process. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=3D1631227 Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Stefano Garzarella (cherry picked from commit 1bffe1ae7a7b707c3a14ea2ccd00d3609d3ce4d8) Signed-off-by: Michael Roth --- block.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/block.c b/block.c index 16615bc876..9ae5c0ed2f 100644 --- a/block.c +++ b/block.c @@ -5672,10 +5672,6 @@ void bdrv_detach_aio_context(BlockDriverState *bs) BdrvAioNotifier *baf, *baf_tmp; BdrvChild *child; =20 - if (!bs->drv) { - return; - } - assert(!bs->walking_aio_notifiers); bs->walking_aio_notifiers =3D true; QLIST_FOREACH_SAFE(baf, &bs->aio_notifiers, list, baf_tmp) { @@ -5690,7 +5686,7 @@ void bdrv_detach_aio_context(BlockDriverState *bs) */ bs->walking_aio_notifiers =3D false; =20 - if (bs->drv->bdrv_detach_aio_context) { + if (bs->drv && bs->drv->bdrv_detach_aio_context) { bs->drv->bdrv_detach_aio_context(bs); } QLIST_FOREACH(child, &bs->children, next) { @@ -5709,10 +5705,6 @@ void bdrv_attach_aio_context(BlockDriverState *bs, BdrvAioNotifier *ban, *ban_tmp; BdrvChild *child; =20 - if (!bs->drv) { - return; - } - if (bs->quiesce_counter) { aio_disable_external(new_context); } @@ -5722,7 +5714,7 @@ void bdrv_attach_aio_context(BlockDriverState *bs, QLIST_FOREACH(child, &bs->children, next) { bdrv_attach_aio_context(child->bs, new_context); } - if (bs->drv->bdrv_attach_aio_context) { + if (bs->drv && bs->drv->bdrv_attach_aio_context) { bs->drv->bdrv_attach_aio_context(bs, new_context); } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979688; cv=none; d=zoho.com; s=zohoarc; b=ds4M8JI8SH1FuONBqwu0SLBWo7bN0LZEx3xGJ0RSsrs7XoDVLPnsdUZ48NB9vd0PakWnBWEKeyjUHlb5I5qnkB0HjDVWPhpzxPkV6VMwH7tTDTTjnyQx7+FlzA5g+mBHN2pEQuonHiSwMvwqsXqMVx1i4RxI2ZiXx2MNs8RB1h8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979688; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=NtQCEHFdN2h6m3MezyyuRdMPTqvL8HnHRs3TW+e7Z/I=; b=mITOa3iEsto0nlGvgqE5Clt2BE1f/96H9MsaQ0Fspl9wNoB9FbUpMjnkjTFrLuSk5MVnDWphNW3p5glgWS+GzUBbhgipWxbtXVME6B89GxeQsGTS+2HyLxxQsZJV9Rznpdy6L4iiARubVhZAML/9T5533n3TNiA1xhf8zKkRDPA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979688668565.6587055075696; Tue, 1 Oct 2019 18:28:08 -0700 (PDT) Received: from localhost ([::1]:50172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTQy-0004hL-QH for importer@patchew.org; Tue, 01 Oct 2019 21:28:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40587) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsa-0001zN-DZ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002Yx-Ke for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:28 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:33448 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsQ-00020n-Fe; Tue, 01 Oct 2019 19:48:23 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQa5079025; Tue, 1 Oct 2019 19:47:57 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f013j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:57 -0400 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlXRJ079208; Tue, 1 Oct 2019 19:47:57 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f013e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:57 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njkti031621; Tue, 1 Oct 2019 23:47:56 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fd2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:56 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nluux50921774 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:56 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 081B428059; Tue, 1 Oct 2019 23:47:56 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E10712805A; Tue, 1 Oct 2019 23:47:55 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:55 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 07/97] cutils: Fix size_to_str() on 32-bit platforms Date: Tue, 1 Oct 2019 18:44:46 -0500 Message-Id: <20191001234616.7825-8-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake When extracting a human-readable size formatter, we changed 'uint64_t div' pre-patch to 'unsigned long div' post-patch. Which breaks on 32-bit platforms, resulting in 'inf' instead of intended values larger than 999GB. Fixes: 22951aaa CC: qemu-stable@nongnu.org Reported-by: Max Reitz Signed-off-by: Eric Blake Reviewed-by: Max Reitz Signed-off-by: Kevin Wolf (cherry picked from commit 754da86714d550c3f995f11a2587395081362e0a) Signed-off-by: Michael Roth --- util/cutils.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/cutils.c b/util/cutils.c index e098debdc0..d682c90901 100644 --- a/util/cutils.c +++ b/util/cutils.c @@ -825,7 +825,7 @@ const char *qemu_ether_ntoa(const MACAddr *mac) char *size_to_str(uint64_t val) { static const char *suffixes[] =3D { "", "Ki", "Mi", "Gi", "Ti", "Pi", = "Ei" }; - unsigned long div; + uint64_t div; int i; =20 /* --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979001; cv=none; d=zoho.com; s=zohoarc; b=Ihgcug4hx6YywRx92rKRPQ9E1OlovDl/zxdMfsBUqpVDF31XXy6qbVcsXHirlpDaIIqGq0Hfixdqnhny+srrqxrZy0MVkVvG0D9ndMwt+c9GVm5A65wG2EPQd5TLWwAS/XxK5PStQyKV2bOcdILf+2biJVylMV9vhDYRFoSO8+Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979001; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=v2zaLboOxzYNvnEvQDD4LanDYg9SgxLcztlw9YLtDxE=; b=nbCdcfQJACenUdXCYUKfkmSLWcyJIvQ6aaPhpE17NmmIQVQBXmc/H040DXViEp4+JTTvdumDExJEni0Q+f5h9sXYKbeHyrKu2W7VbJ6UTcs5F1MeeGmIRPmN9u6awkxnjcaFp1rO6/VvR73tFo0maTZmrlkoD41EArBPgh4fonk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979001820157.44839284539785; Tue, 1 Oct 2019 18:16:41 -0700 (PDT) Received: from localhost ([::1]:50096 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTFv-0002TP-Nn for importer@patchew.org; Tue, 01 Oct 2019 21:16:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41282) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsp-0002EG-5G for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsh-0002vG-UQ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:39708 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-000285-RK; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPGi179294; Tue, 1 Oct 2019 19:48:04 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnd5t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:03 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlhwq180306; Tue, 1 Oct 2019 19:48:03 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnd5j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:03 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk0T006371; Tue, 1 Oct 2019 23:48:03 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye9k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:03 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm1ht31850886 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:01 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8D3F128059; Tue, 1 Oct 2019 23:48:01 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 71ACE2805C; Tue, 1 Oct 2019 23:48:01 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:01 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 08/97] Makefile: add nit-picky mode to sphinx-build Date: Tue, 1 Oct 2019 18:44:47 -0500 Message-Id: <20191001234616.7825-9-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow If we add references that don't resolve (or accidentally remove them), it will be helpful to have warning messages alerting us to that. Further, turn those warnings into errors so we can be alerted to these problems sooner rather than later. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20190426221528.30293-2-jsnow@redhat.com [adjusted commit message. --js] Signed-off-by: John Snow (cherry picked from commit 9e5b6cb87db66dfb606604fe6cf40e5ddf1ef0e7) Signed-off-by: Michael Roth --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 04a0d45050..d4c5750256 100644 --- a/Makefile +++ b/Makefile @@ -899,7 +899,7 @@ docs/version.texi: $(SRC_PATH)/VERSION sphinxdocs: $(MANUAL_BUILDDIR)/devel/index.html $(MANUAL_BUILDDIR)/interop= /index.html =20 # Canned command to build a single manual -build-manual =3D $(call quiet-command,sphinx-build $(if $(V),,-q) -b html = -D version=3D$(VERSION) -D release=3D"$(FULL_VERSION)" -d .doctrees/$1 $(SR= C_PATH)/docs/$1 $(MANUAL_BUILDDIR)/$1 ,"SPHINX","$(MANUAL_BUILDDIR)/$1") +build-manual =3D $(call quiet-command,sphinx-build $(if $(V),,-q) -W -n -b= html -D version=3D$(VERSION) -D release=3D"$(FULL_VERSION)" -d .doctrees/$= 1 $(SRC_PATH)/docs/$1 $(MANUAL_BUILDDIR)/$1 ,"SPHINX","$(MANUAL_BUILDDIR)/$= 1") # We assume all RST files in the manual's directory are used in it manual-deps =3D $(wildcard $(SRC_PATH)/docs/$1/*.rst) $(SRC_PATH)/docs/$1/= conf.py $(SRC_PATH)/docs/conf.py =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980645; cv=none; d=zoho.com; s=zohoarc; b=QwYjVW1ksBZFXJuI6AB44B4saiHrsfD6Ij9H/IlIA2yWk1eMyQrKLMROuY8+0uKKfmc2o79n21X1Ru1YS58atROmyH+7DYFzsBlSESI4CorM0KrDTlySxlgRHQrWxJWiMcc0lnTUbN0nL1oLa7RYhTmH8+290yUJW8JSyrud2rU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980645; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=R21Ci/A71e+yLNcJRgY2OcAsiijkiBVm4xWq/lur6jo=; b=dj2jV4/K3Hy3UkLC+xMrWFg/1nc1havrZQtEB+flqw+oeJaXBzQCf8uyOvHOGp14HpbTBvp4+uDDttfjDUdYSLgd1Br+E6KsBXk3VaKolwWH0NuP7VWJanCnD9Wnzq8zDSn8kiTx6nTZ2k+BgiQ/SjO3KrcOx5F9ryfttu7oUhY= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569980645219414.05517275042257; Tue, 1 Oct 2019 18:44:05 -0700 (PDT) Received: from localhost ([::1]:50330 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTgQ-0005uH-6v for importer@patchew.org; Tue, 01 Oct 2019 21:44:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42071) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRtM-0002ox-Qn for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRtG-0003bm-2z for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:16 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:55392) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRtF-00039L-Gx; Tue, 01 Oct 2019 19:49:10 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NmlYw004963; Tue, 1 Oct 2019 19:48:47 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdfxwea5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:47 -0400 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NiSkf113972; Tue, 1 Oct 2019 19:47:17 -0400 Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdfxwe9q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:17 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlSx022434; Tue, 1 Oct 2019 23:47:16 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04dal.us.ibm.com with ESMTP id 2v9y57yd54-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:16 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlGla54788522 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:16 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EF0172805E; Tue, 1 Oct 2019 23:47:15 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A26B428059; Tue, 1 Oct 2019 23:47:15 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:15 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 09/97] docs/interop/bitmaps: rewrite and modernize doc Date: Tue, 1 Oct 2019 18:44:48 -0500 Message-Id: <20191001234616.7825-10-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow This just about rewrites the entirety of the bitmaps.rst document to make it consistent with the 4.0 release. I have added new features seen in the 4.0 release, as well as tried to clarify some points that keep coming up when discussing this feature both in-house and upstream. It does not yet cover pull backups or migration details, but I intend to keep extending this document to cover those cases. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20190426221528.30293-3-jsnow@redhat.com [Adjusted commit message. --js] Signed-off-by: John Snow (cherry picked from commit 90edef80a0852cf8a3d2668898ee40e8970e4314) Signed-off-by: Michael Roth --- docs/interop/bitmaps.rst | 1573 ++++++++++++++++++++++++++++++-------- 1 file changed, 1251 insertions(+), 322 deletions(-) diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst index 7bcfe7f461..510e8809a9 100644 --- a/docs/interop/bitmaps.rst +++ b/docs/interop/bitmaps.rst @@ -1,5 +1,5 @@ .. - Copyright 2015 John Snow and Red Hat, Inc. + Copyright 2019 John Snow and Red Hat, Inc. All rights reserved. =20 This file is licensed via The FreeBSD Documentation License, the full @@ -9,547 +9,1476 @@ Dirty Bitmaps and Incremental Backup =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -- Dirty Bitmaps are objects that track which data needs to be backed up - for the next incremental backup. +Dirty Bitmaps are in-memory objects that track writes to block devices. Th= ey +can be used in conjunction with various block job operations to perform +incremental or differential backup regimens. =20 -- Dirty bitmaps can be created at any time and attached to any node - (not just complete drives). +This document explains the conceptual mechanisms, as well as up-to-date, +complete and comprehensive documentation on the API to manipulate them. +(Hopefully, the "why", "what", and "how".) + +The intended audience for this document is developers who are adding QEMU +backup features to management applications, or power users who run and +administer QEMU directly via QMP. =20 .. contents:: =20 +Overview +-------- + +Bitmaps are bit vectors where each '1' bit in the vector indicates a modif= ied +("dirty") segment of the corresponding block device. The size of the segme= nt +that is tracked is the granularity of the bitmap. If the granularity of a +bitmap is 64K, each '1' bit means that a 64K region as a whole may have +changed in some way, possibly by as little as one byte. + +Smaller granularities mean more accurate tracking of modified disk data, b= ut +requires more computational overhead and larger bitmap sizes. Larger +granularities mean smaller bitmap sizes, but less targeted backups. + +The size of a bitmap (in bytes) can be computed as such: + ``size`` =3D ceil(ceil(``image_size`` / ``granularity``) / 8) + +e.g. the size of a 64KiB granularity bitmap on a 2TiB image is: + ``size`` =3D ((2147483648K / 64K) / 8) + =3D 4194304B =3D 4MiB. + +QEMU uses these bitmaps when making incremental backups to know which sect= ions +of the file to copy out. They are not enabled by default and must be +explicitly added in order to begin tracking writes. + +Bitmaps can be created at any time and can be attached to any arbitrary bl= ock +node in the storage graph, but are most useful conceptually when attached = to +the root node attached to the guest's storage device model. + +That is to say: It's likely most useful to track the guest's writes to dis= k, +but you could theoretically track things like qcow2 metadata changes by +attaching the bitmap elsewhere in the storage graph. This is beyond the sc= ope +of this document. + +QEMU supports persisting these bitmaps to disk via the qcow2 image format. +Bitmaps which are stored or loaded in this way are called "persistent", +whereas bitmaps that are not are called "transient". + +QEMU also supports the migration of both transient bitmaps (tracking any +arbitrary image format) or persistent bitmaps (qcow2) via live migration. + +Supported Image Formats +----------------------- + +QEMU supports all documented features below on the qcow2 image format. + +However, qcow2 is only strictly necessary for the persistence feature, whi= ch +writes bitmap data to disk upon close. If persistence is not required for a +specific use case, all bitmap features excepting persistence are available= for +any arbitrary image format. + +For example, Dirty Bitmaps can be combined with the 'raw' image format, but +any changes to the bitmap will be discarded upon exit. + +.. warning:: Transient bitmaps will not be saved on QEMU exit! Persistent + bitmaps are available only on qcow2 images. + Dirty Bitmap Names ------------------ =20 -- A dirty bitmap's name is unique to the node, but bitmaps attached to - different nodes can share the same name. +Bitmap objects need a method to reference them in the API. All API-created= and +managed bitmaps have a human-readable name chosen by the user at creation +time. =20 -- Dirty bitmaps created for internal use by QEMU may be anonymous and - have no name, but any user-created bitmaps must have a name. There - can be any number of anonymous bitmaps per node. +- A bitmap's name is unique to the node, but bitmaps attached to different + nodes can share the same name. Therefore, all bitmaps are addressed via + their (node, name) pair. =20 -- The name of a user-created bitmap must not be empty (""). +- The name of a user-created bitmap cannot be empty (""). =20 -Bitmap Modes ------------- +- Transient bitmaps can have JSON unicode names that are effectively not + length limited. (QMP protocol may restrict messages to less than 64MiB.) + +- Persistent storage formats may impose their own requirements on bitmap n= ames + and namespaces. Presently, only qcow2 supports persistent bitmaps. See + docs/interop/qcow2.txt for more details on restrictions. Notably: + + - qcow2 bitmap names are limited to between 1 and 1023 bytes long. + + - No two bitmaps saved to the same qcow2 file may share the same name. + +- QEMU occasionally uses bitmaps for internal use which have no name. They= are + hidden from API query calls, cannot be manipulated by the external API, = are + never persistent, nor ever migrated. + +Bitmap Status +------------- + +Dirty Bitmap objects can be queried with the QMP command `query-block +`_, and are visible via the +`BlockDirtyInfo `_ QAPI structure. + +This struct shows the name, granularity, and dirty byte count for each bit= map. +Additionally, it shows several boolean status indicators: + +- ``recording``: This bitmap is recording writes. +- ``busy``: This bitmap is in-use by an operation. +- ``persistent``: This bitmap is a persistent type. +- ``inconsistent``: This bitmap is corrupted and cannot be used. + +The ``+busy`` status prohibits you from deleting, clearing, or otherwise +modifying a bitmap, and happens when the bitmap is being used for a backup +operation or is in the process of being loaded from a migration. Many of t= he +commands documented below will refuse to work on such bitmaps. + +The ``+inconsistent`` status similarly prohibits almost all operations, +notably allowing only the ``block-dirty-bitmap-remove`` operation. + +There is also a deprecated ``status`` field of type `DirtyBitmapStatus +`_. A bitmap historically had +five visible states: + + #. ``Frozen``: This bitmap is currently in-use by an operation and is + immutable. It can't be deleted, renamed, reset, etc. + + (This is now ``+busy``.) + + #. ``Disabled``: This bitmap is not recording new writes. + + (This is now ``-recording -busy``.) + + #. ``Active``: This bitmap is recording new writes. + + (This is now ``+recording -busy``.) =20 -- A bitmap can be "frozen," which means that it is currently in-use by - a backup operation and cannot be deleted, renamed, written to, reset, - etc. + #. ``Locked``: This bitmap is in-use by an operation, and is immutable. + The difference from "Frozen" was primarily implementation details. =20 -- The normal operating mode for a bitmap is "active." + (This is now ``+busy``.) + + #. ``Inconsistent``: This persistent bitmap was not saved to disk + correctly, and can no longer be used. It remains in memory to serve = as + an indicator of failure. + + (This is now ``+inconsistent``.) + +These states are directly replaced by the status indicators and should not= be +used. The difference between ``Frozen`` and ``Locked`` is an implementation +detail and should not be relevant to external users. =20 Basic QMP Usage --------------- =20 +The primary interface to manipulating bitmap objects is via the QMP +interface. If you are not familiar, see docs/interop/qmp-intro.txt for a b= road +overview, and `qemu-qmp-ref `_ for a full reference of = all +QMP commands. + Supported Commands ~~~~~~~~~~~~~~~~~~ =20 +There are six primary bitmap-management API commands: + - ``block-dirty-bitmap-add`` - ``block-dirty-bitmap-remove`` - ``block-dirty-bitmap-clear`` +- ``block-dirty-bitmap-disable`` +- ``block-dirty-bitmap-enable`` +- ``block-dirty-bitmap-merge`` =20 -Creation -~~~~~~~~ +And one related query command: =20 -- To create a new bitmap, enabled, on the drive with id=3Ddrive0: +- ``query-block`` =20 -.. code:: json +Creation: block-dirty-bitmap-add +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 - { "execute": "block-dirty-bitmap-add", - "arguments": { - "node": "drive0", - "name": "bitmap0" - } - } +`block-dirty-bitmap-add +`_: =20 -- This bitmap will have a default granularity that matches the cluster - size of its associated drive, if available, clamped to between [4KiB, - 64KiB]. The current default for qcow2 is 64KiB. +Creates a new bitmap that tracks writes to the specified node. granularity, +persistence, and recording state can be adjusted at creation time. =20 -- To create a new bitmap that tracks changes in 32KiB segments: +.. admonition:: Example =20 -.. code:: json + to create a new, actively recording persistent bitmap: =20 - { "execute": "block-dirty-bitmap-add", - "arguments": { - "node": "drive0", - "name": "bitmap0", - "granularity": 32768 - } - } + .. code:: json =20 -Deletion -~~~~~~~~ + -> { "execute": "block-dirty-bitmap-add", + "arguments": { + "node": "drive0", + "name": "bitmap0", + "persistent": true, + } + } =20 -- Bitmaps that are frozen cannot be deleted. + <- { "return": {} } =20 -- Deleting the bitmap does not impact any other bitmaps attached to the - same node, nor does it affect any backups already created from this - node. +- This bitmap will have a default granularity that matches the cluster siz= e of + its associated drive, if available, clamped to between [4KiB, 64KiB]. The + current default for qcow2 is 64KiB. =20 -- Because bitmaps are only unique to the node to which they are - attached, you must specify the node/drive name here, too. +.. admonition:: Example =20 -.. code:: json + To create a new, disabled (``-recording``), transient bitmap that tracks + changes in 32KiB segments: =20 - { "execute": "block-dirty-bitmap-remove", - "arguments": { - "node": "drive0", - "name": "bitmap0" - } - } + .. code:: json =20 -Resetting -~~~~~~~~~ + -> { "execute": "block-dirty-bitmap-add", + "arguments": { + "node": "drive0", + "name": "bitmap1", + "granularity": 32768, + "disabled": true + } + } =20 -- Resetting a bitmap will clear all information it holds. + <- { "return": {} } =20 -- An incremental backup created from an empty bitmap will copy no data, - as if nothing has changed. +Deletion: block-dirty-bitmap-remove +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -.. code:: json +`block-dirty-bitmap-remove +`_: + +Deletes a bitmap. Bitmaps that are ``+busy`` cannot be removed. + +- Deleting a bitmap does not impact any other bitmaps attached to the same + node, nor does it affect any backups already created from this bitmap or + node. + +- Because bitmaps are only unique to the node to which they are attached, = you + must specify the node/drive name here, too. + +- Deleting a persistent bitmap will remove it from the qcow2 file. + +.. admonition:: Example + + Remove a bitmap named ``bitmap0`` from node ``drive0``: + + .. code:: json + + -> { "execute": "block-dirty-bitmap-remove", + "arguments": { + "node": "drive0", + "name": "bitmap0" + } + } + + <- { "return": {} } + +Resetting: block-dirty-bitmap-clear +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`block-dirty-bitmap-clear +`_: + +Clears all dirty bits from a bitmap. ``+busy`` bitmaps cannot be cleared. + +- An incremental backup created from an empty bitmap will copy no data, as= if + nothing has changed. + +.. admonition:: Example + + Clear all dirty bits from bitmap ``bitmap0`` on node ``drive0``: + + .. code:: json + + -> { "execute": "block-dirty-bitmap-clear", + "arguments": { + "node": "drive0", + "name": "bitmap0" + } + } + + <- { "return": {} } + +Enabling: block-dirty-bitmap-enable +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`block-dirty-bitmap-enable +`_: =20 - { "execute": "block-dirty-bitmap-clear", - "arguments": { - "node": "drive0", - "name": "bitmap0" - } - } +"Enables" a bitmap, setting the ``recording`` bit to true, causing writes = to +begin being recorded. ``+busy`` bitmaps cannot be enabled. + +- Bitmaps default to being enabled when created, unless configured otherwi= se. + +- Persistent enabled bitmaps will remember their ``+recording`` status on + load. + +.. admonition:: Example + + To set ``+recording`` on bitmap ``bitmap0`` on node ``drive0``: + + .. code:: json + + -> { "execute": "block-dirty-bitmap-enable", + "arguments": { + "node": "drive0", + "name": "bitmap0" + } + } + + <- { "return": {} } + +Enabling: block-dirty-bitmap-disable +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`block-dirty-bitmap-disable +`_: + +"Disables" a bitmap, setting the ``recording`` bit to false, causing furth= er +writes to begin being ignored. ``+busy`` bitmaps cannot be disabled. + +.. warning:: + + This is potentially dangerous: QEMU makes no effort to stop any writes if + there are disabled bitmaps on a node, and will not mark any disabled bit= maps + as ``+inconsistent`` if any such writes do happen. Backups made from such + bitmaps will not be able to be used to reconstruct a coherent image. + +- Disabling a bitmap may be useful for examining which sectors of a disk + changed during a specific time period, or for explicit management of + differential backup windows. + +- Persistent disabled bitmaps will remember their ``-recording`` status on + load. + +.. admonition:: Example + + To set ``-recording`` on bitmap ``bitmap0`` on node ``drive0``: + + .. code:: json + + -> { "execute": "block-dirty-bitmap-disable", + "arguments": { + "node": "drive0", + "name": "bitmap0" + } + } + + <- { "return": {} } + +Merging, Copying: block-dirty-bitmap-merge +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`block-dirty-bitmap-merge +`_: + +Merges one or more bitmaps into a target bitmap. For any segment that is d= irty +in any one source bitmap, the target bitmap will mark that segment dirty. + +- Merge takes one or more bitmaps as a source and merges them together int= o a + single destination, such that any segment marked as dirty in any source + bitmap(s) will be marked dirty in the destination bitmap. + +- Merge does not create the destination bitmap if it does not exist. A bla= nk + bitmap can be created beforehand to achieve the same effect. + +- The destination is not cleared prior to merge, so subsequent merge + operations will continue to cumulatively mark more segments as dirty. + +- If the merge operation should fail, the destination bitmap is guaranteed= to + be unmodified. The operation may fail if the source or destination bitma= ps + are busy, or have different granularities. + +- Bitmaps can only be merged on the same node. There is only one "node" + argument, so all bitmaps must be attached to that same node. + +- Copy can be achieved by merging from a single source to an empty + destination. + +.. admonition:: Example + + Merge the data from ``bitmap0`` into the bitmap ``new_bitmap`` on node + ``drive0``. If ``new_bitmap`` was empty prior to this command, this achie= ves + a copy. + + .. code:: json + + -> { "execute": "block-dirty-bitmap-merge", + "arguments": { + "node": "drive0", + "target": "new_bitmap", + "bitmaps: [ "bitmap0" ] + } + } + + <- { "return": {} } + +Querying: query-block +~~~~~~~~~~~~~~~~~~~~~ + +`query-block +`_: + +Not strictly a bitmaps command, but will return information about any bitm= aps +attached to nodes serving as the root for guest devices. + +- The "inconsistent" bit will not appear when it is false, appearing only = when + the value is true to indicate there is a problem. + +.. admonition:: Example + + Query the block sub-system of QEMU. The following json has trimmed irrele= vant + keys from the response to highlight only the bitmap-relevant portions of = the + API. This result highlights a bitmap ``bitmap0`` attached to the root nod= e of + device ``drive0``. + + .. code:: json + + -> { + "execute": "query-block", + "arguments": {} + } + + <- { + "return": [ { + "dirty-bitmaps": [ { + "status": "active", + "count": 0, + "busy": false, + "name": "bitmap0", + "persistent": false, + "recording": true, + "granularity": 65536 + } ], + "device": "drive0", + } ] + } + +Bitmap Persistence +------------------ + +As outlined in `Supported Image Formats`_, QEMU can persist bitmaps to qco= w2 +files. Demonstrated in `Creation: block-dirty-bitmap-add`_, passing +``persistent: true`` to ``block-dirty-bitmap-add`` will persist that bitma= p to +disk. + +Persistent bitmaps will be automatically loaded into memory upon load, and +will be written back to disk upon close. Their usage should be mostly +transparent. + +However, if QEMU does not get a chance to close the file cleanly, the bitm= ap +will be marked as ``+inconsistent`` at next load and considered unsafe to = use +for any operation. At this point, the only valid operation on such bitmaps= is +``block-dirty-bitmap-remove``. + +Losing a bitmap in this way does not invalidate any existing backups that = have +been made from this bitmap, but no further backups will be able to be issu= ed +for this chain. =20 Transactions ------------ =20 +Transactions are a QMP feature that allows you to submit multiple QMP comm= ands +at once, being guaranteed that they will all succeed or fail atomically, +together. The interaction of bitmaps and transactions are demonstrated bel= ow. + +See `transaction `_ in the QMP refere= nce +for more details. + Justification ~~~~~~~~~~~~~ =20 -Bitmaps can be safely modified when the VM is paused or halted by using -the basic QMP commands. For instance, you might perform the following -actions: +Bitmaps can generally be modified at any time, but certain operations often +only make sense when paired directly with other commands. When a VM is pau= sed, +it's easy to ensure that no guest writes occur between individual QMP +commands. When a VM is running, this is difficult to accomplish with +individual QMP commands that may allow guest writes to occur inbetween each +command. =20 -1. Boot the VM in a paused state. -2. Create a full drive backup of drive0. -3. Create a new bitmap attached to drive0. -4. Resume execution of the VM. -5. Incremental backups are ready to be created. +For example, using only individual QMP commands, we could: =20 -At this point, the bitmap and drive backup would be correctly in sync, -and incremental backups made from this point forward would be correctly -aligned to the full drive backup. +#. Boot the VM in a paused state. +#. Create a full drive backup of drive0. +#. Create a new bitmap attached to drive0, confident that nothing has been + written to drive0 in the meantime. +#. Resume execution of the VM. +#. At a later point, issue incremental backups from ``bitmap0``. =20 -This is not particularly useful if we decide we want to start -incremental backups after the VM has been running for a while, for which -we will need to perform actions such as the following: +At this point, the bitmap and drive backup would be correctly in sync, and +incremental backups made from this point forward would be correctly aligne= d to +the full drive backup. =20 -1. Boot the VM and begin execution. -2. Using a single transaction, perform the following operations: +This is not particularly useful if we decide we want to start incremental +backups after the VM has been running for a while, for which we would want= to +perform actions such as the following: + +#. Boot the VM and begin execution. +#. Using a single transaction, perform the following operations: =20 - Create ``bitmap0``. - Create a full drive backup of ``drive0``. =20 -3. Incremental backups are now ready to be created. +#. At a later point, issue incremental backups from ``bitmap0``. + +.. note:: As a consideration, if ``bitmap0`` is created prior to the full + drive backup, incremental backups can still be authored from this + bitmap, but they will copy extra segments reflecting writes that + occurred prior to the backup operation. Transactions allow us to + narrow critical points in time to reduce waste, or, in the other + direction, to ensure that no segments are omitted. =20 Supported Bitmap Transactions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 - ``block-dirty-bitmap-add`` - ``block-dirty-bitmap-clear`` +- ``block-dirty-bitmap-enable`` +- ``block-dirty-bitmap-disable`` +- ``block-dirty-bitmap-merge`` =20 -The usages are identical to their respective QMP commands, but see below -for examples. +The usages for these commands are identical to their respective QMP comman= ds, +but see the sections below for concrete examples. =20 -Example: New Incremental Backup -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Incremental Backups - Push Model +-------------------------------- =20 -As outlined in the justification, perhaps we want to create a new -incremental backup chain attached to a drive. +Incremental backups are simply partial disk images that can be combined wi= th +other partial disk images on top of a base image to reconstruct a full bac= kup +from the point in time at which the incremental backup was issued. =20 -.. code:: json +The "Push Model" here references the fact that QEMU is "pushing" the modif= ied +blocks out to a destination. We will be using the `drive-backup +`_ and `blockdev-backup +`_ QMP commands to create both +full and incremental backups. =20 - { "execute": "transaction", - "arguments": { - "actions": [ - {"type": "block-dirty-bitmap-add", - "data": {"node": "drive0", "name": "bitmap0"} }, - {"type": "drive-backup", - "data": {"device": "drive0", "target": "/path/to/full_backup.im= g", - "sync": "full", "format": "qcow2"} } - ] - } - } +Both of these commands are jobs, which have their own QMP API for querying= and +management documented in `Background jobs +`_. =20 Example: New Incremental Backup Anchor Point ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -Maybe we just want to create a new full backup with an existing bitmap -and want to reset the bitmap to track the new chain. +As outlined in the Transactions - `Justification`_ section, perhaps we wan= t to +create a new incremental backup chain attached to a drive. + +This example creates a new, full backup of "drive0" and accompanies it wit= h a +new, empty bitmap that records writes from this point in time forward. + +.. note:: Any new writes that happen after this command is issued, even wh= ile + the backup job runs, will be written locally and not to the back= up + destination. These writes will be recorded in the bitmap + accordingly. + +.. code:: json + + -> { + "execute": "transaction", + "arguments": { + "actions": [ + { + "type": "block-dirty-bitmap-add", + "data": { + "node": "drive0", + "name": "bitmap0" + } + }, + { + "type": "drive-backup", + "data": { + "device": "drive0", + "target": "/path/to/drive0.full.qcow2", + "sync": "full", + "format": "qcow2" + } + } + ] + } + } + + <- { "return": {} } + + <- { + "timestamp": { + "seconds": 1555436945, + "microseconds": 179620 + }, + "data": { + "status": "created", + "id": "drive0" + }, + "event": "JOB_STATUS_CHANGE" + } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + <- { + "timestamp": {...}, + "data": { + "status": "concluded", + "id": "drive0" + }, + "event": "JOB_STATUS_CHANGE" + } + + <- { + "timestamp": {...}, + "data": { + "status": "null", + "id": "drive0" + }, + "event": "JOB_STATUS_CHANGE" + } + +A full explanation of the job transition semantics and the JOB_STATUS_CHAN= GE +event are beyond the scope of this document and will be omitted in all +subsequent examples; above, several more events have been omitted for brev= ity. + +.. note:: Subsequent examples will omit all events except BLOCK_JOB_COMPLE= TED + except where necessary to illustrate workflow differences. + + Omitted events and json objects will be represented by ellipses: + ``...`` + +Example: Resetting an Incremental Backup Anchor Point +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If we want to start a new backup chain with an existing bitmap, we can also +use a transaction to reset the bitmap while making a new full backup: =20 .. code:: json =20 - { "execute": "transaction", - "arguments": { - "actions": [ - {"type": "block-dirty-bitmap-clear", - "data": {"node": "drive0", "name": "bitmap0"} }, - {"type": "drive-backup", - "data": {"device": "drive0", "target": "/path/to/new_full_backu= p.img", - "sync": "full", "format": "qcow2"} } - ] - } - } - -Incremental Backups -------------------- - -The star of the show. - -**Nota Bene!** Only incremental backups of entire drives are supported -for now. So despite the fact that you can attach a bitmap to any -arbitrary node, they are only currently useful when attached to the root -node. This is because drive-backup only supports drives/devices instead -of arbitrary nodes. + -> { + "execute": "transaction", + "arguments": { + "actions": [ + { + "type": "block-dirty-bitmap-clear", + "data": { + "node": "drive0", + "name": "bitmap0" + } + }, + { + "type": "drive-backup", + "data": { + "device": "drive0", + "target": "/path/to/drive0.new_full.qcow2", + "sync": "full", + "format": "qcow2" + } + } + ] + } + } + + <- { "return": {} } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... + +The result of this example is identical to the first, but we clear an exis= ting +bitmap instead of adding a new one. + +.. tip:: In both of these examples, "bitmap0" is tied conceptually to the + creation of new, full backups. This relationship is not saved or + remembered by QEMU; it is up to the operator or management layer = to + remember which bitmaps are associated with which backups. =20 Example: First Incremental Backup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -1. Create a full backup and sync it to the dirty bitmap, as in the - transactional examples above; or with the VM offline, manually create - a full copy and then create a new bitmap before the VM begins - execution. +#. Create a full backup and sync it to a dirty bitmap using any method: =20 - - Let's assume the full backup is named ``full_backup.img``. - - Let's assume the bitmap you created is ``bitmap0`` attached to - ``drive0``. + - Either of the two live backup method demonstrated above, + - Using QMP commands with the VM paused as in the `Justification`_ sect= ion, + or + - With the VM offline, manually copy the image and start the VM in a pa= used + state, careful to add a new bitmap before the VM begins execution. =20 -2. Create a destination image for the incremental backup that utilizes - the full backup as a backing image. + Whichever method is chosen, let's assume that at the end of this step: =20 - - Let's assume the new incremental image is named - ``incremental.0.img``. + - The full backup is named ``drive0.full.qcow2``. + - The bitmap we created is named ``bitmap0``, attached to ``drive0``. + +#. Create a destination image for the incremental backup that utilizes the + full backup as a backing image. + + - Let's assume the new incremental image is named ``drive0.inc0.qcow2``: =20 .. code:: bash =20 - $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F = qcow2 + $ qemu-img create -f qcow2 drive0.inc0.qcow2 \ + -b drive0.full.qcow2 -F qcow2 =20 -3. Issue the incremental backup command: +#. Issue an incremental backup command: =20 .. code:: json =20 - { "execute": "drive-backup", + -> { + "execute": "drive-backup", "arguments": { "device": "drive0", "bitmap": "bitmap0", - "target": "incremental.0.img", + "target": "drive0.inc0.qcow2", "format": "qcow2", "sync": "incremental", "mode": "existing" } } =20 + <- { "return": {} } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... + +This copies any blocks modified since the full backup was created into the +``drive0.inc0.qcow2`` file. During the operation, ``bitmap0`` is marked +``+busy``. If the operation is successful, ``bitmap0`` will be cleared to +reflect the "incremental" backup regimen, which only copies out new changes +from each incremental backup. + +.. note:: Any new writes that occur after the backup operation starts do n= ot + get copied to the destination. The backup's "point in time" is w= hen + the backup starts, not when it ends. These writes are recorded i= n a + special bitmap that gets re-added to bitmap0 when the backup end= s so + that the next incremental backup can copy them out. + Example: Second Incremental Backup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -1. Create a new destination image for the incremental backup that points - to the previous one, e.g.: ``incremental.1.img`` +#. Create a new destination image for the incremental backup that points to + the previous one, e.g.: ``drive0.inc1.qcow2`` =20 .. code:: bash =20 - $ qemu-img create -f qcow2 incremental.1.img -b incremental.0.img -= F qcow2 + $ qemu-img create -f qcow2 drive0.inc1.qcow2 \ + -b drive0.inc0.qcow2 -F qcow2 =20 -2. Issue a new incremental backup command. The only difference here is - that we have changed the target image below. +#. Issue a new incremental backup command. The only difference here is tha= t we + have changed the target image below. =20 .. code:: json =20 - { "execute": "drive-backup", + -> { + "execute": "drive-backup", "arguments": { "device": "drive0", "bitmap": "bitmap0", - "target": "incremental.1.img", + "target": "drive0.inc1.qcow2", "format": "qcow2", "sync": "incremental", "mode": "existing" } } =20 -Errors ------- + <- { "return": {} } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... + +Because the first incremental backup from the previous example completed +successfully, ``bitmap0`` was synchronized with ``drive0.inc0.qcow2``. Her= e, +we use ``bitmap0`` again to create a new incremental backup that targets t= he +previous one, creating a chain of three images: + +.. admonition:: Diagram + + .. code:: text + + +-------------------+ +-------------------+ +-------------------+ + | drive0.full.qcow2 |<--| drive0.inc0.qcow2 |<--| drive0.inc1.qcow2 | + +-------------------+ +-------------------+ +-------------------+ + +Each new incremental backup re-synchronizes the bitmap to the latest backup +authored, allowing a user to continue to "consume" it to create new backup= s on +top of an existing chain. + +In the above diagram, neither drive0.inc1.qcow2 nor drive0.inc0.qcow2 are +complete images by themselves, but rely on their backing chain to reconstr= uct +a full image. The dependency terminates with each full backup. + +Each backup in this chain remains independent, and is unchanged by new ent= ries +made later in the chain. For instance, drive0.inc0.qcow2 remains a perfect= ly +valid backup of the disk as it was when that backup was issued. + +Example: Incremental Push Backups without Backing Files +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Backup images are best kept off-site, so we often will not have the preced= ing +backups in a chain available to link against. This is not a problem at bac= kup +time; we simply do not set the backing image when creating the destination +image: + +#. Create a new destination image with no backing file set. We will need to + specify the size of the base image, because the backing file isn't + available for QEMU to use to determine it. + + .. code:: bash + + $ qemu-img create -f qcow2 drive0.inc2.qcow2 64G + + .. note:: Alternatively, you can omit ``mode: "existing"`` from the push + backup commands to have QEMU create an image without a backing + file for you, but you lose control over format options like + compatibility and preallocation presets. + +#. Issue a new incremental backup command. Apart from the new destination + image, there is no difference from the last two examples. + + .. code:: json + + -> { + "execute": "drive-backup", + "arguments": { + "device": "drive0", + "bitmap": "bitmap0", + "target": "drive0.inc2.qcow2", + "format": "qcow2", + "sync": "incremental", + "mode": "existing" + } + } + + <- { "return": {} } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... + +The only difference from the perspective of the user is that you will need= to +set the backing image when attempting to restore the backup: + +.. code:: bash + + $ qemu-img rebase drive0.inc2.qcow2 \ + -u -b drive0.inc1.qcow2 + +This uses the "unsafe" rebase mode to simply set the backing file to a file +that isn't present. + +It is also possible to use ``--image-opts`` to specify the entire backing +chain by hand as an ephemeral property at runtime, but that is beyond the +scope of this document. + +Example: Multi-drive Incremental Backup +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Assume we have a VM with two drives, "drive0" and "drive1" and we wish to = back +both of them up such that the two backups represent the same crash-consist= ent +point in time. + +#. For each drive, create an empty image: + + .. code:: bash + + $ qemu-img create -f qcow2 drive0.full.qcow2 64G + $ qemu-img create -f qcow2 drive1.full.qcow2 64G + +#. Create a full (anchor) backup for each drive, with accompanying bitmaps: + + .. code:: json + + -> { + "execute": "transaction", + "arguments": { + "actions": [ + { + "type": "block-dirty-bitmap-add", + "data": { + "node": "drive0", + "name": "bitmap0" + } + }, + { + "type": "block-dirty-bitmap-add", + "data": { + "node": "drive1", + "name": "bitmap0" + } + }, + { + "type": "drive-backup", + "data": { + "device": "drive0", + "target": "/path/to/drive0.full.qcow2", + "sync": "full", + "format": "qcow2" + } + }, + { + "type": "drive-backup", + "data": { + "device": "drive1", + "target": "/path/to/drive1.full.qcow2", + "sync": "full", + "format": "qcow2" + } + } + ] + } + } + + <- { "return": {} } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive1", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... + +#. Later, create new destination images for each of the incremental backups + that point to their respective full backups: + + .. code:: bash + + $ qemu-img create -f qcow2 drive0.inc0.qcow2 \ + -b drive0.full.qcow2 -F qcow2 + $ qemu-img create -f qcow2 drive1.inc0.qcow2 \ + -b drive1.full.qcow2 -F qcow2 + +#. Issue a multi-drive incremental push backup transaction: + + .. code:: json + + -> { + "execute": "transaction", + "arguments": { + "actions": [ + { + "type": "drive-backup", + "data": { + "device": "drive0", + "bitmap": "bitmap0", + "format": "qcow2", + "mode": "existing", + "sync": "incremental", + "target": "drive0.inc0.qcow2" + } + }, + { + "type": "drive-backup", + "data": { + "device": "drive1", + "bitmap": "bitmap0", + "format": "qcow2", + "mode": "existing", + "sync": "incremental", + "target": "drive1.inc0.qcow2" + } + }, + ] + } + } + + <- { "return": {} } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... + + <- { + "timestamp": {...}, + "data": { + "device": "drive1", + "type": "backup", + "speed": 0, + "len": 68719476736, + "offset": 68719476736 + }, + "event": "BLOCK_JOB_COMPLETED" + } + + ... =20 -- In the event of an error that occurs after a backup job is - successfully launched, either by a direct QMP command or a QMP - transaction, the user will receive a ``BLOCK_JOB_COMPLETE`` event with - a failure message, accompanied by a ``BLOCK_JOB_ERROR`` event. +Push Backup Errors & Recovery +----------------------------- =20 -- In the case of an event being cancelled, the user will receive a - ``BLOCK_JOB_CANCELLED`` event instead of a pair of COMPLETE and ERROR - events. +In the event of an error that occurs after a push backup job is successful= ly +launched, either by an individual QMP command or a QMP transaction, the us= er +will receive a ``BLOCK_JOB_COMPLETE`` event with a failure message, +accompanied by a ``BLOCK_JOB_ERROR`` event. =20 -- In either case, the incremental backup data contained within the - bitmap is safely rolled back, and the data within the bitmap is not - lost. The image file created for the failed attempt can be safely - deleted. +In the case of a job being cancelled, the user will receive a +``BLOCK_JOB_CANCELLED`` event instead of a pair of COMPLETE and ERROR +events. =20 -- Once the underlying problem is fixed (e.g. more storage space is - freed up), you can simply retry the incremental backup command with - the same bitmap. +In either failure case, the bitmap used for the failed operation is not +cleared. It will contain all of the dirty bits it did at the start of the +operation, plus any new bits that got marked during the operation. =20 -Example -~~~~~~~ +Effectively, the "point in time" that a bitmap is recording differences +against is kept at the issuance of the last successful incremental backup, +instead of being moved forward to the start of this now-failed backup. =20 -1. Create a target image: +Once the underlying problem is addressed (e.g. more storage space is alloc= ated +on the destination), the incremental backup command can be retried with the +same bitmap. + +Example: Individual Failures +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Incremental Push Backup jobs that fail individually behave simply as +described above. This example demonstrates the single-job failure case: + +#. Create a target image: =20 .. code:: bash =20 - $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F = qcow2 + $ qemu-img create -f qcow2 drive0.inc0.qcow2 \ + -b drive0.full.qcow2 -F qcow2 =20 -2. Attempt to create an incremental backup via QMP: +#. Attempt to create an incremental backup via QMP: =20 .. code:: json =20 - { "execute": "drive-backup", + -> { + "execute": "drive-backup", "arguments": { "device": "drive0", "bitmap": "bitmap0", - "target": "incremental.0.img", + "target": "drive0.inc0.qcow2", "format": "qcow2", "sync": "incremental", "mode": "existing" } } =20 -3. Receive an event notifying us of failure: + <- { "return": {} } + +#. Receive a pair of events indicating failure: =20 .. code:: json =20 - { "timestamp": { "seconds": 1424709442, "microseconds": 844524 }, - "data": { "speed": 0, "offset": 0, "len": 67108864, - "error": "No space left on device", - "device": "drive1", "type": "backup" }, - "event": "BLOCK_JOB_COMPLETED" } + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "action": "report", + "operation": "write" + }, + "event": "BLOCK_JOB_ERROR" + } + + <- { + "timestamp": {...}, + "data": { + "speed": 0, + "offset": 0, + "len": 67108864, + "error": "No space left on device", + "device": "drive0", + "type": "backup" + }, + "event": "BLOCK_JOB_COMPLETED" + } =20 -4. Delete the failed incremental, and re-create the image. +#. Delete the failed image, and re-create it. =20 .. code:: bash =20 - $ rm incremental.0.img - $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F = qcow2 + $ rm drive0.inc0.qcow2 + $ qemu-img create -f qcow2 drive0.inc0.qcow2 \ + -b drive0.full.qcow2 -F qcow2 =20 -5. Retry the command after fixing the underlying problem, such as +#. Retry the command after fixing the underlying problem, such as freeing up space on the backup volume: =20 .. code:: json =20 - { "execute": "drive-backup", + -> { + "execute": "drive-backup", "arguments": { "device": "drive0", "bitmap": "bitmap0", - "target": "incremental.0.img", + "target": "drive0.inc0.qcow2", "format": "qcow2", "sync": "incremental", "mode": "existing" } } =20 -6. Receive confirmation that the job completed successfully: + <- { "return": {} } + +#. Receive confirmation that the job completed successfully: =20 .. code:: json =20 - { "timestamp": { "seconds": 1424709668, "microseconds": 526525 }, - "data": { "device": "drive1", "type": "backup", - "speed": 0, "len": 67108864, "offset": 67108864}, - "event": "BLOCK_JOB_COMPLETED" } + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 67108864, + "offset": 67108864 + }, + "event": "BLOCK_JOB_COMPLETED" + } =20 -Partial Transactional Failures -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Example: Partial Transactional Failures +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -- Sometimes, a transaction will succeed in launching and return - success, but then later the backup jobs themselves may fail. It is - possible that a management application may have to deal with a - partial backup failure after a successful transaction. +QMP commands like `drive-backup = `_ +conceptually only start a job, and so transactions containing these comman= ds +may succeed even if the job it created later fails. This might have surpri= sing +interactions with notions of how a "transaction" ought to behave. =20 -- If multiple backup jobs are specified in a single transaction, when - one of them fails, it will not interact with the other backup jobs in - any way. +This distinction means that on occasion, a transaction containing such job +launching commands may appear to succeed and return success, but later +individual jobs associated with the transaction may fail. It is possible t= hat +a management application may have to deal with a partial backup failure af= ter +a "successful" transaction. =20 -- The job(s) that succeeded will clear the dirty bitmap associated with - the operation, but the job(s) that failed will not. It is not "safe" - to delete any incremental backups that were created successfully in - this scenario, even though others failed. +If multiple backup jobs are specified in a single transaction, if one of t= hose +jobs fails, it will not interact with the other backup jobs in any way by +default. The job(s) that succeeded will clear the dirty bitmap associated = with +the operation, but the job(s) that failed will not. It is therefore not sa= fe +to delete any incremental backups that were created successfully in this +scenario, even though others failed. =20 -Example -^^^^^^^ +This example illustrates a transaction with two backup jobs, where one fai= ls +and one succeeds: =20 -- QMP example highlighting two backup jobs: +#. Issue the transaction to start a backup of both drives. =20 .. code:: json =20 - { "execute": "transaction", + -> { + "execute": "transaction", "arguments": { "actions": [ - { "type": "drive-backup", - "data": { "device": "drive0", "bitmap": "bitmap0", - "format": "qcow2", "mode": "existing", - "sync": "incremental", "target": "d0-incr-1.qcow2= " } }, - { "type": "drive-backup", - "data": { "device": "drive1", "bitmap": "bitmap1", - "format": "qcow2", "mode": "existing", - "sync": "incremental", "target": "d1-incr-1.qcow2= " } }, - ] + { + "type": "drive-backup", + "data": { + "device": "drive0", + "bitmap": "bitmap0", + "format": "qcow2", + "mode": "existing", + "sync": "incremental", + "target": "drive0.inc0.qcow2" + } + }, + { + "type": "drive-backup", + "data": { + "device": "drive1", + "bitmap": "bitmap0", + "format": "qcow2", + "mode": "existing", + "sync": "incremental", + "target": "drive1.inc0.qcow2" + } + }] } } =20 -- QMP example response, highlighting one success and one failure: +#. Receive notice that the Transaction was accepted, and jobs were + launched: =20 - - Acknowledgement that the Transaction was accepted and jobs were - launched: + .. code:: json =20 - .. code:: json + <- { "return": {} } =20 - { "return": {} } +#. Receive notice that the first job has completed: =20 - - Later, QEMU sends notice that the first job was completed: + .. code:: json =20 - .. code:: json + <- { + "timestamp": {...}, + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 67108864, + "offset": 67108864 + }, + "event": "BLOCK_JOB_COMPLETED" + } =20 - { "timestamp": { "seconds": 1447192343, "microseconds": 615698 }, - "data": { "device": "drive0", "type": "backup", - "speed": 0, "len": 67108864, "offset": 67108864 }, - "event": "BLOCK_JOB_COMPLETED" - } +#. Receive notice that the second job has failed: =20 - - Later yet, QEMU sends notice that the second job has failed: + .. code:: json + + <- { + "timestamp": {...}, + "data": { + "device": "drive1", + "action": "report", + "operation": "read" + }, + "event": "BLOCK_JOB_ERROR" + } + + ... + + <- { + "timestamp": {...}, + "data": { + "speed": 0, + "offset": 0, + "len": 67108864, + "error": "Input/output error", + "device": "drive1", + "type": "backup" + }, + "event": "BLOCK_JOB_COMPLETED" + } =20 - .. code:: json +At the conclusion of the above example, ``drive0.inc0.qcow2`` is valid and +must be kept, but ``drive1.inc0.qcow2`` is incomplete and should be +deleted. If a VM-wide incremental backup of all drives at a point-in-time = is +to be made, new backups for both drives will need to be made, taking into +account that a new incremental backup for drive0 needs to be based on top = of +``drive0.inc0.qcow2``. =20 - { "timestamp": { "seconds": 1447192399, "microseconds": 683015 }, - "data": { "device": "drive1", "action": "report", - "operation": "read" }, - "event": "BLOCK_JOB_ERROR" } +For this example, an incremental backup for ``drive0`` was created, but not +for ``drive1``. The last VM-wide crash-consistent backup that is available= in +this case is the full backup: =20 - .. code:: json +.. code:: text =20 - { "timestamp": { "seconds": 1447192399, "microseconds": - 685853 }, "data": { "speed": 0, "offset": 0, "len": 67108864, - "error": "Input/output error", "device": "drive1", "type": - "backup" }, "event": "BLOCK_JOB_COMPLETED" } + [drive0.full.qcow2] <-- [drive0.inc0.qcow2] + [drive1.full.qcow2] =20 -- In the above example, ``d0-incr-1.qcow2`` is valid and must be kept, - but ``d1-incr-1.qcow2`` is invalid and should be deleted. If a VM-wide - incremental backup of all drives at a point-in-time is to be made, - new backups for both drives will need to be made, taking into account - that a new incremental backup for drive0 needs to be based on top of - ``d0-incr-1.qcow2``. +To repair this, issue a new incremental backup across both drives. The res= ult +will be backup chains that resemble the following: =20 -Grouped Completion Mode -~~~~~~~~~~~~~~~~~~~~~~~ +.. code:: text =20 -- While jobs launched by transactions normally complete or fail on - their own, it is possible to instruct them to complete or fail - together as a group. + [drive0.full.qcow2] <-- [drive0.inc0.qcow2] <-- [drive0.inc1.qco= w2] + [drive1.full.qcow2] <-------------------------- [drive1.inc1.qco= w2] =20 -- QMP transactions take an optional properties structure that can - affect the semantics of the transaction. +Example: Grouped Completion Mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -- The "completion-mode" transaction property can be either "individual" - which is the default, legacy behavior described above, or "grouped," - a new behavior detailed below. +While jobs launched by transactions normally complete or fail individually, +it's possible to instruct them to complete or fail together as a group. QMP +transactions take an optional properties structure that can affect the +behavior of the transaction. =20 -- Delayed Completion: In grouped completion mode, no jobs will report - success until all jobs are ready to report success. +The ``completion-mode`` transaction property can be either ``individual`` +which is the default legacy behavior described above, or ``grouped``, deta= iled +below. =20 -- Grouped failure: If any job fails in grouped completion mode, all - remaining jobs will be cancelled. Any incremental backups will - restore their dirty bitmap objects as if no backup command was ever - issued. +In ``grouped`` completion mode, no jobs will report success until all jobs= are +ready to report success. If any job fails, all other jobs will be cancelle= d. =20 - - Regardless of if QEMU reports a particular incremental backup job - as CANCELLED or as an ERROR, the in-memory bitmap will be - restored. +Regardless of if a participating incremental backup job failed or was +cancelled, their associated bitmaps will all be held at their existing +points-in-time, as in individual failure cases. =20 -Example -^^^^^^^ +Here's the same multi-drive backup scenario from `Example: Partial +Transactional Failures`_, but with the ``grouped`` completion-mode property +applied: =20 -- Here's the same example scenario from above with the new property: +#. Issue the multi-drive incremental backup transaction: =20 .. code:: json =20 - { "execute": "transaction", + -> { + "execute": "transaction", "arguments": { - "actions": [ - { "type": "drive-backup", - "data": { "device": "drive0", "bitmap": "bitmap0", - "format": "qcow2", "mode": "existing", - "sync": "incremental", "target": "d0-incr-1.qcow2= " } }, - { "type": "drive-backup", - "data": { "device": "drive1", "bitmap": "bitmap1", - "format": "qcow2", "mode": "existing", - "sync": "incremental", "target": "d1-incr-1.qcow2= " } }, - ], "properties": { "completion-mode": "grouped" - } + }, + "actions": [ + { + "type": "drive-backup", + "data": { + "device": "drive0", + "bitmap": "bitmap0", + "format": "qcow2", + "mode": "existing", + "sync": "incremental", + "target": "drive0.inc0.qcow2" + } + }, + { + "type": "drive-backup", + "data": { + "device": "drive1", + "bitmap": "bitmap0", + "format": "qcow2", + "mode": "existing", + "sync": "incremental", + "target": "drive1.inc0.qcow2" + } + }] } } =20 -- QMP example response, highlighting a failure for ``drive2``: +#. Receive notice that the Transaction was accepted, and jobs were launche= d: =20 - - Acknowledgement that the Transaction was accepted and jobs were - launched: + .. code:: json + + <- { "return": {} } =20 - .. code:: json +#. Receive notification that the backup job for ``drive1`` has failed: =20 - { "return": {} } + .. code:: json =20 - - Later, QEMU sends notice that the second job has errored out, but - that the first job was also cancelled: + <- { + "timestamp": {...}, + "data": { + "device": "drive1", + "action": "report", + "operation": "read" + }, + "event": "BLOCK_JOB_ERROR" + } =20 - .. code:: json + <- { + "timestamp": {...}, + "data": { + "speed": 0, + "offset": 0, + "len": 67108864, + "error": "Input/output error", + "device": "drive1", + "type": "backup" + }, + "event": "BLOCK_JOB_COMPLETED" + } =20 - { "timestamp": { "seconds": 1447193702, "microseconds": 632377 }, - "data": { "device": "drive1", "action": "report", - "operation": "read" }, - "event": "BLOCK_JOB_ERROR" } +#. Receive notification that the job for ``drive0`` has been cancelled: =20 - .. code:: json + .. code:: json =20 - { "timestamp": { "seconds": 1447193702, "microseconds": 640074 }, - "data": { "speed": 0, "offset": 0, "len": 67108864, - "error": "Input/output error", - "device": "drive1", "type": "backup" }, - "event": "BLOCK_JOB_COMPLETED" } + <- { + "timestamp": {...} + "data": { + "device": "drive0", + "type": "backup", + "speed": 0, + "len": 67108864, + "offset": 16777216 + }, + "event": "BLOCK_JOB_CANCELLED" + } =20 - .. code:: json +At the conclusion of *this* example, both jobs have been aborted due to a +failure. Both destination images should be deleted and are no longer of us= e. =20 - { "timestamp": { "seconds": 1447193702, "microseconds": 640163 }, - "data": { "device": "drive0", "type": "backup", "speed": 0, - "len": 67108864, "offset": 16777216 }, - "event": "BLOCK_JOB_CANCELLED" } +The transaction as a whole can simply be re-issued at a later time. =20 .. raw:: html =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980406; cv=none; d=zoho.com; s=zohoarc; b=TAtt1IGNo7xSY9mlOuJ6Iu+YSKSRhk4CNUx0lgR7A0cQIWQVxZhSNMQhtfmimxTnCfi7CQrgOJYot2G1AMiEb1zgFWE8JNeAIEzVNnwAlU8MfbAxP8aKkZGgf92b/cGYOhWK68MHWYraGtoLHcyJhsExv9Nu+M/JLxFV11vOJ5g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980406; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=CaB7QMBvFweh6DvcooehnPZ7HUM115Vt6zqAyvCOzq8=; b=VVbJ2dL7Fu2QsJQuhliXciIL3hxFS9IE1Gx14GHDH7QvWkyG2bLUjjH5bo029dAfKAkJiX/TwLTtausKyKIkmfdWZuOB6btgcG2sqO8r15xkSp8olnec7PSXmTIKUcD/Jk+nnwr5dLkkEEqgAhyF9Ti6dpMei7sMXF0oo04W3Sc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569980406943643.4065695602936; Tue, 1 Oct 2019 18:40:06 -0700 (PDT) Received: from localhost ([::1]:50302 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTcZ-0001t8-JC for importer@patchew.org; Tue, 01 Oct 2019 21:40:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41958) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRtI-0002fE-2l for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRtG-0003cS-Gx for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:11 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:13148) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRtD-00034r-FJ; Tue, 01 Oct 2019 19:49:07 -0400 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NmY9E144416; Tue, 1 Oct 2019 19:48:34 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccr3f2e8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:33 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkFE006374; Tue, 1 Oct 2019 23:47:17 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye6x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:17 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlGwA36897024 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:16 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7E7C828066; Tue, 1 Oct 2019 23:47:16 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 61C0328058; Tue, 1 Oct 2019 23:47:16 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:16 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 10/97] spapr/xive: fix EQ page addresses above 64GB Date: Tue, 1 Oct 2019 18:44:49 -0500 Message-Id: <20191001234616.7825-11-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NmY9E144416 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Gibson , qemu-stable@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: C=C3=A9dric Le Goater The high order bits of the address of the OS event queue is stored in bits [4-31] of word2 of the XIVE END internal structures and the low order bits in word3. This structure is using Big Endian ordering and computing the value requires some simple arithmetic which happens to be wrong. The mask removing bits [0-3] of word2 is applied to the wrong value and the resulting address is bogus when above 64GB. Guests with more than 64GB of RAM will allocate pages for the OS event queues which will reside above the 64GB limit. In this case, the XIVE device model will wake up the CPUs in case of a notification, such as IPIs, but the update of the event queue will be written at the wrong place in memory. The result is uncertain as the guest memory is trashed and IPI are not delivered. Introduce a helper xive_end_qaddr() to compute this value correctly in all places where it is used. Signed-off-by: C=C3=A9dric Le Goater Message-Id: <20190508171946.657-3-clg@kaod.org> Reviewed-by: Greg Kurz Signed-off-by: David Gibson (cherry picked from commit 13df93244efbd4bb8b4cf4e26104a26033178674) Signed-off-by: Michael Roth --- hw/intc/spapr_xive.c | 3 +-- hw/intc/xive.c | 9 +++------ include/hw/ppc/xive_regs.h | 6 ++++++ 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 097f88d460..db75f5d608 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -1144,8 +1144,7 @@ static target_ulong h_int_get_queue_config(PowerPCCPU= *cpu, } =20 if (xive_end_is_enqueue(end)) { - args[1] =3D (uint64_t) be32_to_cpu(end->w2 & 0x0fffffff) << 32 - | be32_to_cpu(end->w3); + args[1] =3D xive_end_qaddr(end); args[2] =3D xive_get_field32(END_W0_QSIZE, end->w0) + 12; } else { args[1] =3D 0; diff --git a/hw/intc/xive.c b/hw/intc/xive.c index a0b87001da..dcf2fcd108 100644 --- a/hw/intc/xive.c +++ b/hw/intc/xive.c @@ -1042,8 +1042,7 @@ static const TypeInfo xive_source_info =3D { =20 void xive_end_queue_pic_print_info(XiveEND *end, uint32_t width, Monitor *= mon) { - uint64_t qaddr_base =3D (uint64_t) be32_to_cpu(end->w2 & 0x0fffffff) <= < 32 - | be32_to_cpu(end->w3); + uint64_t qaddr_base =3D xive_end_qaddr(end); uint32_t qsize =3D xive_get_field32(END_W0_QSIZE, end->w0); uint32_t qindex =3D xive_get_field32(END_W1_PAGE_OFF, end->w1); uint32_t qentries =3D 1 << (qsize + 10); @@ -1072,8 +1071,7 @@ void xive_end_queue_pic_print_info(XiveEND *end, uint= 32_t width, Monitor *mon) =20 void xive_end_pic_print_info(XiveEND *end, uint32_t end_idx, Monitor *mon) { - uint64_t qaddr_base =3D (uint64_t) be32_to_cpu(end->w2 & 0x0fffffff) <= < 32 - | be32_to_cpu(end->w3); + uint64_t qaddr_base =3D xive_end_qaddr(end); uint32_t qindex =3D xive_get_field32(END_W1_PAGE_OFF, end->w1); uint32_t qgen =3D xive_get_field32(END_W1_GENERATION, end->w1); uint32_t qsize =3D xive_get_field32(END_W0_QSIZE, end->w0); @@ -1101,8 +1099,7 @@ void xive_end_pic_print_info(XiveEND *end, uint32_t e= nd_idx, Monitor *mon) =20 static void xive_end_enqueue(XiveEND *end, uint32_t data) { - uint64_t qaddr_base =3D (uint64_t) be32_to_cpu(end->w2 & 0x0fffffff) <= < 32 - | be32_to_cpu(end->w3); + uint64_t qaddr_base =3D xive_end_qaddr(end); uint32_t qsize =3D xive_get_field32(END_W0_QSIZE, end->w0); uint32_t qindex =3D xive_get_field32(END_W1_PAGE_OFF, end->w1); uint32_t qgen =3D xive_get_field32(END_W1_GENERATION, end->w1); diff --git a/include/hw/ppc/xive_regs.h b/include/hw/ppc/xive_regs.h index bf36678a24..1a8c5b5e64 100644 --- a/include/hw/ppc/xive_regs.h +++ b/include/hw/ppc/xive_regs.h @@ -208,6 +208,12 @@ typedef struct XiveEND { #define xive_end_is_backlog(end) (be32_to_cpu((end)->w0) & END_W0_BACKLOG) #define xive_end_is_escalate(end) (be32_to_cpu((end)->w0) & END_W0_ESCALAT= E_CTL) =20 +static inline uint64_t xive_end_qaddr(XiveEND *end) +{ + return ((uint64_t) be32_to_cpu(end->w2) & 0x0fffffff) << 32 | + be32_to_cpu(end->w3); +} + /* Notification Virtual Target (NVT) */ typedef struct XiveNVT { uint32_t w0; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569973878; cv=none; d=zoho.com; s=zohoarc; b=HtSrxoMMRFDsNb76K8G92QKdp4JjWpT/pnCRGy8i0oege2O7z04L5NUBdDH6GNpBdFH4qDHmhYZP23YmbTfqgkULAC0StMLq49HPQEXTarPsfjmF1KWYXoqoDce68iZcshm9GP2PSH5aDbflGRB5W2jtL8ujwrDbH+V4S7yVVRI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569973878; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=lfUvHnLZCHF+4oQzrte3z5RUtJrR1uIXRS0QkfGgoqA=; b=gJMLJnUg5ktsoXCwCQvIi9uY7ZcpkQ//bKRGV/X7couTW4x2h98kMYNyy0oI2gMDwT9VO+xFSBsANqXa3JSMJLTLPp+ipvCPkTirbzQsOwCq4Hg9/Y2RvIg4QBUqKS9qANmXWurtFF1mvsq39ud9Ab7nAoKCL3NwR6TMvidkYjI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569973878602210.45165704629198; Tue, 1 Oct 2019 16:51:18 -0700 (PDT) Received: from localhost ([::1]:49404 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRv6-0003u0-IQ for importer@patchew.org; Tue, 01 Oct 2019 19:51:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38871) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrc-00017J-Ug for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001J9-2l for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:28 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:45310 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRra-0001HP-St; Tue, 01 Oct 2019 19:47:26 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPsT179315; Tue, 1 Oct 2019 19:47:25 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncpq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlP6a179311; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncmn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkDv027150; Tue, 1 Oct 2019 23:47:18 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bf1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:18 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlHrs55443852 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:17 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 89FDA2805E; Tue, 1 Oct 2019 23:47:17 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6E77028059; Tue, 1 Oct 2019 23:47:17 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:17 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 11/97] kbd-state: fix autorepeat handling Date: Tue, 1 Oct 2019 18:44:50 -0500 Message-Id: <20191001234616.7825-12-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Gerd Hoffmann Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Gerd Hoffmann When allowing multiple down-events in a row (key autorepeat) we can't use change_bit() any more to update the state, because autorepeat events don't change the key state. We have to explicitly use set_bit() and clear_bit() instead. Cc: qemu-stable@nongnu.org Fixes: 35921860156e kbd-state: don't block auto-repeat events Buglink: https://bugs.launchpad.net/qemu/+bug/1828272 Signed-off-by: Gerd Hoffmann Message-id: 20190514042443.10735-1-kraxel@redhat.com (cherry picked from commit 5fff13f245cddd3bc260dfe6ebe1b1f05b72116f) Signed-off-by: Michael Roth --- ui/kbd-state.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/ui/kbd-state.c b/ui/kbd-state.c index f3ab2d7a66..1668d17dda 100644 --- a/ui/kbd-state.c +++ b/ui/kbd-state.c @@ -59,7 +59,11 @@ void qkbd_state_key_event(QKbdState *kbd, QKeyCode qcode= , bool down) } =20 /* update key and modifier state */ - change_bit(qcode, kbd->keys); + if (down) { + set_bit(qcode, kbd->keys); + } else { + clear_bit(qcode, kbd->keys); + } switch (qcode) { case Q_KEY_CODE_SHIFT: case Q_KEY_CODE_SHIFT_R: --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974387; cv=none; d=zoho.com; s=zohoarc; b=Syey9nb+0BAD0we/RpeMTLHRZ+TJJce9JP4nfR32JeE103+a1pBxoGkveRiKn97seY+GT9EMSTzaF0GbbishXIIyR7DOyzXqylxjbF4aEWj4qnZhRhZ3XuiyciAMiHIn8/g/kXbd9IXtgIodd01NupiLAnyM0B/MpNFwlXExQ+Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974387; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=zyGdc90GwnVE5FAvU3EM10814sGfqyXr2eysqlJSBZY=; b=iO62CvcmdqhkrdXGfCLSEXyp2NlD2ZBWHVuXM17kEaDDFIkGQZpITON65pyw07Q67wFnI4KN85yN40uDhriWHs496WICeVE2w/u986d7sUt0fC/EBexrhZS5epMEMa36KRt+Q+KzJA6SukOh2EIqWPd8Z0zq9QNJuUHH5lS5j0c= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569974387654428.90797258765565; Tue, 1 Oct 2019 16:59:47 -0700 (PDT) Received: from localhost ([::1]:49508 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFS3U-0003Ly-Mk for importer@patchew.org; Tue, 01 Oct 2019 19:59:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38911) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017R-Ou for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrc-0001Kw-13 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:25098) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrb-0001Hh-AQ; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPU9074818; Tue, 1 Oct 2019 19:47:25 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vprf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlOqo074760; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpps-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:24 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njlm9032693; Tue, 1 Oct 2019 23:47:18 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma02dal.us.ibm.com with ESMTP id 2v9y57qb65-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:18 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlInf52429232 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:18 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 132622805E; Tue, 1 Oct 2019 23:47:18 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EBB1B28058; Tue, 1 Oct 2019 23:47:17 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:17 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 12/97] usb-tablet: fix serial compat property Date: Tue, 1 Oct 2019 18:44:51 -0500 Message-Id: <20191001234616.7825-13-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NlPU9074818 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Gerd Hoffmann Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Gerd Hoffmann s/kbd/tablet/, fixes cut+paste bug. Cc: qemu-stable@nongnu.org Reported-by: Dr. David Alan Gilbert Signed-off-by: Gerd Hoffmann Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-id: 20190520081805.15019-1-kraxel@redhat.com (cherry picked from commit 442bac16a6cd708a9f87adb0a263f9d833f03ed5) Signed-off-by: Michael Roth --- hw/core/machine.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index 743fef2898..b6766632d9 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -33,7 +33,7 @@ GlobalProperty hw_compat_3_1[] =3D { { "tpm-tis", "ppi", "false" }, { "usb-kbd", "serial", "42" }, { "usb-mouse", "serial", "42" }, - { "usb-kbd", "serial", "42" }, + { "usb-tablet", "serial", "42" }, { "virtio-blk-device", "discard", "false" }, { "virtio-blk-device", "write-zeroes", "false" }, }; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974791; cv=none; d=zoho.com; s=zohoarc; b=P/AD1oCCTiCU+FnpsU3UUQhL0Psc4y2EjGlj2Ae3S6eOxeCyHoBmstyBPnUe1veW6KDpcXdXXqR2rTpQOCl9jzA90GsQMcKdDqtUYNHNarlQ6bXxX/AXvlY6X9Azs99I7CnJreVkkZGekFKMNRqqYQkSHqXoyGVI7SungyVT2+c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974791; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=heOQlq1KWaUyiORCQzac29wqHTdRJGPnyorz6AdNppc=; b=jTGfDDiSWwhgJ+EUHUyx6JTXN23s6yPSTjWyinRspPinx8Pbu3pRvldxJchAyoxwPjq6jUw/lHJeEts75UlEta4EGPFycXYnrz0BTxCbVXJIvQRkxqnj3yAuiYqvCn8NOmDhqIp9zz4sHvFjxYkURnYy8KYa8VPRqDTxET0581Y= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997479150987.87684047370192; Tue, 1 Oct 2019 17:06:31 -0700 (PDT) Received: from localhost ([::1]:49550 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFS9w-00075V-Iq for importer@patchew.org; Tue, 01 Oct 2019 20:06:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39027) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrl-00018M-LW for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrc-0001LP-3o for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:33 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:56242 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrb-0001Iw-KB; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQEF078969; Tue, 1 Oct 2019 19:47:26 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00rx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlQ8Y078951; Tue, 1 Oct 2019 19:47:26 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00pu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njock031648; Tue, 1 Oct 2019 23:47:19 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fa8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:19 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlIUk48038200 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:18 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 95F0A28060; Tue, 1 Oct 2019 23:47:18 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 79CC628059; Tue, 1 Oct 2019 23:47:18 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:18 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 13/97] block/file-posix: Unaligned O_DIRECT block-status Date: Tue, 1 Oct 2019 18:44:52 -0500 Message-Id: <20191001234616.7825-14-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Currently, qemu crashes whenever someone queries the block status of an unaligned image tail of an O_DIRECT image: $ echo > foo $ qemu-img map --image-opts driver=3Dfile,filename=3Dfoo,cache.direct=3Don Offset Length Mapped to File qemu-img: block/io.c:2093: bdrv_co_block_status: Assertion `*pnum && QEMU_IS_ALIGNED(*pnum, align) && align > offset - aligned_offset' failed. This is because bdrv_co_block_status() checks that the result returned by the driver's implementation is aligned to the request_alignment, but file-posix can fail to do so, which is actually mentioned in a comment there: "[...] possibly including a partial sector at EOF". Fix this by rounding up those partial sectors. There are two possible alternative fixes: (1) We could refuse to open unaligned image files with O_DIRECT altogether. That sounds reasonable until you realize that qcow2 does necessarily not fill up its metadata clusters, and that nobody runs qemu-img create with O_DIRECT. Therefore, unpreallocated qcow2 files usually have an unaligned image tail. (2) bdrv_co_block_status() could ignore unaligned tails. It actually throws away everything past the EOF already, so that sounds reasonable. Unfortunately, the block layer knows file lengths only with a granularity of BDRV_SECTOR_SIZE, so bdrv_co_block_status() usually would have to guess whether its file length information is inexact or whether the driver is broken. Fixing what raw_co_block_status() returns is the safest thing to do. There seems to be no other block driver that sets request_alignment and does not make sure that it always returns aligned values. Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf (cherry picked from commit 9c3db310ff0b7473272ae8dce5e04e2f8a825390) Signed-off-by: Michael Roth --- block/file-posix.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/block/file-posix.c b/block/file-posix.c index 1cf4ee49eb..c185f34a2e 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -2475,6 +2475,8 @@ static int coroutine_fn raw_co_block_status(BlockDriv= erState *bs, off_t data =3D 0, hole =3D 0; int ret; =20 + assert(QEMU_IS_ALIGNED(offset | bytes, bs->bl.request_alignment)); + ret =3D fd_open(bs); if (ret < 0) { return ret; @@ -2500,6 +2502,20 @@ static int coroutine_fn raw_co_block_status(BlockDri= verState *bs, /* On a data extent, compute bytes to the end of the extent, * possibly including a partial sector at EOF. */ *pnum =3D MIN(bytes, hole - offset); + + /* + * We are not allowed to return partial sectors, though, so + * round up if necessary. + */ + if (!QEMU_IS_ALIGNED(*pnum, bs->bl.request_alignment)) { + int64_t file_length =3D raw_getlength(bs); + if (file_length > 0) { + /* Ignore errors, this is just a safeguard */ + assert(hole =3D=3D file_length); + } + *pnum =3D ROUND_UP(*pnum, bs->bl.request_alignment); + } + ret =3D BDRV_BLOCK_DATA; } else { /* On a hole, compute bytes to the beginning of the next extent. = */ --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569973876; cv=none; d=zoho.com; s=zohoarc; b=FCHQFvwUwVseajLCGV4Py6PEzhqVO6J+5CPUip7Q+G6h4M97O3D6BGY93JqfFzv7jUufLMg9/nGg0rgyke3nDGZ9N+n/6JiGXtvmFZS4I540Zp2+85M8hVubdf/sqiSTQw5mgly2sDnFoQ7iUJQ5FU0EQDExOShQ+OzHlcd6ukU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569973876; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=pyS7nRABsYMgZ8hIkbW5Q08TwDZu2fVq48TI5f6/IEw=; b=RNHLGV+TpjUpqL/C+8s+riwIWx5AF5f3xdtDVbmo+RF2kknN0TtAmIB72gHK4F5ROxacNWMOQwVtmy7DBKGJP5yvQhZt/uY3su04XhfHJyjg8jydEq+jh3A/wkkfh5x5wTTZ7SGfTGWK8wX8dOMP1spG8l6mnQcLwEZWjCu7oW0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997387638439.436825225647794; Tue, 1 Oct 2019 16:51:16 -0700 (PDT) Received: from localhost ([::1]:49408 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRvF-00045c-9K for importer@patchew.org; Tue, 01 Oct 2019 19:51:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38891) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017L-99 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001JH-3X for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:28 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:37212 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRra-0001IH-TL; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPVu179319; Tue, 1 Oct 2019 19:47:26 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncq1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlPcJ179290; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncn2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkJo027153; Tue, 1 Oct 2019 23:47:20 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bf4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:19 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlJg944564960 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:19 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2A18B28059; Tue, 1 Oct 2019 23:47:19 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 01BBF28058; Tue, 1 Oct 2019 23:47:19 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:18 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 14/97] iotests: Test unaligned raw images with O_DIRECT Date: Tue, 1 Oct 2019 18:44:53 -0500 Message-Id: <20191001234616.7825-15-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz We already have 221 for accesses through the page cache, but it is better to create a new file for O_DIRECT instead of integrating those test cases into 221. This way, we can make use of _supported_cache_modes (and _default_cache_mode) so the test is automatically skipped on filesystems that do not support O_DIRECT. As part of the split, add _supported_cache_modes to 221. With that, it no longer fails when run with -c none or -c directsync. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf (cherry picked from commit 2fab30c80b33cdc6157c7efe6207e54b6835cf92) Conflicts: tests/qemu-iotests/group *fix context deps on test groups not in 4.0 Signed-off-by: Michael Roth --- tests/qemu-iotests/221 | 4 ++ tests/qemu-iotests/253 | 84 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/253.out | 14 +++++++ tests/qemu-iotests/group | 1 + 4 files changed, 103 insertions(+) create mode 100755 tests/qemu-iotests/253 create mode 100644 tests/qemu-iotests/253.out diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 index 808cd9a289..92c9b13cd8 100755 --- a/tests/qemu-iotests/221 +++ b/tests/qemu-iotests/221 @@ -1,6 +1,7 @@ #!/usr/bin/env bash # # Test qemu-img vs. unaligned images +# (See also 253, which is the O_DIRECT version) # # Copyright (C) 2018 Red Hat, Inc. # @@ -37,6 +38,9 @@ _supported_fmt raw _supported_proto file _supported_os Linux =20 +_default_cache_mode writeback +_supported_cache_modes writeback writethrough unsafe + echo echo "=3D=3D=3D Check mapping of unaligned raw image =3D=3D=3D" echo diff --git a/tests/qemu-iotests/253 b/tests/qemu-iotests/253 new file mode 100755 index 0000000000..d88d5afa45 --- /dev/null +++ b/tests/qemu-iotests/253 @@ -0,0 +1,84 @@ +#!/usr/bin/env bash +# +# Test qemu-img vs. unaligned images; O_DIRECT version +# (Originates from 221) +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +seq=3D"$(basename $0)" +echo "QA output created by $seq" + +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt raw +_supported_proto file +_supported_os Linux + +_default_cache_mode none +_supported_cache_modes none directsync + +echo +echo "=3D=3D=3D Check mapping of unaligned raw image =3D=3D=3D" +echo + +# We do not know how large a physical sector is, but it is certainly +# going to be a factor of 1 MB +size=3D$((1 * 1024 * 1024 - 1)) + +# qemu-img create rounds size up to BDRV_SECTOR_SIZE +_make_test_img $size +$QEMU_IMG map --output=3Djson --image-opts \ + "driver=3D$IMGFMT,file.driver=3Dfile,file.filename=3D$TEST_IMG,cache.d= irect=3Don" \ + | _filter_qemu_img_map + +# so we resize it and check again +truncate --size=3D$size "$TEST_IMG" +$QEMU_IMG map --output=3Djson --image-opts \ + "driver=3D$IMGFMT,file.driver=3Dfile,file.filename=3D$TEST_IMG,cache.d= irect=3Don" \ + | _filter_qemu_img_map + +# qemu-io with O_DIRECT always writes whole physical sectors. Again, +# we do not know how large a physical sector is, so we just start +# writing from a 64 kB boundary, which should always be aligned. +offset=3D$((1 * 1024 * 1024 - 64 * 1024)) +$QEMU_IO -c "w $offset $((size - offset))" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=3Djson --image-opts \ + "driver=3D$IMGFMT,file.driver=3Dfile,file.filename=3D$TEST_IMG,cache.d= irect=3Don" \ + | _filter_qemu_img_map + +# Resize it and check again -- contrary to 221, we may not get partial +# sectors here, so there should be only two areas (one zero, one +# data). +truncate --size=3D$size "$TEST_IMG" +$QEMU_IMG map --output=3Djson --image-opts \ + "driver=3D$IMGFMT,file.driver=3Dfile,file.filename=3D$TEST_IMG,cache.d= irect=3Don" \ + | _filter_qemu_img_map + +# success, all done +echo '*** done' +rm -f $seq.full +status=3D0 diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out new file mode 100644 index 0000000000..607c0baa0b --- /dev/null +++ b/tests/qemu-iotests/253.out @@ -0,0 +1,14 @@ +QA output created by 253 + +=3D=3D=3D Check mapping of unaligned raw image =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1048575 +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false,= "offset": OFFSET}] +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false,= "offset": OFFSET}] +wrote 65535/65535 bytes at offset 983040 +63.999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, = "offset": OFFSET}, +{ "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": tru= e, "offset": OFFSET}] +[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, = "offset": OFFSET}, +{ "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": tru= e, "offset": OFFSET}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index bae7718380..4cd2fe80a6 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -248,3 +248,4 @@ 246 rw auto quick 247 rw auto quick 248 rw auto quick +253 rw auto quick --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974137; cv=none; d=zoho.com; s=zohoarc; b=iYg8UR/OXtZ3ZoMjHmGsh3vuCim9ncIeRxKlcVWn7zORmVEZ8NiHiyO2hq9Zjf8qVY8tfKsOerbySehIZpx8fNhY5yYKqO+nuSjUtbllyi7Zy3Q2/txijTrhTSRXoxMiqlBOoJAEsUj6X+QO15j0sAas2z5sNDvjS9fp4VGRXUA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974137; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=plNBFiqHP4Dy+BMT7eTGYnzOdQu54ONIO4OsYGCqCJw=; b=e86h1Qokg9yYp8Nvt4MQO+Ihs+qyNEYJ6/5D6fxDcpceELLlYUnXy7HihqkQJA3BbXwlzSoooChH/Nyg0M4Zpqv7gPQq3q9S/2SAJhqx6dEwM5jr49F5kkuon+sU0v5Um0/iESTrT5HgkB3k+V2IqQr1ltjJb3JwndDMvXaRqCg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569974137584521.7624706083947; Tue, 1 Oct 2019 16:55:37 -0700 (PDT) Received: from localhost ([::1]:49432 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRzS-0007qB-6g for importer@patchew.org; Tue, 01 Oct 2019 19:55:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38907) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017O-OO for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001Kx-Vl for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:29 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:4656 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrb-0001Iq-F5; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQET078956; Tue, 1 Oct 2019 19:47:26 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00rw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlQQP078950; Tue, 1 Oct 2019 19:47:26 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00q4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk06006371; Tue, 1 Oct 2019 23:47:20 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye73-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:20 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlJqO44564964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:19 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A9B902805A; Tue, 1 Oct 2019 23:47:19 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8CE4A28058; Tue, 1 Oct 2019 23:47:19 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:19 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 15/97] s390x/cpumodel: ignore csske for expansion Date: Tue, 1 Oct 2019 18:44:54 -0500 Message-Id: <20191001234616.7825-16-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Christian Borntraeger , Cornelia Huck , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Christian Borntraeger csske will be removed in a future machine. Ignore it for expanding the cpu model. Otherwise qemu falls back to z9. Signed-off-by: Christian Borntraeger Cc: qemu-stable@nongnu.org Reviewed-by: David Hildenbrand Message-Id: <20190429090250.7648-3-borntraeger@de.ibm.com> Signed-off-by: Cornelia Huck (cherry picked from commit eaf6f642abf1d4d24791b70728d4068428fc4658) Signed-off-by: Michael Roth --- target/s390x/cpu_models.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c index eb125d4d0d..e154d53eda 100644 --- a/target/s390x/cpu_models.c +++ b/target/s390x/cpu_models.c @@ -1327,6 +1327,8 @@ static void init_ignored_base_feat(void) S390_FEAT_KM_TDEA_192, S390_FEAT_KIMD_SHA_1, S390_FEAT_KLMD_SHA_1, + /* CSSKE is deprecated on newer generations */ + S390_FEAT_CONDITIONAL_SSKE, }; int i; =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974385; cv=none; d=zoho.com; s=zohoarc; b=Jn1U91+Ag8eJPSRtl4HaCtsUrE5cXLvNgIexM1P+16bDEcZVFXssmfLz06xeFKh2jUTAc4h7SSRE4A3qMNaq7Myff1TXRahgaZM2eXxLzK4ujvgEc0bNgFqlVW1eiyRzkUtrQRPJvo1srDHzVtikL9c1shSmOOGO7Vt+Obyk62E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974385; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=AAwvrUmwx2t+K5JqOG502Z5iEpnnhLmmhpgPV307fhU=; b=Gdk71L21dy8RkO1nLhgNjiKudxtaDATAzv7tmduDjVqFIm8tLtPfmHgZmoJVIwnGUnjlTTi7TAZuNzCYIPO6waDjIKVn6vnrYiwI8QEHVRFjymZhQXX2ZdwrF1NYPtrs4tl8R78ZM296+C3XNbIs7mltXIyiy9Yqxs3aycYtrCc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569974385358204.50025295860928; Tue, 1 Oct 2019 16:59:45 -0700 (PDT) Received: from localhost ([::1]:49504 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFS3U-0003Kl-3V for importer@patchew.org; Tue, 01 Oct 2019 19:59:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38896) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017M-G4 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001Jt-FZ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:28 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:28734) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRra-0001Hd-UH; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOD8053194; Tue, 1 Oct 2019 19:47:25 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17160q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlOQK053212; Tue, 1 Oct 2019 19:47:24 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1715yp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:24 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjnoZ032172; Tue, 1 Oct 2019 23:47:22 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma01wdc.us.ibm.com with ESMTP id 2v9y57fga0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:22 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlKfE16122634 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:20 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3039928059; Tue, 1 Oct 2019 23:47:20 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 14DE528058; Tue, 1 Oct 2019 23:47:20 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:20 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 16/97] blockdev-backup: don't check aio_context too early Date: Tue, 1 Oct 2019 18:44:55 -0500 Message-Id: <20191001234616.7825-17-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow in blockdev_backup_prepare, we check to make sure that the target is associated with a compatible aio context. However, do_blockdev_backup is called later and has some logic to move the target to a compatible aio_context. The transaction version will fail certain commands needlessly early as a result. Allow blockdev_backup_prepare to simply call do_blockdev_backup, which will ultimately decide if the contexts are compatible or not. Note: the transaction version has always disallowed this operation since its initial commit bd8baecd (2014), whereas the version of qmp_blockdev_backup at the time, from commit c29c1dd312f, tried to enforce the aio_context switch instead. It's not clear, and I can't see from the mailing list archives at the time, why the two functions take a different approach. It wasn't until later in efd7556708b (2016) that the standalone version tried to determine if it could set the context or not. Reported-by: aihua liang Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=3D1683498 Signed-off-by: John Snow Message-id: 20190523170643.20794-2-jsnow@redhat.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz (cherry picked from commit d81e1efbea7d19c2f142d300df56538c73800590) Signed-off-by: Michael Roth --- blockdev.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index 4775a07d93..d358169995 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1871,10 +1871,6 @@ static void blockdev_backup_prepare(BlkActionState *= common, Error **errp) } =20 aio_context =3D bdrv_get_aio_context(bs); - if (aio_context !=3D bdrv_get_aio_context(target)) { - error_setg(errp, "Backup between two IO threads is not implemented= "); - return; - } aio_context_acquire(aio_context); state->bs =3D bs; =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979940; cv=none; d=zoho.com; s=zohoarc; b=cPeJ4rCYJbbLIa4YOBAUBdxnhHQV6PKaG8SaBW/26ZhFdKMws6KnAOiqihYB8D+QCboezix0jN5f1GglBYFc0DxWpMB1lz1JkgW0/mnU4iA2lla2aAjaInQvvUmRw291j71TYbhx9a3BKRWBSeLWTd/PRNKbz1Kz2PAKVn5DdA8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979940; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=noRUb0YJQ+OD8JwOhV/Mh8hos4EL+PWEOhDeNCz7TeA=; b=DMRegI4OFB1yc0SufjUa40CHKng2/ldnvcqr5J+BcRgzNUsji2jSSwLach/LND2/u2g0lhkLy1YzarxLK61R9UB12el60UIxo9WPpNGDG+a/hBAy+KlAibXjAG6MfbV67oK0WTsGJRNg/HSYjG6Lq5qBVPkRQ3W6ryLRD9PJHCI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979940292440.15142848281073; Tue, 1 Oct 2019 18:32:20 -0700 (PDT) Received: from localhost ([::1]:50206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTV4-0000wn-GG for importer@patchew.org; Tue, 01 Oct 2019 21:32:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40659) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsc-00021T-7X for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002aU-Of for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:29 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:38014) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsV-0001Fk-Sw; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlNdj193539; Tue, 1 Oct 2019 19:47:23 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsfxp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:23 -0400 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlMnJ193520; Tue, 1 Oct 2019 19:47:22 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsfx8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:22 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk07006371; Tue, 1 Oct 2019 23:47:21 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye74-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:21 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlKPb12190534 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:20 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A903E28059; Tue, 1 Oct 2019 23:47:20 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8D16628058; Tue, 1 Oct 2019 23:47:20 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:20 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 17/97] block: Drain source node in bdrv_replace_node() Date: Tue, 1 Oct 2019 18:44:56 -0500 Message-Id: <20191001234616.7825-18-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Kevin Wolf Instead of just asserting that no requests are in flight in bdrv_replace_node(), which is a requirement that most callers ignore, we can just drain the source node right there. This fixes at least starting a commit job while I/O is active on the backing chain, but probably other callers, too. Having requests in flight on the target node isn't a problem because the target just gets new parents, but the call path of running requests isn't modified. So we can just drop this assertion without a replacement. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=3D1711643 Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz (cherry picked from commit f871abd60f4b67547e62c57c9bec19420052be39) *prereq for d81e1efb tests Signed-off-by: Michael Roth --- block.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/block.c b/block.c index 9ae5c0ed2f..c0e1c86911 100644 --- a/block.c +++ b/block.c @@ -3987,13 +3987,13 @@ void bdrv_replace_node(BlockDriverState *from, Bloc= kDriverState *to, uint64_t perm =3D 0, shared =3D BLK_PERM_ALL; int ret; =20 - assert(!atomic_read(&from->in_flight)); - assert(!atomic_read(&to->in_flight)); - /* Make sure that @from doesn't go away until we have successfully att= ached * all of its parents to @to. */ bdrv_ref(from); =20 + assert(qemu_get_current_aio_context() =3D=3D qemu_get_aio_context()); + bdrv_drained_begin(from); + /* Put all parents into @list and calculate their cumulative permissio= ns */ QLIST_FOREACH_SAFE(c, &from->parents, next_parent, next) { assert(c->bs =3D=3D from); @@ -4034,6 +4034,7 @@ void bdrv_replace_node(BlockDriverState *from, BlockD= riverState *to, =20 out: g_slist_free(list); + bdrv_drained_end(from); bdrv_unref(from); } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974387; cv=none; d=zoho.com; s=zohoarc; b=PTiBxPNjSXFH02ll2ZPOHkAAvSFgEwuxJFZsAoBCR8GbYc01mywut9reBJoWAlivlcbKW25BvS6981+jPHsVlF7lb6yR9ATmtJnwBcFdu4gYtWsZ2f9hch33lqdw6cWTg1V/tpdD/KnYBNa3OwZHf9J5vTVzGaIvbC7lgBUuPtE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974387; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=sad90od1dyk9EWI+SJYhPoEsQQ8cSZ/SnlOT3yVBoLU=; b=BEOsY8oahBY75R4M/tIbNOpPV/5bP0lmzX0dkJDdUbwlLuNaZQ/l2jmByGOBcZ/7Tlp+sNrS4/olshpEZjbR5XbNSS7sdLBwyjf4aiT+heyok055q8N/P9m8+SSwY0UipPVv8feplXO+z3y1Y8dUE7m6i3Z7GShR0hmwcBMoLrc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569974387728852.9542341786462; Tue, 1 Oct 2019 16:59:47 -0700 (PDT) Received: from localhost ([::1]:49507 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFS3W-0003Lr-L8 for importer@patchew.org; Tue, 01 Oct 2019 19:59:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38966) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrj-00018C-0K for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001Js-FW for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:42112) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRra-0001He-U6; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOat053387; Tue, 1 Oct 2019 19:47:24 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17160g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:24 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlO5Q053078; Tue, 1 Oct 2019 19:47:24 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg171600-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:24 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njlt0032690; Tue, 1 Oct 2019 23:47:22 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma02dal.us.ibm.com with ESMTP id 2v9y57qb69-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:22 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlLLu44302728 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:21 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 306A728059; Tue, 1 Oct 2019 23:47:21 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 14D3828058; Tue, 1 Oct 2019 23:47:21 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:21 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 18/97] iotests: Test commit job start with concurrent I/O Date: Tue, 1 Oct 2019 18:44:57 -0500 Message-Id: <20191001234616.7825-19-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Kevin Wolf This tests that concurrent requests are correctly drained before making graph modifications instead of running into assertions in bdrv_replace_node(). Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz (cherry picked from commit ac6fb43eae1f5029b51e0a3d975fe2111cc8b976) Conflicts: tests/qemu-iotests/group *prereq for d81e1efb tests Signed-off-by: Michael Roth --- tests/qemu-iotests/255 | 83 +++++++++++++++++++++++++++++++++++ tests/qemu-iotests/255.out | 16 +++++++ tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 10 ++++- 4 files changed, 109 insertions(+), 1 deletion(-) create mode 100755 tests/qemu-iotests/255 create mode 100644 tests/qemu-iotests/255.out diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255 new file mode 100755 index 0000000000..c0bb37a9b0 --- /dev/null +++ b/tests/qemu-iotests/255 @@ -0,0 +1,83 @@ +#!/usr/bin/env python +# +# Test commit job graph modifications while requests are active +# +# Copyright (C) 2019 Red Hat, Inc. +# +# Creator/Owner: Kevin Wolf +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import iotests +from iotests import imgfmt + +iotests.verify_image_format(supported_fmts=3D['qcow2']) + +def blockdev_create(vm, options): + result =3D vm.qmp_log('blockdev-create', + filters=3D[iotests.filter_qmp_testfiles], + job_id=3D'job0', options=3Doptions) + + if 'return' in result: + assert result['return'] =3D=3D {} + vm.run_job('job0') + iotests.log("") + +with iotests.FilePath('t.qcow2') as disk_path, \ + iotests.FilePath('t.qcow2.mid') as mid_path, \ + iotests.FilePath('t.qcow2.base') as base_path, \ + iotests.VM() as vm: + + iotests.log("=3D=3D=3D Create backing chain and start VM =3D=3D=3D") + iotests.log("") + + size =3D 128 * 1024 * 1024 + size_str =3D str(size) + + iotests.create_image(base_path, size) + iotests.qemu_img_log('create', '-f', iotests.imgfmt, mid_path, size_st= r) + iotests.qemu_img_log('create', '-f', iotests.imgfmt, disk_path, size_s= tr) + + # Create a backing chain like this: + # base <- [throttled: bps-read=3D4096] <- mid <- overlay + + vm.add_object('throttle-group,x-bps-read=3D4096,id=3Dthrottle0') + vm.add_blockdev('file,filename=3D%s,node-name=3Dbase' % (base_path)) + vm.add_blockdev('throttle,throttle-group=3Dthrottle0,file=3Dbase,node-= name=3Dthrottled') + vm.add_blockdev('file,filename=3D%s,node-name=3Dmid-file' % (mid_path)) + vm.add_blockdev('qcow2,file=3Dmid-file,node-name=3Dmid,backing=3Dthrot= tled') + vm.add_drive_raw('if=3Dnone,id=3Doverlay,driver=3Dqcow2,file=3D%s,back= ing=3Dmid' % (disk_path)) + + vm.launch() + + iotests.log("=3D=3D=3D Start background read requests =3D=3D=3D") + iotests.log("") + + def start_requests(): + vm.hmp_qemu_io('overlay', 'aio_read 0 4k') + vm.hmp_qemu_io('overlay', 'aio_read 0 4k') + + start_requests() + + iotests.log("=3D=3D=3D Run a commit job =3D=3D=3D") + iotests.log("") + + result =3D vm.qmp_log('block-commit', job_id=3D'job0', auto_finalize= =3DFalse, + device=3D'overlay', top_node=3D'mid') + + vm.run_job('job0', auto_finalize=3DFalse, pre_finalize=3Dstart_request= s, + auto_dismiss=3DTrue) + + vm.shutdown() diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out new file mode 100644 index 0000000000..9a2d7cbb77 --- /dev/null +++ b/tests/qemu-iotests/255.out @@ -0,0 +1,16 @@ +=3D=3D=3D Create backing chain and start VM =3D=3D=3D + +Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=3Dqcow2 size=3D134217728 cluste= r_size=3D65536 lazy_refcounts=3Doff refcount_bits=3D16 + +Formatting 'TEST_DIR/PID-t.qcow2', fmt=3Dqcow2 size=3D134217728 cluster_si= ze=3D65536 lazy_refcounts=3Doff refcount_bits=3D16 + +=3D=3D=3D Start background read requests =3D=3D=3D + +=3D=3D=3D Run a commit job =3D=3D=3D + +{"execute": "block-commit", "arguments": {"auto-finalize": false, "device"= : "overlay", "job-id": "job0", "top-node": "mid"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "job0"}} +{"return": {}} +{"data": {"id": "job0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", "= timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "job0", "len": 134217728, "offset": 134217728, "speed"= : 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micr= oseconds": "USECS", "seconds": "SECS"}} diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 4cd2fe80a6..19e2bb3aec 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -249,3 +249,4 @@ 247 rw auto quick 248 rw auto quick 253 rw auto quick +255 rw auto quick diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 997dc910cb..8b209ea0ee 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -126,6 +126,11 @@ def qemu_img_pipe(*args): sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode,= ' '.join(qemu_img_args + list(args)))) return subp.communicate()[0] =20 +def qemu_img_log(*args): + result =3D qemu_img_pipe(*args) + log(result, filters=3D[filter_testfiles]) + return result + def img_info_log(filename, filter_path=3DNone, imgopts=3DFalse, extra_args= =3D[]): args =3D [ 'info' ] if imgopts: @@ -533,7 +538,8 @@ class VM(qtest.QEMUQtestMachine): return result =20 # Returns None on success, and an error string on failure - def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse): + def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse, + pre_finalize=3DNone): error =3D None while True: for ev in self.get_qmp_events_filtered(wait=3DTrue): @@ -546,6 +552,8 @@ class VM(qtest.QEMUQtestMachine): error =3D j['error'] log('Job failed: %s' % (j['error'])) elif status =3D=3D 'pending' and not auto_finalize: + if pre_finalize: + pre_finalize() self.qmp_log('job-finalize', id=3Djob) elif status =3D=3D 'concluded' and not auto_dismiss: self.qmp_log('job-dismiss', id=3Djob) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569981009; cv=none; d=zoho.com; s=zohoarc; b=CLiWCOvzE4tZDx4rxzl22dNydrRiT+rVeO6JA/9mzq5rsBVRsBoSxEjRy1QhUE1vCQZxuxh6tqfEgyK1Xgp3M2r+SA87kHzLzExlYeBVmIXJJYf6PQf/pyfP6XmSfE4uCw3dd4ihwDy/6KQHbc5q+f9ZdwIjef3wJsGn4ec+OnQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569981009; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=taXdLAatWE3t4eDy9BuH+Y07ljD7zsfBVNuVbuaMKPQ=; b=N2o1A+zJ9HXu9WEECW5PNkwIFtFEoMzerKa3GZzbQo4vgsF+ufQKBroI6C7RdOoytlZENQnm0pRP3JKvwqlBm7QhvzUhvZjSBxDas/q+OMJJX/OClxnDz3q/7NwhLCWZm8F8Xl05B0wFe80yQf7VZFnzeJ6IbgDxTfG5IwEIFz4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569981009554176.55078455256864; Tue, 1 Oct 2019 18:50:09 -0700 (PDT) Received: from localhost ([::1]:50388 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTmK-0003Ga-6u for importer@patchew.org; Tue, 01 Oct 2019 21:50:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44142) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRwT-0005mE-7J for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:52:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRwR-00074Z-JF for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:52:28 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:47096 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRwJ-0006wB-VJ; Tue, 01 Oct 2019 19:52:20 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NpmTl140377; Tue, 1 Oct 2019 19:52:17 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4dgx3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:52:17 -0400 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NmOrc133005; Tue, 1 Oct 2019 19:48:24 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4de13-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:23 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlOr031629; Tue, 1 Oct 2019 23:47:22 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fab-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:22 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlL7E44302732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:21 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B476B2805E; Tue, 1 Oct 2019 23:47:21 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9790728058; Tue, 1 Oct 2019 23:47:21 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:21 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 19/97] iotests.py: do not use infinite waits Date: Tue, 1 Oct 2019 18:44:58 -0500 Message-Id: <20191001234616.7825-20-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010204 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Cap waits to 60 seconds so that iotests can fail gracefully if something goes wrong. Signed-off-by: John Snow Message-id: 20190523170643.20794-3-jsnow@redhat.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz (cherry picked from commit 8b6f5f8b9f3bec5cbeebefab34bae0102a2581b3) Signed-off-by: Michael Roth --- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 8b209ea0ee..8061f01ade 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -521,7 +521,7 @@ class VM(qtest.QEMUQtestMachine): output_list +=3D [key + '=3D' + obj[key]] return ','.join(output_list) =20 - def get_qmp_events_filtered(self, wait=3DTrue): + def get_qmp_events_filtered(self, wait=3D60.0): result =3D [] for ev in self.get_qmp_events(wait=3Dwait): result.append(filter_qmp_event(ev)) @@ -539,10 +539,10 @@ class VM(qtest.QEMUQtestMachine): =20 # Returns None on success, and an error string on failure def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse, - pre_finalize=3DNone): + pre_finalize=3DNone, wait=3D60.0): error =3D None while True: - for ev in self.get_qmp_events_filtered(wait=3DTrue): + for ev in self.get_qmp_events_filtered(wait=3Dwait): if ev['event'] =3D=3D 'JOB_STATUS_CHANGE': status =3D ev['data']['status'] if status =3D=3D 'aborting': @@ -633,7 +633,7 @@ class QMPTestCase(unittest.TestCase): self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filena= me[5:])), self.vm.flatten_qmp_object(reference)) =20 - def cancel_and_wait(self, drive=3D'drive0', force=3DFalse, resume=3DFa= lse): + def cancel_and_wait(self, drive=3D'drive0', force=3DFalse, resume=3DFa= lse, wait=3D60.0): '''Cancel a block job and wait for it to finish, returning the eve= nt''' result =3D self.vm.qmp('block-job-cancel', device=3Ddrive, force= =3Dforce) self.assert_qmp(result, 'return', {}) @@ -644,7 +644,7 @@ class QMPTestCase(unittest.TestCase): cancelled =3D False result =3D None while not cancelled: - for event in self.vm.get_qmp_events(wait=3DTrue): + for event in self.vm.get_qmp_events(wait=3Dwait): if event['event'] =3D=3D 'BLOCK_JOB_COMPLETED' or \ event['event'] =3D=3D 'BLOCK_JOB_CANCELLED': self.assert_qmp(event, 'data/device', drive) @@ -657,10 +657,10 @@ class QMPTestCase(unittest.TestCase): self.assert_no_active_block_jobs() return result =20 - def wait_until_completed(self, drive=3D'drive0', check_offset=3DTrue): + def wait_until_completed(self, drive=3D'drive0', check_offset=3DTrue, = wait=3D60.0): '''Wait for a block job to finish, returning the event''' while True: - for event in self.vm.get_qmp_events(wait=3DTrue): + for event in self.vm.get_qmp_events(wait=3Dwait): if event['event'] =3D=3D 'BLOCK_JOB_COMPLETED': self.assert_qmp(event, 'data/device', drive) self.assert_qmp_absent(event, 'data/error') --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974830; cv=none; d=zoho.com; s=zohoarc; b=F0B/fmfNvFRCsWa5RKw48N9r8pQ+9auX+Nr9gXLg5Bmg2AoiEmAJYaetG0HRFO2xgIgzM1E84jxr/P3+0XP09lRx5TwBfkmE1SIdbCrii+WWJvnDvFfpAGVmAxntAy4dC2a4cBnDMjZ/r3lCH/Rdag6+FoTutRLrbXFv58Mh+sY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974830; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=QythZ57+IJrvEYF9EDhI6XQPVDZlNTHRxtNEv35G5gU=; b=fLePj6fhnm1eMNY7Ovg8Uvpaq7rF4dpCPzOh5IgamFZP/f+kR9ETYcYCnWx09mWHfKEM2C34vb4u5bvKGbiC6dIo2rwhjmHqZcH23x8SDFu092wdjouFsAVtjz8kGJBDHW7oyeyqQRIr6K66ETUeuy+s6Cly7uDNtukxawEjpTI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569974830419536.9479393625267; Tue, 1 Oct 2019 17:07:10 -0700 (PDT) Received: from localhost ([::1]:49556 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSAf-0007bF-7V for importer@patchew.org; Tue, 01 Oct 2019 20:07:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39024) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrl-00018L-LT for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrc-0001LG-1T for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:33 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:30718) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrb-0001IG-Aa; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPCU074810; Tue, 1 Oct 2019 19:47:26 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlPWM074813; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpqp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlOs031629; Tue, 1 Oct 2019 23:47:22 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fac-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:22 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlMtm54133138 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:22 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 36F8B28058; Tue, 1 Oct 2019 23:47:22 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1A8EC28059; Tue, 1 Oct 2019 23:47:22 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:22 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 20/97] QEMUMachine: add events_wait method Date: Tue, 1 Oct 2019 18:44:59 -0500 Message-Id: <20191001234616.7825-21-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Instead of event_wait which looks for a single event, add an events_wait which can look for any number of events simultaneously. However, it will still only return one at a time, whichever happens first. Signed-off-by: John Snow Message-id: 20190523170643.20794-4-jsnow@redhat.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz (cherry picked from commit f6f4b3f045ea18e3fa93a50cd0462236c428d62e) Signed-off-by: Michael Roth --- python/qemu/__init__.py | 69 +++++++++++++++++++++++++++++------------ 1 file changed, 49 insertions(+), 20 deletions(-) diff --git a/python/qemu/__init__.py b/python/qemu/__init__.py index fd144c0006..4cb8eb2f01 100644 --- a/python/qemu/__init__.py +++ b/python/qemu/__init__.py @@ -408,42 +408,71 @@ class QEMUMachine(object): self._qmp.clear_events() return events =20 - def event_wait(self, name, timeout=3D60.0, match=3DNone): + @staticmethod + def event_match(event, match=3DNone): """ - Wait for specified timeout on named event in QMP; optionally filter - results by match. + Check if an event matches optional match criteria. =20 - The 'match' is checked to be a recursive subset of the 'event'; sk= ips - branch processing on match's value None - {"foo": {"bar": 1}} matches {"foo": None} - {"foo": {"bar": 1}} does not matches {"foo": {"baz": None}} + The match criteria takes the form of a matching subdict. The event= is + checked to be a superset of the subdict, recursively, with matching + values whenever those values are not None. + + Examples, with the subdict queries on the left: + - None matches any object. + - {"foo": None} matches {"foo": {"bar": 1}} + - {"foo": {"baz": None}} does not match {"foo": {"bar": 1}} + - {"foo": {"baz": 2}} matches {"foo": {"bar": 1, "baz": 2}} """ - def event_match(event, match=3DNone): - if match is None: - return True + if match is None: + return True =20 - for key in match: - if key in event: - if isinstance(event[key], dict): - if not event_match(event[key], match[key]): - return False - elif event[key] !=3D match[key]: + for key in match: + if key in event: + if isinstance(event[key], dict): + if not QEMUMachine.event_match(event[key], match[key]): return False - else: + elif event[key] !=3D match[key]: return False + else: + return False + return True =20 - return True + def event_wait(self, name, timeout=3D60.0, match=3DNone): + """ + event_wait waits for and returns a named event from QMP with a tim= eout. + + name: The event to wait for. + timeout: QEMUMonitorProtocol.pull_event timeout parameter. + match: Optional match criteria. See event_match for details. + """ + return self.events_wait([(name, match)], timeout) + + def events_wait(self, events, timeout=3D60.0): + """ + events_wait waits for and returns a named event from QMP with a ti= meout. + + events: a sequence of (name, match_criteria) tuples. + The match criteria are optional and may be None. + See event_match for details. + timeout: QEMUMonitorProtocol.pull_event timeout parameter. + """ + def _match(event): + for name, match in events: + if (event['event'] =3D=3D name and + self.event_match(event, match)): + return True + return False =20 # Search cached events for event in self._events: - if (event['event'] =3D=3D name) and event_match(event, match): + if _match(event): self._events.remove(event) return event =20 # Poll for new events while True: event =3D self._qmp.pull_event(wait=3Dtimeout) - if (event['event'] =3D=3D name) and event_match(event, match): + if _match(event): return event self._events.append(event) =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974135; cv=none; d=zoho.com; s=zohoarc; b=ZCYDBMn6FnIpLBPH4iRQMzbcx768Dn1hlwLy0epZVsHqoVaP6GBIKHoZdUXVHXcAh4xgz35cDamykg3EQzY1QMpdIusohhv9xW2lkTR2VLC9BjL1vUrbw7CFRhxaKHQdG+i5jTNd/g1pj5HMWuDeyWkW9UxZVP91JBqPl2kieIQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974135; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Dk+ZOxRa74AL4w2Ia+I4xdcAorhk89TBP5CUDfAUUO0=; b=DU7gO4i5Zk/Qj24GN0BPNrvUBEatb3kvkGD9XqO/j0mnbZVf/xC3DqQDd8nZitLkTLCXB33d4kgpqpIPGsmh1JUo4W1DbgwLyuPF4l8uD0I8WYlJyYKaWIbn/92mk2ot2iC9WornUi8pdgb+1LOwya+0DkcYUo1KVE/bK+d6AYQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569974135034101.76370907401031; Tue, 1 Oct 2019 16:55:35 -0700 (PDT) Received: from localhost ([::1]:49436 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRzR-0007sk-UJ for importer@patchew.org; Tue, 01 Oct 2019 19:55:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38904) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017N-Nx for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001Ka-Pu for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:25066) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrb-0001Hf-60; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOJ2053133; Tue, 1 Oct 2019 19:47:25 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg171618-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlPFP053472; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17160j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk67029377; Tue, 1 Oct 2019 23:47:24 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fage-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:23 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlN7E43188586 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:23 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 31CD928058; Tue, 1 Oct 2019 23:47:23 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 158E82805A; Tue, 1 Oct 2019 23:47:23 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:23 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 21/97] iotests.py: Fix VM.run_job Date: Tue, 1 Oct 2019 18:45:00 -0500 Message-Id: <20191001234616.7825-22-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz log() is in the current module, there is no need to prefix it. In fact, doing so may make VM.run_job() unusable in tests that never use iotests.log() themselves. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf (cherry picked from commit 86a4f599a67b9b709109c7a7c8b7eb91d21c21fd) *prereq for d6a79af0e6 Signed-off-by: Michael Roth --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 8061f01ade..fa00b61892 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -560,7 +560,7 @@ class VM(qtest.QEMUQtestMachine): elif status =3D=3D 'null': return error else: - iotests.log(ev) + log(ev) =20 def node_info(self, node_name): nodes =3D self.qmp('query-named-block-nodes') --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569973874; cv=none; d=zoho.com; s=zohoarc; b=j/jodlUkws+hlDI3uxVCqQ2yjr/DA+g1nzt0jxH0NK8OqaWKu7PTQzhBlBYtefYw9Ij5M/d8SXjuREyZ7BfxNDaXgx3ZvUDfTgYclPsvVNSMwMQ9qhaHNo22F1PELhJP4j5q0uOd+wNeQQ0X/uOzhyaumH/3/nmoimeOBs19ygg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569973874; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=/fxPS40thQGf9THPqYnHyQynuncRV/Yk2X0J/VbhOAw=; b=IOQRNXNPXmu0W6o2f4aWu3bvwjjbBrSkUimwsmJurAe+UVcuWHcBZYEwwpIq+hmv5bVxjXyfqmAkfGUTPgmZUnnGBjRV+o0w2mgTvEUCUPjL7boOCwHZy3VYbnjNcB9pXVMrloNuoyIjNCVCmw1dDr+Jpp1qc4pSycuMBbWYbgQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569973874435653.9382174356026; Tue, 1 Oct 2019 16:51:14 -0700 (PDT) Received: from localhost ([::1]:49406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRv9-0003wu-VJ for importer@patchew.org; Tue, 01 Oct 2019 19:51:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38909) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017Q-Oy for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001KJ-Lc for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:21620) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrb-0001I5-2Q; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQ5X053544; Tue, 1 Oct 2019 19:47:26 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17161u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlPsr053508; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17160s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlKX027161; Tue, 1 Oct 2019 23:47:24 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bf8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:24 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlNuP43188590 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:23 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B1CD828059; Tue, 1 Oct 2019 23:47:23 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 952F728058; Tue, 1 Oct 2019 23:47:23 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:23 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 22/97] iotests.py: rewrite run_job to be pickier Date: Tue, 1 Oct 2019 18:45:01 -0500 Message-Id: <20191001234616.7825-23-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Don't pull events out of the queue that don't belong to us; be choosier so that we can use this method to drive jobs that were launched by transactions that may have more jobs. Signed-off-by: John Snow Message-id: 20190523170643.20794-5-jsnow@redhat.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz (cherry picked from commit d6a79af0e641806d6bd6a42a4920e294b5db179c) Signed-off-by: Michael Roth --- tests/qemu-iotests/iotests.py | 48 +++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index fa00b61892..a7006662ff 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -540,27 +540,37 @@ class VM(qtest.QEMUQtestMachine): # Returns None on success, and an error string on failure def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse, pre_finalize=3DNone, wait=3D60.0): + match_device =3D {'data': {'device': job}} + match_id =3D {'data': {'id': job}} + events =3D [ + ('BLOCK_JOB_COMPLETED', match_device), + ('BLOCK_JOB_CANCELLED', match_device), + ('BLOCK_JOB_ERROR', match_device), + ('BLOCK_JOB_READY', match_device), + ('BLOCK_JOB_PENDING', match_id), + ('JOB_STATUS_CHANGE', match_id) + ] error =3D None while True: - for ev in self.get_qmp_events_filtered(wait=3Dwait): - if ev['event'] =3D=3D 'JOB_STATUS_CHANGE': - status =3D ev['data']['status'] - if status =3D=3D 'aborting': - result =3D self.qmp('query-jobs') - for j in result['return']: - if j['id'] =3D=3D job: - error =3D j['error'] - log('Job failed: %s' % (j['error'])) - elif status =3D=3D 'pending' and not auto_finalize: - if pre_finalize: - pre_finalize() - self.qmp_log('job-finalize', id=3Djob) - elif status =3D=3D 'concluded' and not auto_dismiss: - self.qmp_log('job-dismiss', id=3Djob) - elif status =3D=3D 'null': - return error - else: - log(ev) + ev =3D filter_qmp_event(self.events_wait(events)) + if ev['event'] !=3D 'JOB_STATUS_CHANGE': + log(ev) + continue + status =3D ev['data']['status'] + if status =3D=3D 'aborting': + result =3D self.qmp('query-jobs') + for j in result['return']: + if j['id'] =3D=3D job: + error =3D j['error'] + log('Job failed: %s' % (j['error'])) + elif status =3D=3D 'pending' and not auto_finalize: + if pre_finalize: + pre_finalize() + self.qmp_log('job-finalize', id=3Djob) + elif status =3D=3D 'concluded' and not auto_dismiss: + self.qmp_log('job-dismiss', id=3Djob) + elif status =3D=3D 'null': + return error =20 def node_info(self, node_name): nodes =3D self.qmp('query-named-block-nodes') --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569981246; cv=none; d=zoho.com; s=zohoarc; b=Vl8rCi82H3KdG1+9NJjm0eTC9XCCgl5sPSrGHm/0/uXlufbi6g2R8lYMpVe7vNkOunKW0tBR4UdN4lDSKjXtHZr8g3fr/aiuA2M7Qvt3GaD8qwqNQpT7Tg5B2I8NNx9QhGJuZ7G5CrCaZZpjNZBiAshzWVG5hejHPAgH1SdoK/0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569981246; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=n/XdKT9tnMZR6EC5FBZgxLii+gyWGNign6ZasfIJG20=; b=RDVQ8InTa0UouKpVq0GZIdo4C1FMsmx0nAPJWAzgAtn7WIekykpus1E0Si7DXygJZwCmxzOANlfekkney1LRESqD4mHubm8a89XhQWlogteU6hvVDjQN1toABvsyR/6TxLRyfOlgUhRD3v9C/JHPudy1JwgjdMZEThpPAPQqFFY= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569981246268807.185333300392; Tue, 1 Oct 2019 18:54:06 -0700 (PDT) Received: from localhost ([::1]:50416 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTq6-0006z7-N7 for importer@patchew.org; Tue, 01 Oct 2019 21:54:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42027) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRtL-0002lz-Br for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRtJ-0003h8-3r for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:49:14 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:57580) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRtI-0003EN-Ou; Tue, 01 Oct 2019 19:49:12 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NmnCY005158; Tue, 1 Oct 2019 19:48:51 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdfxwett-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:49 -0400 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NmmJ1005008; Tue, 1 Oct 2019 19:48:48 -0400 Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdfxwec3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:48 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlEk022439; Tue, 1 Oct 2019 23:47:25 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma04dal.us.ibm.com with ESMTP id 2v9y57yd5d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:25 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlOdR47513940 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:24 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3FC8B2805A; Tue, 1 Oct 2019 23:47:24 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 18D1E28058; Tue, 1 Oct 2019 23:47:24 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:24 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 23/97] iotests: add iotest 256 for testing blockdev-backup across iothread contexts Date: Tue, 1 Oct 2019 18:45:02 -0500 Message-Id: <20191001234616.7825-24-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Signed-off-by: John Snow Message-id: 20190523170643.20794-6-jsnow@redhat.com Reviewed-by: Max Reitz [mreitz: Moved from 250 to 256] Signed-off-by: Max Reitz (cherry picked from commit ba7704f2228f16ed61b9903801e28e17666c7e38) Signed-off-by: Michael Roth --- tests/qemu-iotests/256 | 122 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/256.out | 119 ++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 242 insertions(+) create mode 100755 tests/qemu-iotests/256 create mode 100644 tests/qemu-iotests/256.out diff --git a/tests/qemu-iotests/256 b/tests/qemu-iotests/256 new file mode 100755 index 0000000000..c594a43205 --- /dev/null +++ b/tests/qemu-iotests/256 @@ -0,0 +1,122 @@ +#!/usr/bin/env python +# +# Test incremental/backup across iothread contexts +# +# Copyright (c) 2019 John Snow for Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# owner=3Djsnow@redhat.com + +import os +import iotests +from iotests import log + +iotests.verify_image_format(supported_fmts=3D['qcow2']) +size =3D 64 * 1024 * 1024 + +with iotests.FilePath('img0') as img0_path, \ + iotests.FilePath('img1') as img1_path, \ + iotests.FilePath('img0-full') as img0_full_path, \ + iotests.FilePath('img1-full') as img1_full_path, \ + iotests.FilePath('img0-incr') as img0_incr_path, \ + iotests.FilePath('img1-incr') as img1_incr_path, \ + iotests.VM() as vm: + + def create_target(filepath, name, size): + basename =3D os.path.basename(filepath) + nodename =3D "file_{}".format(basename) + log(vm.command('blockdev-create', job_id=3D'job1', + options=3D{ + 'driver': 'file', + 'filename': filepath, + 'size': 0, + })) + vm.run_job('job1') + log(vm.command('blockdev-add', driver=3D'file', + node_name=3Dnodename, filename=3Dfilepath)) + log(vm.command('blockdev-create', job_id=3D'job2', + options=3D{ + 'driver': iotests.imgfmt, + 'file': nodename, + 'size': size, + })) + vm.run_job('job2') + log(vm.command('blockdev-add', driver=3Diotests.imgfmt, + node_name=3Dname, + file=3Dnodename)) + + log('--- Preparing images & VM ---\n') + vm.add_object('iothread,id=3Diothread0') + vm.add_object('iothread,id=3Diothread1') + vm.add_device('virtio-scsi-pci,id=3Dscsi0,iothread=3Diothread0') + vm.add_device('virtio-scsi-pci,id=3Dscsi1,iothread=3Diothread1') + iotests.qemu_img_create('-f', iotests.imgfmt, img0_path, str(size)) + iotests.qemu_img_create('-f', iotests.imgfmt, img1_path, str(size)) + vm.add_drive(img0_path, interface=3D'none') + vm.add_device('scsi-hd,id=3Ddevice0,drive=3Ddrive0,bus=3Dscsi0.0') + vm.add_drive(img1_path, interface=3D'none') + vm.add_device('scsi-hd,id=3Ddevice1,drive=3Ddrive1,bus=3Dscsi1.0') + + log('--- Starting VM ---\n') + vm.launch() + + log('--- Create Targets & Full Backups ---\n') + create_target(img0_full_path, 'img0-full', size) + create_target(img1_full_path, 'img1-full', size) + ret =3D vm.qmp_log('transaction', indent=3D2, actions=3D[ + { 'type': 'block-dirty-bitmap-add', + 'data': { 'node': 'drive0', 'name': 'bitmap0' }}, + { 'type': 'block-dirty-bitmap-add', + 'data': { 'node': 'drive1', 'name': 'bitmap1' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive0', + 'target': 'img0-full', + 'sync': 'full', + 'job-id': 'j0' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive1', + 'target': 'img1-full', + 'sync': 'full', + 'job-id': 'j1' }} + ]) + if "error" in ret: + raise Exception(ret['error']['desc']) + vm.run_job('j0', auto_dismiss=3DTrue) + vm.run_job('j1', auto_dismiss=3DTrue) + + log('\n--- Create Targets & Incremental Backups ---\n') + create_target(img0_incr_path, 'img0-incr', size) + create_target(img1_incr_path, 'img1-incr', size) + ret =3D vm.qmp_log('transaction', indent=3D2, actions=3D[ + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive0', + 'target': 'img0-incr', + 'sync': 'incremental', + 'bitmap': 'bitmap0', + 'job-id': 'j2' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive1', + 'target': 'img1-incr', + 'sync': 'incremental', + 'bitmap': 'bitmap1', + 'job-id': 'j3' }} + ]) + if "error" in ret: + raise Exception(ret['error']['desc']) + vm.run_job('j2', auto_dismiss=3DTrue) + vm.run_job('j3', auto_dismiss=3DTrue) + + log('\n--- Done ---') + vm.shutdown() diff --git a/tests/qemu-iotests/256.out b/tests/qemu-iotests/256.out new file mode 100644 index 0000000000..eec38614ec --- /dev/null +++ b/tests/qemu-iotests/256.out @@ -0,0 +1,119 @@ +--- Preparing images & VM --- + +--- Starting VM --- + +--- Create Targets & Full Backups --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "name": "bitmap0", + "node": "drive0" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "name": "bitmap1", + "node": "drive1" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "device": "drive0", + "job-id": "j0", + "sync": "full", + "target": "img0-full" + }, + "type": "blockdev-backup" + }, + { + "data": { + "device": "drive1", + "job-id": "j1", + "sync": "full", + "target": "img1-full" + }, + "type": "blockdev-backup" + } + ] + } +} +{ + "return": {} +} +{"data": {"device": "j0", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} +{"data": {"device": "j1", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} + +--- Create Targets & Incremental Backups --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "bitmap": "bitmap0", + "device": "drive0", + "job-id": "j2", + "sync": "incremental", + "target": "img0-incr" + }, + "type": "blockdev-backup" + }, + { + "data": { + "bitmap": "bitmap1", + "device": "drive1", + "job-id": "j3", + "sync": "incremental", + "target": "img1-incr" + }, + "type": "blockdev-backup" + } + ] + } +} +{ + "return": {} +} +{"data": {"device": "j2", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} +{"data": {"device": "j3", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} + +--- Done --- diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 19e2bb3aec..7aa6dc9875 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -250,3 +250,4 @@ 248 rw auto quick 253 rw auto quick 255 rw auto quick +256 rw auto quick --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974133; cv=none; d=zoho.com; s=zohoarc; b=FJu5RjySNMeJ2uO8T7cjLn5VB2fDfT8raNiLwccg7wgEluyOhrXYa3lY2M6TB8vTQsu5fNS1znd2PJ6O5FrHoSGKGquDDDyOwMDs+6NOCyfbP7AFNGC1JmRjyJkIoIOtzb8qcFYKFeIzB21DCQlNVic2MxxbWPBex0Rqq89Wlpc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974133; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=UxxvDKh2QEMXCP4HrxuAoLYW4eklpmXKsWRi1LX6YUE=; b=P5JL4HXnDPOQstusFtQie6MgXH6rGU3b4b2mn4yrukhjsdAfb8LgpQaTSDvI8jqgOTHLbpkyUsyHxPK9Gn+SbPpUseAVMOTBskkFwM6bqhqoGTKYXrHkSzpgE+2pSckKfvMfTsAcyVUw0Vzoaq8GAXB4uxOs5ZGNyqhQUGq0X4o= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569974133341187.52481946121793; Tue, 1 Oct 2019 16:55:33 -0700 (PDT) Received: from localhost ([::1]:49434 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRzP-0007qc-GO for importer@patchew.org; Tue, 01 Oct 2019 19:55:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38908) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrd-00017P-Ou for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrb-0001KP-M0 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:29 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:36852 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrb-0001IB-60; Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOIB098942; Tue, 1 Oct 2019 19:47:26 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9utp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlPfR099017; Tue, 1 Oct 2019 19:47:25 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9utd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:25 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njl16006380; Tue, 1 Oct 2019 23:47:25 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye78-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:25 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlONh47513944 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:24 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BB6922805A; Tue, 1 Oct 2019 23:47:24 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9F4DC28058; Tue, 1 Oct 2019 23:47:24 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:24 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 24/97] migration/dirty-bitmaps: change bitmap enumeration method Date: Tue, 1 Oct 2019 18:45:03 -0500 Message-Id: <20191001234616.7825-25-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0b-001b2d01.pphosted.com id x91NlOIB098942 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org, Stefan Hajnoczi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: John Snow Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices. Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name. This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug. Reported-by: Peter Maydell Reported-by: Coverity =F0=9F=98=85 Reported-by: aihua liang Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: John Snow Message-id: 20190514201926.10407-1-jsnow@redhat.com Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=3D1652490 Fixes: Coverity CID 1390625 CC: Stefan Hajnoczi Signed-off-by: John Snow (cherry picked from commit 592203e7cfbd1ad261178431fcf390adfe8b16df) Signed-off-by: Michael Roth --- migration/block-dirty-bitmap.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index d1bb863cb6..4a896a09eb 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -273,7 +273,6 @@ static int init_dirty_bitmap_migration(void) BlockDriverState *bs; BdrvDirtyBitmap *bitmap; DirtyBitmapMigBitmapState *dbms; - BdrvNextIterator it; Error *local_err =3D NULL; =20 dirty_bitmap_mig_state.bulk_completed =3D false; @@ -281,13 +280,8 @@ static int init_dirty_bitmap_migration(void) dirty_bitmap_mig_state.prev_bitmap =3D NULL; dirty_bitmap_mig_state.no_bitmaps =3D false; =20 - for (bs =3D bdrv_first(&it); bs; bs =3D bdrv_next(&it)) { - const char *drive_name =3D bdrv_get_device_or_node_name(bs); - - /* skip automatically inserted nodes */ - while (bs && bs->drv && bs->implicit) { - bs =3D backing_bs(bs); - } + for (bs =3D bdrv_next_all_states(NULL); bs; bs =3D bdrv_next_all_state= s(bs)) { + const char *name =3D bdrv_get_device_or_node_name(bs); =20 for (bitmap =3D bdrv_dirty_bitmap_next(bs, NULL); bitmap; bitmap =3D bdrv_dirty_bitmap_next(bs, bitmap)) @@ -296,7 +290,7 @@ static int init_dirty_bitmap_migration(void) continue; } =20 - if (drive_name =3D=3D NULL) { + if (!name || strcmp(name, "") =3D=3D 0) { error_report("Found bitmap '%s' in unnamed node %p. It can= 't " "be migrated", bdrv_dirty_bitmap_name(bitmap)= , bs); goto fail; @@ -313,7 +307,7 @@ static int init_dirty_bitmap_migration(void) =20 dbms =3D g_new0(DirtyBitmapMigBitmapState, 1); dbms->bs =3D bs; - dbms->node_name =3D drive_name; + dbms->node_name =3D name; dbms->bitmap =3D bitmap; dbms->total_sectors =3D bdrv_nb_sectors(bs); dbms->sectors_per_chunk =3D CHUNK_SIZE * 8 * --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975190; cv=none; d=zoho.com; s=zohoarc; b=UA6cCNaZiUDwebnWh8IrhpRh/A6QHFzE0bapH28M4x/G9PF1060lvRnOyC7LXXtkB8glOtzLIbn/MWuEKknyljE+w008DPycTOmSPxYiwc1jIujR/FovRBWylbCU6+U7cciqnfNT4goIEMC+XWb6f3DucgV4mvswyhEguLV1pcw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975190; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ChgFBHZuLbuJA9a+RB32ER3uoD/dtd6ykVwYmjD/MV8=; b=O6/6krU4L6KmiblH1Y2lhcMBNpa3aUL3oVsGzVBkThkwghTAhG5gzjwn97v50BVwXvapaZXMvSQiYHDk+EIo5+ctCyk/nxN7J59a9/4UiOdLnAPW7cjdudKSMKAPs93O2JOX7lZH3/wieVPADKT1N7PJIWHDb7fxk/pOiCwbREQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975190165878.3955688078357; Tue, 1 Oct 2019 17:13:10 -0700 (PDT) Received: from localhost ([::1]:49598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSGN-0003kU-Uj for importer@patchew.org; Tue, 01 Oct 2019 20:13:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39128) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrn-0001Ao-MM for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrl-0001Us-LI for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:39 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:59064) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRre-0001Mx-3S; Tue, 01 Oct 2019 19:47:33 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlODv074763; Tue, 1 Oct 2019 19:47:27 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpsg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:27 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlQa4074887; Tue, 1 Oct 2019 19:47:26 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vprx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:26 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjmQj032160; Tue, 1 Oct 2019 23:47:27 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma01wdc.us.ibm.com with ESMTP id 2v9y57fga6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:27 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlPgt52429262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:25 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 42A2128059; Tue, 1 Oct 2019 23:47:25 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2716428058; Tue, 1 Oct 2019 23:47:25 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:25 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 25/97] vhost: fix vhost_log size overflow during migration Date: Tue, 1 Oct 2019 18:45:04 -0500 Message-Id: <20191001234616.7825-26-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Hangjing , qemu-stable@nongnu.org, "Michael S . Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Li Hangjing When a guest which doesn't support multiqueue is migrated with a multi queu= es vhost-user-blk deivce, a crash will occur like: 0 qemu_memfd_alloc (name=3D, size=3D562949953421312, s= eals=3D, fd=3D0x7f87171fe8b4, errp=3D0x7f87171fe8a8) a= t util/memfd.c:153 1 0x00007f883559d7cf in vhost_log_alloc (size=3D70368744177664, share=3Dtru= e) at hw/virtio/vhost.c:186 2 0x00007f88355a0758 in vhost_log_get (listener=3D0x7f8838bd7940, enable=3D= 1) at qemu-2-12/hw/virtio/vhost.c:211 3 vhost_dev_log_resize (listener=3D0x7f8838bd7940, enable=3D1) at hw/virtio= /vhost.c:263 4 vhost_migration_log (listener=3D0x7f8838bd7940, enable=3D1) at hw/virtio/= vhost.c:787 5 0x00007f88355463d6 in memory_global_dirty_log_start () at memory.c:2503 6 0x00007f8835550577 in ram_init_bitmaps (f=3D0x7f88384ce600, opaque=3D0x7f= 8836024098) at migration/ram.c:2173 7 ram_init_all (f=3D0x7f88384ce600, opaque=3D0x7f8836024098) at migration/r= am.c:2192 8 ram_save_setup (f=3D0x7f88384ce600, opaque=3D0x7f8836024098) at migration= /ram.c:2219 9 0x00007f88357a419d in qemu_savevm_state_setup (f=3D0x7f88384ce600) at mig= ration/savevm.c:1002 10 0x00007f883579fc3e in migration_thread (opaque=3D0x7f8837530400) at migr= ation/migration.c:2382 11 0x00007f8832447893 in start_thread () from /lib64/libpthread.so.0 12 0x00007f8832178bfd in clone () from /lib64/libc.so.6 This is because vhost_get_log_size() returns a overflowed vhost-log size. In this function, it uses the uninitialized variable vqs->used_phys and vqs->used_size to get the vhost-log size. Signed-off-by: Li Hangjing Reviewed-by: Xie Yongji Reviewed-by: Chai Wen Message-Id: <20190603061524.24076-1-lihangjing@baidu.com> Cc: qemu-stable@nongnu.org Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit 240e647a14df9677b3a501f7b8b870e40aac3fd5) Signed-off-by: Michael Roth --- hw/virtio/vhost.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 7f61018f2a..6d3a013f49 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -131,6 +131,11 @@ static int vhost_sync_dirty_bitmap(struct vhost_dev *d= ev, } for (i =3D 0; i < dev->nvqs; ++i) { struct vhost_virtqueue *vq =3D dev->vqs + i; + + if (!vq->used_phys && !vq->used_size) { + continue; + } + vhost_dev_sync_region(dev, section, start_addr, end_addr, vq->used= _phys, range_get_last(vq->used_phys, vq->used_size)= ); } @@ -168,6 +173,11 @@ static uint64_t vhost_get_log_size(struct vhost_dev *d= ev) } for (i =3D 0; i < dev->nvqs; ++i) { struct vhost_virtqueue *vq =3D dev->vqs + i; + + if (!vq->used_phys && !vq->used_size) { + continue; + } + uint64_t last =3D vq->used_phys + vq->used_size - 1; log_size =3D MAX(log_size, last / VHOST_LOG_CHUNK + 1); } --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976591; cv=none; d=zoho.com; s=zohoarc; b=nykPzw0aDGVzMfDecur9mbxI0bgDhb6baACigSzvFyI+2mEwVOdTXbIqlc3tJ33uxuHT1NF9uKSosteGGfAMdIaLHHrGfDYolwwtLVSqMeeCi0rNFRtTtDexZ+qWCECNANIyohmsiRH8WsRESyrDcSAqn8lkXFwjhEJa33VlP5c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976591; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=CGdI8iRm3629ECMhSXpjS0ma/DhC/Qgb/XJmYioALFM=; b=IbCWXSck746qZT+O+VBynzRoC2McYVl6+gMaUe01XK0cMzseP4SgZ3rJRk1no1LqenzWXNL5td6Zg+Etttfbsqi9u6rA/CyqYb2BgK+4Hya4llPB4rHLzPI/fYqgfrYO2lIuzoMAM2pWPm6aFse5HV3gEDXSmY5Qzc2c3vE4yDI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976591528342.0428205438235; Tue, 1 Oct 2019 17:36:31 -0700 (PDT) Received: from localhost ([::1]:49778 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSd3-0008TB-Sh for importer@patchew.org; Tue, 01 Oct 2019 20:36:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39978) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsG-0001fa-5h for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsD-0002AW-BU for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:07 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:9938 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs6-0001PB-NG; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOvT037397; Tue, 1 Oct 2019 19:47:27 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b2yu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk0B006371; Tue, 1 Oct 2019 23:47:26 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye7e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:26 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlPth40304912 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:25 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C011A2805A; Tue, 1 Oct 2019 23:47:25 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A2CDA28058; Tue, 1 Oct 2019 23:47:25 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:25 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 26/97] target/ppc: Fix xvabs[sd]p, xvnabs[sd]p, xvneg[sd]p, xvcpsgn[sd]p Date: Tue, 1 Oct 2019 18:45:05 -0500 Message-Id: <20191001234616.7825-27-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anton Blanchard , qemu-stable@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Anton Blanchard We were using set_cpu_vsr*() when we should have used get_cpu_vsr*(). Fixes: 8b3b2d75c7c0 ("introduce get_cpu_vsr{l,h}() and set_cpu_vsr{l,h}() h= elpers for VSR register access") Signed-off-by: Anton Blanchard Message-Id: <20190509104912.6b754dff@kryten> Reviewed-by: Mark Cave-Ayland Signed-off-by: David Gibson (upstream commit 77bd8937c03dd55e57cc257951ad07c185303c3e) Acked-by: David Gibson Signed-off-by: Michael Roth --- target/ppc/translate/vsx-impl.inc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/target/ppc/translate/vsx-impl.inc.c b/target/ppc/translate/vsx= -impl.inc.c index 489b2436e4..1b483fc27e 100644 --- a/target/ppc/translate/vsx-impl.inc.c +++ b/target/ppc/translate/vsx-impl.inc.c @@ -858,8 +858,8 @@ static void glue(gen_, name)(DisasContext * ctx) = \ xbh =3D tcg_temp_new_i64(); \ xbl =3D tcg_temp_new_i64(); \ sgm =3D tcg_temp_new_i64(); \ - set_cpu_vsrh(xB(ctx->opcode), xbh); \ - set_cpu_vsrl(xB(ctx->opcode), xbl); \ + get_cpu_vsrh(xbh, xB(ctx->opcode)); \ + get_cpu_vsrl(xbl, xB(ctx->opcode)); \ tcg_gen_movi_i64(sgm, sgn_mask); \ switch (op) { \ case OP_ABS: { \ --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976346; cv=none; d=zoho.com; s=zohoarc; b=k6x9JwfJHIjr97OAEDpezX79QTacCaHTbiE15UIW2hj36dJnYjZvN/XI9kPDTac7cSvHyh1irL7enWo8GNY++YFjlWWdnEpY8lWn10kHWuW4e2f1mZogsuXbfnEHeqD7inEqHpyx5juwJFpwwamPn2rO33uUs5uTAnnwWB3ZkiY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976346; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=IgA+VG54KoV9eaZAD7UCUVjrmSGf7jTElvz6IPgwVOE=; b=TBo5Q9yd2s1xkfpuek2DSllkhFpnDvot9ZwXQy4LvhniBBrzdWWiK8euhbbYNkr5KJ20K706HnIa6Yp4krIfDJt8q5nVifet43bEmabg0WWsvjhEwHWgeU09gJaPZIb/CLyPINU4YvacahaZ3U09yYlDHVhREGLp5253TMhVS+Y= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976346742474.37251016512073; Tue, 1 Oct 2019 17:32:26 -0700 (PDT) Received: from localhost ([::1]:49748 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSZ7-0004Uo-5s for importer@patchew.org; Tue, 01 Oct 2019 20:32:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39940) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsF-0001eQ-6l for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsC-00028W-F4 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:06 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:53786) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs5-0001Pb-Vv; Tue, 01 Oct 2019 19:47:58 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPfA107899; Tue, 1 Oct 2019 19:47:28 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurp1g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:27 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkE5027150; Tue, 1 Oct 2019 23:47:27 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bfp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:27 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlQu155050638 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:26 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F1002805A; Tue, 1 Oct 2019 23:47:26 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2421628058; Tue, 1 Oct 2019 23:47:26 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:26 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 27/97] target/ppc: Fix xvxsigdp Date: Tue, 1 Oct 2019 18:45:06 -0500 Message-Id: <20191001234616.7825-28-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anton Blanchard , qemu-stable@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Anton Blanchard Fix a typo in xvxsigdp where we put both results into the lower doubleword. Fixes: dd977e4f45cb ("target/ppc: Optimize x[sv]xsigdp using deposit_i64()") Signed-off-by: Anton Blanchard Message-Id: <20190507004811.29968-1-anton@ozlabs.org> Signed-off-by: David Gibson (upstream commit cf4e9363f7fd889d8d804c8f78e8927782c2aa48) Acked-by: David Gibson Signed-off-by: Michael Roth --- target/ppc/translate/vsx-impl.inc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/ppc/translate/vsx-impl.inc.c b/target/ppc/translate/vsx= -impl.inc.c index 1b483fc27e..96e4ff4411 100644 --- a/target/ppc/translate/vsx-impl.inc.c +++ b/target/ppc/translate/vsx-impl.inc.c @@ -1819,7 +1819,7 @@ static void gen_xvxsigdp(DisasContext *ctx) tcg_gen_movi_i64(t0, 0x0010000000000000); tcg_gen_movcond_i64(TCG_COND_EQ, t0, exp, zr, zr, t0); tcg_gen_movcond_i64(TCG_COND_EQ, t0, exp, nan, zr, t0); - tcg_gen_deposit_i64(xth, t0, xbl, 0, 52); + tcg_gen_deposit_i64(xtl, t0, xbl, 0, 52); set_cpu_vsrl(xT(ctx->opcode), xtl); =20 tcg_temp_free_i64(t0); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975734; cv=none; d=zoho.com; s=zohoarc; b=Y5HtOOOWZQBQ8+CFol+KMlXH3LNUjhuqE0mla3fWi6WWIdNODLHUaIMK3c15LHNV67tBvsSnWrEufO0fu5qbFoSHQFOnZYVtANFSIZpy5jSsOWYg12hDR3IO2iXdPrRCgjg+sntIlgcTKbUWz0RWgV1EVaGUIf2VFhKxBtoFsiM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975734; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=AYuHog4pWubgVFpJAxCnrlpNQz2kT4dYeSJNKNcAyK4=; b=LvQrmYvpkGLJL0NAhurkEX7fsF01eEeitDs+vIyqzjWol0LuGxVwdK1zMQ8JuOG1RB6NF09tuWjlLoz5vOaJS6FwBLNPsfKxHWA+gFvwrm93Lyh4L1m1J5uq9cuQMu4Em3UmeIsuA08xuW4MkXO4hhSvB7v+pLE2jT/vRQlfjuU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975734692121.96838694389805; Tue, 1 Oct 2019 17:22:14 -0700 (PDT) Received: from localhost ([::1]:49684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSPF-0004M3-Fe for importer@patchew.org; Tue, 01 Oct 2019 20:22:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39430) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRru-0001Lp-TD for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrs-0001iX-Nq for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:46 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:55262) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrl-0001PI-PP; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQT2053551; Tue, 1 Oct 2019 19:47:28 -0400 Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17162k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:28 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlEo022439; Tue, 1 Oct 2019 23:47:27 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma04dal.us.ibm.com with ESMTP id 2v9y57yd5v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:27 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlQEF40304926 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:26 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA2CC2805E; Tue, 1 Oct 2019 23:47:26 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9D53628058; Tue, 1 Oct 2019 23:47:26 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:26 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 28/97] target/ppc: Fix xxbrq, xxbrw Date: Tue, 1 Oct 2019 18:45:07 -0500 Message-Id: <20191001234616.7825-29-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anton Blanchard , qemu-stable@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Anton Blanchard Fix a typo in xxbrq and xxbrw where we put both results into the lower doubleword. Fixes: 8b3b2d75c7c0 ("introduce get_cpu_vsr{l,h}() and set_cpu_vsr{l,h}() h= elpers for VSR register access") Signed-off-by: Anton Blanchard Message-Id: <20190507004811.29968-3-anton@ozlabs.org> Signed-off-by: David Gibson (upstream commit d47a751adab7833e9831408376077bc8dba41d5d) Acked-by: David Gibson Signed-off-by: Michael Roth --- target/ppc/translate/vsx-impl.inc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/target/ppc/translate/vsx-impl.inc.c b/target/ppc/translate/vsx= -impl.inc.c index 96e4ff4411..7778d5d651 100644 --- a/target/ppc/translate/vsx-impl.inc.c +++ b/target/ppc/translate/vsx-impl.inc.c @@ -1192,7 +1192,7 @@ static void gen_xxbrq(DisasContext *ctx) tcg_gen_bswap64_i64(xtl, xbh); set_cpu_vsrl(xT(ctx->opcode), xtl); tcg_gen_mov_i64(xth, t0); - set_cpu_vsrl(xT(ctx->opcode), xth); + set_cpu_vsrh(xT(ctx->opcode), xth); =20 tcg_temp_free_i64(t0); tcg_temp_free_i64(xth); @@ -1220,7 +1220,7 @@ static void gen_xxbrw(DisasContext *ctx) get_cpu_vsrl(xbl, xB(ctx->opcode)); =20 gen_bswap32x4(xth, xtl, xbh, xbl); - set_cpu_vsrl(xT(ctx->opcode), xth); + set_cpu_vsrh(xT(ctx->opcode), xth); set_cpu_vsrl(xT(ctx->opcode), xtl); =20 tcg_temp_free_i64(xth); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975424; cv=none; d=zoho.com; s=zohoarc; b=CCvuShUrRh4mBHKbPmPagHyW8hiVu7wISc/NIxr6n5SfWnY5MiklfhGez9DfbIuNw8Z5Fhl+nycoP7WC2Fg5oZMc75iqpGbqM6H4OrTVDGlL5gK/RGiy10ZfzuhyczTHpq1kH3tA3eDkcdoRn4iL8oE3vorz/BK1s8xl2iWNRlE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975424; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=8jpvvQMnYnzFD/SYIO76B2PN0vvPIU6wjX2Q2eL0qP4=; b=cBqTkQqo/LXMj3CkyXOaTWj2pRxFFGE1KXHC52+7XCioYBL8EdULMkLk0XsSyNxiv/7UHGwlx6EPbF9OsWDJajpNSoVw3VWKYeU8neOBe6A+An/ZxyFIpUYdAoXlbMvBBccGkeHziQp8/pWdqgLIXsgNH2Q171Kkf4K22HPgwiU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975424748659.0476344849301; Tue, 1 Oct 2019 17:17:04 -0700 (PDT) Received: from localhost ([::1]:49630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSKC-0007IF-Um for importer@patchew.org; Tue, 01 Oct 2019 20:17:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39549) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrx-0001QI-Op for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrv-0001o7-MH for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:49 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:8896) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrm-0001Pp-2d; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPKh053495; Tue, 1 Oct 2019 19:47:28 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17162m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:28 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkY7031624; Tue, 1 Oct 2019 23:47:27 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587faq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:27 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlRpB4129424 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:27 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3B37628059; Tue, 1 Oct 2019 23:47:27 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1F61328058; Tue, 1 Oct 2019 23:47:27 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:27 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 29/97] target/ppc: Fix vsum2sws Date: Tue, 1 Oct 2019 18:45:08 -0500 Message-Id: <20191001234616.7825-30-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=645 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anton Blanchard , qemu-stable@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Anton Blanchard A recent cleanup changed the pre zeroing of the result from 64 bit to 32 bit operations: - result.u64[i] =3D 0; + result.VsrW(i) =3D 0; This corrupts the result. Fixes: 60594fea298d ("target/ppc: remove various HOST_WORDS_BIGENDIAN hacks= in int_helper.c") Signed-off-by: Anton Blanchard Message-Id: <20190507004811.29968-9-anton@ozlabs.org> Signed-off-by: David Gibson (upstream commit 7fa0ddc1d63806769d1b6246a62708d3bde39037) Acked-by: David Gibson Signed-off-by: Michael Roth --- target/ppc/int_helper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 162add561e..6bd1d32b1d 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -2030,7 +2030,7 @@ void helper_vsum2sws(CPUPPCState *env, ppc_avr_t *r, = ppc_avr_t *a, ppc_avr_t *b) for (i =3D 0; i < ARRAY_SIZE(r->u64); i++) { int64_t t =3D (int64_t)b->VsrSW(upper + i * 2); =20 - result.VsrW(i) =3D 0; + result.VsrD(i) =3D 0; for (j =3D 0; j < ARRAY_SIZE(r->u64); j++) { t +=3D a->VsrSW(2 * i + j); } --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974969; cv=none; d=zoho.com; s=zohoarc; b=Dhr/Ta8qg6Fp0uOg7BuwwDJkui+c6EMiNFLI4hPGSx7BL9PzT8uDP+n/+LLM91ZzOtrwmcD2bG1EFrVZgd2wAih2PjpfVT26iigpMWlVFi3dWdT7IG2c1BuSE2dGFucUOMUiNS9mJMva4wdImsPqoZ91HE+SjQ90kmm2hgcig5I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974969; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=HpfKLDi2Rx2JdfDNDS9ByjXSHqvyq73IRa5mt5fabj4=; b=ie/0Zl532RcPDqtvZjM09Q70WTQlx6eXDbCu9D5BVa9Q3YQ2+38dzsGimDfBjht2NH5CSgY+thkjfocSDQgLuplkkA+Ctispz9LmpeWUKG/Huaxa4PKN/9jc/fy1ps17pHh9WfeDoBSb33WGnSQjWboG5CxDBleboXc7eLbqxNU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997496992194.86822235540978; Tue, 1 Oct 2019 17:09:29 -0700 (PDT) Received: from localhost ([::1]:49580 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSCu-0001Le-DL for importer@patchew.org; Tue, 01 Oct 2019 20:09:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39471) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrv-0001Mw-K0 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrt-0001k7-BR for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:47 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:25498 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrl-0001Qr-Tv; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPOW179355; Tue, 1 Oct 2019 19:47:30 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncr7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:29 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlT3023573; Tue, 1 Oct 2019 23:47:29 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577gxh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:29 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlRnn4129426 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:27 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B94672805A; Tue, 1 Oct 2019 23:47:27 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9CFA328058; Tue, 1 Oct 2019 23:47:27 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:27 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 30/97] target/ppc: Fix lxvw4x, lxvh8x and lxvb16x Date: Tue, 1 Oct 2019 18:45:09 -0500 Message-Id: <20191001234616.7825-31-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=955 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anton Blanchard , qemu-stable@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Anton Blanchard During the conversion these instructions were incorrectly treated as stores. We need to use set_cpu_vsr* and not get_cpu_vsr*. Fixes: 8b3b2d75c7c0 ("introduce get_cpu_vsr{l,h}() and set_cpu_vsr{l,h}() h= elpers for VSR register access") Signed-off-by: Anton Blanchard Reviewed-by: Mark Cave-Ayland Tested-by: Greg Kurz Reviewed-by: Greg Kurz Message-Id: <20190524065345.25591-1-mark.cave-ayland@ilande.co.uk> Signed-off-by: David Gibson (upstream commit 2a1224359008e23b051b7b45be4789afa0269f8c) Acked-by: David Gibson Signed-off-by: Michael Roth --- target/ppc/translate/vsx-impl.inc.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/target/ppc/translate/vsx-impl.inc.c b/target/ppc/translate/vsx= -impl.inc.c index 7778d5d651..9e59e2a309 100644 --- a/target/ppc/translate/vsx-impl.inc.c +++ b/target/ppc/translate/vsx-impl.inc.c @@ -102,8 +102,7 @@ static void gen_lxvw4x(DisasContext *ctx) } xth =3D tcg_temp_new_i64(); xtl =3D tcg_temp_new_i64(); - get_cpu_vsrh(xth, xT(ctx->opcode)); - get_cpu_vsrl(xtl, xT(ctx->opcode)); + gen_set_access_type(ctx, ACCESS_INT); EA =3D tcg_temp_new(); =20 @@ -126,6 +125,8 @@ static void gen_lxvw4x(DisasContext *ctx) tcg_gen_addi_tl(EA, EA, 8); tcg_gen_qemu_ld_i64(xtl, EA, ctx->mem_idx, MO_BEQ); } + set_cpu_vsrh(xT(ctx->opcode), xth); + set_cpu_vsrl(xT(ctx->opcode), xtl); tcg_temp_free(EA); tcg_temp_free_i64(xth); tcg_temp_free_i64(xtl); @@ -185,8 +186,6 @@ static void gen_lxvh8x(DisasContext *ctx) } xth =3D tcg_temp_new_i64(); xtl =3D tcg_temp_new_i64(); - get_cpu_vsrh(xth, xT(ctx->opcode)); - get_cpu_vsrl(xtl, xT(ctx->opcode)); gen_set_access_type(ctx, ACCESS_INT); =20 EA =3D tcg_temp_new(); @@ -197,6 +196,8 @@ static void gen_lxvh8x(DisasContext *ctx) if (ctx->le_mode) { gen_bswap16x8(xth, xtl, xth, xtl); } + set_cpu_vsrh(xT(ctx->opcode), xth); + set_cpu_vsrl(xT(ctx->opcode), xtl); tcg_temp_free(EA); tcg_temp_free_i64(xth); tcg_temp_free_i64(xtl); @@ -214,14 +215,14 @@ static void gen_lxvb16x(DisasContext *ctx) } xth =3D tcg_temp_new_i64(); xtl =3D tcg_temp_new_i64(); - get_cpu_vsrh(xth, xT(ctx->opcode)); - get_cpu_vsrl(xtl, xT(ctx->opcode)); gen_set_access_type(ctx, ACCESS_INT); EA =3D tcg_temp_new(); gen_addr_reg_index(ctx, EA); tcg_gen_qemu_ld_i64(xth, EA, ctx->mem_idx, MO_BEQ); tcg_gen_addi_tl(EA, EA, 8); tcg_gen_qemu_ld_i64(xtl, EA, ctx->mem_idx, MO_BEQ); + set_cpu_vsrh(xT(ctx->opcode), xth); + set_cpu_vsrl(xT(ctx->opcode), xtl); tcg_temp_free(EA); tcg_temp_free_i64(xth); tcg_temp_free_i64(xtl); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974542; cv=none; d=zoho.com; s=zohoarc; b=j089ufmsnjTXND6MaFYWSOG68Oc7vdW5GDPluYntq+vfUgZplLQ8muKsh/+uUDD0237ZQZLmwfeLA+EprOQZ4h0a/eHg2LnkauFKhqgUunHuh1PSWGVPUY/X59A4wfTLzamRAoVoP56xF1Oprw24eh5Yuwzi4HT0aQU5aE0/DZs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974542; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=5NOCi6QzcJFX8tt6I0FZsLJxjExNAGJ66cC710Dty0U=; b=JT+twljJodjDVwXauDiGfSdKd/Wsr6byMjkfKqfkt0hZ31B73NkLRQWRzTL+4t+HaovUNhz+R5q1gIpBtenvqmW5nW0Nth3OLXFSY+PWk/kap82fXMgfmHxGAdLAiH520RJT3BuZKhc0a7S1OsfiJ36vW03V5KcOfNWb9p9p2gc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15699745429331000.9069141979459; Tue, 1 Oct 2019 17:02:22 -0700 (PDT) Received: from localhost ([::1]:49516 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFS5w-0004xC-Nb for importer@patchew.org; Tue, 01 Oct 2019 20:02:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39311) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrs-0001HY-71 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrn-0001Yi-37 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:43 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:9626 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrm-0001PA-Mp; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlO5q122835; Tue, 1 Oct 2019 19:47:30 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgk4094s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:30 -0400 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlTop123007; Tue, 1 Oct 2019 19:47:29 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgk4094j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:29 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlX7001425; Tue, 1 Oct 2019 23:47:29 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma03wdc.us.ibm.com with ESMTP id 2v9y56yf6s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:29 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlSm744302770 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:28 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BC89428058; Tue, 1 Oct 2019 23:47:28 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9F9C02805A; Tue, 1 Oct 2019 23:47:28 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:28 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 31/97] q35: Revert to kernel irqchip Date: Tue, 1 Oct 2019 18:45:10 -0500 Message-Id: <20191001234616.7825-32-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Williamson , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alex Williamson Backport of QEMU v4.1 commit for stable v4.0.1 release commit c87759ce876a7a0b17c2bf4f0b964bd51f0ee871 Author: Alex Williamson Date: Tue May 14 14:14:41 2019 -0600 q35: Revert to kernel irqchip Commit b2fc91db8447 ("q35: set split kernel irqchip as default") changed the default for the pc-q35-4.0 machine type to use split irqchip, which turned out to have disasterous effects on vfio-pci INTx support. KVM resampling irqfds are registered for handling these interrupts, but these are non-functional in split irqchip mode. We can't simply test for split irqchip in QEMU as userspace handling of this interrupt is a significant performance regression versus KVM handling (GeForce GPUs assigned to Windows VMs are non-functional without forcing MSI mode or re-enabling kernel irqchip). The resolution is to revert the change in default irqchip mode in the pc-q35-4.1 machine and create a pc-q35-4.0.1 machine for the 4.0-stable branch. The qemu-q35-4.0 machine type should not be used in vfio-pci configurations for devices requiring legacy INTx support without explicitly modifying the VM configuration to use kernel irqchip. Link: https://bugs.launchpad.net/qemu/+bug/1826422 Fixes: b2fc91db8447 ("q35: set split kernel irqchip as default") Cc: qemu-stable@nongnu.org Reviewed-by: Peter Xu Signed-off-by: Alex Williamson (upstream commit c87759ce876a7a0b17c2bf4f0b964bd51f0ee871) Reviewed-by: Michael S. Tsirkin *add comments regarding AML mismatch warnings from tests/bios-tables-test.c Signed-off-by: Michael Roth --- hw/core/machine.c | 3 +++ hw/i386/pc.c | 3 +++ hw/i386/pc_q35.c | 16 ++++++++++++++-- include/hw/boards.h | 3 +++ include/hw/i386/pc.h | 3 +++ tests/bios-tables-test.c | 9 +++++++++ 6 files changed, 35 insertions(+), 2 deletions(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index b6766632d9..eb34f53a85 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -24,6 +24,9 @@ #include "hw/pci/pci.h" #include "hw/mem/nvdimm.h" =20 +GlobalProperty hw_compat_4_0[] =3D {}; +const size_t hw_compat_4_0_len =3D G_N_ELEMENTS(hw_compat_4_0); + GlobalProperty hw_compat_3_1[] =3D { { "pcie-root-port", "x-speed", "2_5" }, { "pcie-root-port", "x-width", "1" }, diff --git a/hw/i386/pc.c b/hw/i386/pc.c index f2c15bf1f2..d98b737b8f 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -115,6 +115,9 @@ struct hpet_fw_config hpet_cfg =3D {.count =3D UINT8_MA= X}; /* Physical Address of PVH entry point read from kernel ELF NOTE */ static size_t pvh_start_addr; =20 +GlobalProperty pc_compat_4_0[] =3D {}; +const size_t pc_compat_4_0_len =3D G_N_ELEMENTS(pc_compat_4_0); + GlobalProperty pc_compat_3_1[] =3D { { "intel-iommu", "dma-drain", "off" }, { "Opteron_G3" "-" TYPE_X86_CPU, "rdtscp", "off" }, diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index 372c6b73be..45cc29d1ad 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -357,7 +357,7 @@ static void pc_q35_machine_options(MachineClass *m) m->units_per_default_bus =3D 1; m->default_machine_opts =3D "firmware=3Dbios-256k.bin"; m->default_display =3D "std"; - m->default_kernel_irqchip_split =3D true; + m->default_kernel_irqchip_split =3D false; m->no_floppy =3D 1; machine_class_allow_dynamic_sysbus_dev(m, TYPE_AMD_IOMMU_DEVICE); machine_class_allow_dynamic_sysbus_dev(m, TYPE_INTEL_IOMMU_DEVICE); @@ -365,12 +365,24 @@ static void pc_q35_machine_options(MachineClass *m) m->max_cpus =3D 288; } =20 -static void pc_q35_4_0_machine_options(MachineClass *m) +static void pc_q35_4_0_1_machine_options(MachineClass *m) { pc_q35_machine_options(m); m->alias =3D "q35"; } =20 +DEFINE_Q35_MACHINE(v4_0_1, "pc-q35-4.0.1", NULL, + pc_q35_4_0_1_machine_options); + +static void pc_q35_4_0_machine_options(MachineClass *m) +{ + pc_q35_4_0_1_machine_options(m); + m->default_kernel_irqchip_split =3D true; + m->alias =3D NULL; + compat_props_add(m->compat_props, hw_compat_4_0, hw_compat_4_0_len); + compat_props_add(m->compat_props, pc_compat_4_0, pc_compat_4_0_len); +} + DEFINE_Q35_MACHINE(v4_0, "pc-q35-4.0", NULL, pc_q35_4_0_machine_options); =20 diff --git a/include/hw/boards.h b/include/hw/boards.h index e231860666..fe1885cbff 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -293,6 +293,9 @@ struct MachineState { } \ type_init(machine_initfn##_register_types) =20 +extern GlobalProperty hw_compat_4_0[]; +extern const size_t hw_compat_4_0_len; + extern GlobalProperty hw_compat_3_1[]; extern const size_t hw_compat_3_1_len; =20 diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index ca65ef18af..43df7230a2 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -293,6 +293,9 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t); int e820_get_num_entries(void); bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *); =20 +extern GlobalProperty pc_compat_4_0[]; +extern const size_t pc_compat_4_0_len; + extern GlobalProperty pc_compat_3_1[]; extern const size_t pc_compat_3_1_len; =20 diff --git a/tests/bios-tables-test.c b/tests/bios-tables-test.c index a506dcbb29..996599991e 100644 --- a/tests/bios-tables-test.c +++ b/tests/bios-tables-test.c @@ -354,6 +354,15 @@ static void test_acpi_asl(test_data *data) } else { sdt->tmp_files_retain =3D true; exp_sdt->tmp_files_retain =3D true; + /* NOTE: In QEMU 4.0.1, q35 aliases "pc-q35-4.0.1" instead= of + * "pc-q35-4.0". This gets encoded into the smbios tables = and + * due to the extra characters changes the offset of ACPI + * pointer values allocated by SeaBIOS, causing some tests= to + * report AML mismatches. Since ACPI expected file paths r= ely + * on the "q35" alias being used we can't easily change the + * tests to use pc-q35-4.0, so for now hopefully this frie= ndly + * note will suffice. + */ fprintf(stderr, "acpi-test: Warning! %.4s mismatch. " "Actual [asl:%s, aml:%s], Expected [asl:%s, aml:%s= ].\n", --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975709; cv=none; d=zoho.com; s=zohoarc; b=Bn0KDVNaDHGkAHXlC4FOeYhIC0KEDucXrGhbGYYh12wEXskNPCWHWJi+zzEx4vGfH3sMc+HBwZMDxIEzUA0/N8zKcXb2a2mnmZall/ODK1Ev8DqTVPMUpKmhJFwgJNtBTWsQpfHkEk5FYdqingxI9Ut/A4D43YPTuX8Et72F9xc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975709; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=MbLQRpEulFCTTdv6NPjiQ9km0TVzITQ4itJpjSQJydk=; b=gEO7k2xBSLhfcSKJDNTNSNar52tzcOvKiv+p4WBrIC6Ycj1rsZ6Eei/u3uewVVtvjHsXxbpYJ6ccJ86jFpazuvWNzRsiLto8IwbllLXPQl6ilZzgDbBFNOUPlYl1cXkd3r2lFM1/LRZ7gdYJlSSi5+80p5PRFSKQfbt4Pan1NyI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975709556207.13088877997802; Tue, 1 Oct 2019 17:21:49 -0700 (PDT) Received: from localhost ([::1]:49680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSOp-0003ux-Pz for importer@patchew.org; Tue, 01 Oct 2019 20:21:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39757) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zf-Ct for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00023N-6U for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:11104) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs6-0001Q9-4x; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlP00088131; Tue, 1 Oct 2019 19:47:31 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bt8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:31 -0400 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlUjv088342; Tue, 1 Oct 2019 19:47:31 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bsw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:30 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk0C006371; Tue, 1 Oct 2019 23:47:29 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye7p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:29 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlTEl52429298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:29 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4576A2805C; Tue, 1 Oct 2019 23:47:29 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 294FE28058; Tue, 1 Oct 2019 23:47:29 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:29 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 32/97] vl: Fix -drive / -blockdev persistent reservation management Date: Tue, 1 Oct 2019 18:45:11 -0500 Message-Id: <20191001234616.7825-33-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , qemu-stable@nongnu.org, Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Markus Armbruster qemu-system-FOO's main() acts on command line arguments in its own idiosyncratic order. There's not much method to its madness. Whenever we find a case where one kind of command line argument needs to refer to something created for another kind later, we rejigger the order. Recent commit cda4aa9a5a "vl: Create block backends before setting machine properties" was such a rejigger. Block backends are now created before "delayed" objects. This broke persistent reservation management. Reproducer: $ qemu-system-x86_64 -object pr-manager-helper,id=3Dpr-helper0,path=3D/= tmp/pr-helper0.sock-drive -drive file=3D/dev/mapper/crypt,file.pr-manager= =3Dpr-helper0,format=3Draw,if=3Dnone,id=3Ddrive-scsi0-0-0-2 qemu-system-x86_64: -drive file=3D/dev/mapper/crypt,file.pr-manager=3Dp= r-helper0,format=3Draw,if=3Dnone,id=3Ddrive-scsi0-0-0-2: No persistent rese= rvation manager with id 'pr-helper0' The delayed pr-manager-helper object is created too late for use by -drive or -blockdev. Normal objects are still created in time. pr-manager-helper has always been a delayed object (commit 7c9e527659 "scsi, file-posix: add support for persistent reservation management"). Turns out there's no real reason for that. Make it a normal object. Fixes: cda4aa9a5a08777cf13e164c0543bd4888b8adce Signed-off-by: Markus Armbruster Message-Id: <20190604151251.9903-2-armbru@redhat.com> Reviewed-by: Michal Privoznik Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini (cherry picked from commit 9ea18ed25a36527167e9676f25d983df5e7f76e6) Signed-off-by: Michael Roth --- vl.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/vl.c b/vl.c index c696ad2a13..c3dee09c9b 100644 --- a/vl.c +++ b/vl.c @@ -2825,8 +2825,7 @@ static bool object_create_initial(const char *type, Q= emuOpts *opts) exit(0); } =20 - if (g_str_equal(type, "rng-egd") || - g_str_has_prefix(type, "pr-manager-")) { + if (g_str_equal(type, "rng-egd")) { return false; } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975251; cv=none; d=zoho.com; s=zohoarc; b=NeuAxiJ1ZoIGkCTvhqL8/s4R5uSTLRrpLOoTrNJLi9lKWdbWVM1TNdJ9RXjCYOuteyqwSiaOQLOczobsHFV+bTsdRKJPiRZ32NhRjUOSEmlMHx+ky9oYSDn1S8u3izcXQrM2zhCMghSFfHAsS/XrkcFExIMTFCVyCzyOyJLIs0M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975251; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=qz4Qey6VexDxLrg+gRbTTIAAIZoONOe3isjwzjnQBM8=; b=MZmGt0phTyvAv6xGv1PNOOO4V64SMMT6Q1EecOS9DayQXnAnyMQTuBxtSHMeZEuuu6RjsaQW784s0CHhwDQoDLhfjg7NRUwEAUHREZE4ieix8A1zPKCjL6nj7S9vZZWNJ6GnhPL0csN8lJ1J2g31cJx08WyaGqMYXZdQjRTkipw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975251635348.0796476161378; Tue, 1 Oct 2019 17:14:11 -0700 (PDT) Received: from localhost ([::1]:49606 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSHR-0004ii-Ob for importer@patchew.org; Tue, 01 Oct 2019 20:14:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39221) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrq-0001EH-7v for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrm-0001Y7-Sr for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:41 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:11822 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrm-0001Q4-Ej; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPGX179294; Tue, 1 Oct 2019 19:47:31 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncrt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:31 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlUIA179537; Tue, 1 Oct 2019 19:47:30 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncrj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:30 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkYB031624; Tue, 1 Oct 2019 23:47:30 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fb0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:30 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlTiH43778410 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:29 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BE7C22805A; Tue, 1 Oct 2019 23:47:29 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A234028059; Tue, 1 Oct 2019 23:47:29 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:29 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 33/97] target/i386: add MDS-NO feature Date: Tue, 1 Oct 2019 18:45:12 -0500 Message-Id: <20191001234616.7825-34-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=755 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Oguz Bektas , qemu-stable@nongnu.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Microarchitectural Data Sampling is a hardware vulnerability which allows unprivileged speculative access to data which is available in various CPU internal buffers. Some Intel processors use the ARCH_CAP_MDS_NO bit in the IA32_ARCH_CAPABILITIES MSR to report that they are not vulnerable, make it available to guests. Signed-off-by: Paolo Bonzini Message-Id: <20190516185320.28340-1-pbonzini@redhat.com> Signed-off-by: Eduardo Habkost (cherry picked from commit 20140a82c67467f53814ca197403d5e1b561a5e5) Signed-off-by: Oguz Bektas Signed-off-by: Michael Roth --- target/i386/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index d6bb57d210..ee4b8b47e2 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -1183,7 +1183,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORD= S] =3D { .type =3D MSR_FEATURE_WORD, .feat_names =3D { "rdctl-no", "ibrs-all", "rsba", "skip-l1dfl-vmentry", - "ssb-no", NULL, NULL, NULL, + "ssb-no", "mds-no", NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569974214; cv=none; d=zoho.com; s=zohoarc; b=JgQ5Mp7ZaW5w2FlsT/itDlImSN7dYNDF5ptpVGNO73hlqwxGcEh22skDLleniXkFyCXmhJmWAjdSVoiveabrRAwTf0/L2N8zqUdIe9DG/w8qonhSdCI+TNC2iWfEXDMk0fPANqqmJYQlX+hhiRxwSWecAiY4vQ9KZJyitk1dzr0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569974214; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ZGCifkc9xOsSDWCsO4PptFq+u/dSk95kHBjn+AbHubM=; b=eRspEQnVvR3+XBvHGqBBl3YQM+I39+j5b892hkzywRawxj1RSd+QiBKPepHbp8JxPw5SCKmCfvbIf/Y2mhhHaQot8rE0EUxWO9LA2gmXZzCfv06/dGV7VBe56Q2HoMoOKJ+QOAh+ACHkPBVg6Hfw6X2MaBjeoYZUwS04Y/nH4Ow= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997421469885.19369033153168; Tue, 1 Oct 2019 16:56:54 -0700 (PDT) Received: from localhost ([::1]:49444 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFS0h-0000Vj-Uk for importer@patchew.org; Tue, 01 Oct 2019 19:56:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39246) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrq-0001F0-LL for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrm-0001XU-N0 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:42 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:18000) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrm-0001Qx-1I; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQn5074872; Tue, 1 Oct 2019 19:47:32 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpu9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:32 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlPWO074813; Tue, 1 Oct 2019 19:47:32 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpts-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:32 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk0D006371; Tue, 1 Oct 2019 23:47:30 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye7t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:30 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlUXS53674352 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:30 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3D2272805E; Tue, 1 Oct 2019 23:47:30 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 20C1128059; Tue, 1 Oct 2019 23:47:30 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:30 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 34/97] target/i386: define md-clear bit Date: Tue, 1 Oct 2019 18:45:13 -0500 Message-Id: <20191001234616.7825-35-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=710 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Oguz Bektas , qemu-stable@nongnu.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini md-clear is a new CPUID bit which is set when microcode provides the mechanism to invoke a flush of various exploitable CPU buffers by invoking the VERW instruction. Signed-off-by: Paolo Bonzini Message-Id: <20190515141011.5315-2-berrange@redhat.com> Signed-off-by: Eduardo Habkost (cherry picked from commit b2ae52101fca7f9547ac2f388085dbc58f8fe1c0) Signed-off-by: Oguz Bektas Signed-off-by: Michael Roth --- target/i386/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index ee4b8b47e2..331a364a1b 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -1076,7 +1076,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORD= S] =3D { .feat_names =3D { NULL, NULL, "avx512-4vnniw", "avx512-4fmaps", NULL, NULL, NULL, NULL, - NULL, NULL, NULL, NULL, + NULL, NULL, "md-clear", NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976324; cv=none; d=zoho.com; s=zohoarc; b=ZOVjVvtr4Vpos2PNSM6qRvTUC86zm9oHN4dU0I7pWiVEq+9mQ3U+cT0TStV4Q0wA+kBN4nZi43hOGnACIPbdnxg6yLIBm1VZ/sAO3XTvEhUqyuBATrWnPBktwOoQur0sKU2L5edApdamuQJ5D2MXH+7MeTizwn9PixthwzQ4WAM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976324; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=pRyHMOkcIf//6lrlQHuj7oTzipzy+l8CDNuUNwm4HZ8=; b=OSGZdT56qqv+Q5WWT2FfMrd9GD4M88h5d4435/DkJktYp+1Z4XJbdgHXim/0xQdZHx6seBVgSVbOG6cDfILsvJgkbAtH74iFVS1DE7f71YAmSFVjUv2I+eglWdY6dhSNlq+0yzhxWdsRO0c+7bFI/kus5DnTOLmvVORLbFGUj5M= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976324387867.4906554460522; Tue, 1 Oct 2019 17:32:04 -0700 (PDT) Received: from localhost ([::1]:49742 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSYi-0003xc-1z for importer@patchew.org; Tue, 01 Oct 2019 20:32:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39760) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zj-E7 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00023z-8d for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:1514) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs6-0001R1-8e; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlRp2086527; Tue, 1 Oct 2019 19:47:33 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat80s7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:32 -0400 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlWYu086722; Tue, 1 Oct 2019 19:47:32 -0400 Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat80rw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:32 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkON022431; Tue, 1 Oct 2019 23:47:31 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04dal.us.ibm.com with ESMTP id 2v9y57yd64-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:31 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlU3H50987414 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:30 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BB1F428059; Tue, 1 Oct 2019 23:47:30 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9EF2428058; Tue, 1 Oct 2019 23:47:30 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:30 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 35/97] docs: recommend use of md-clear feature on all Intel CPUs Date: Tue, 1 Oct 2019 18:45:14 -0500 Message-Id: <20191001234616.7825-36-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NlRp2086527 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Oguz Bektas , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , qemu-stable@nongnu.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Daniel P. Berrang=C3=A9 Update x86 CPU model guidance to recommend that the md-clear feature is manually enabled with all Intel CPU models, when supported by the host microcode. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20190515141011.5315-3-berrange@redhat.com> Signed-off-by: Eduardo Habkost (cherry picked from commit 2c7e82a30774730100da9dbe68d2360459030d91) Signed-off-by: Oguz Bektas Signed-off-by: Michael Roth --- docs/qemu-cpu-models.texi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/qemu-cpu-models.texi b/docs/qemu-cpu-models.texi index 23c11dc86f..ad040cfc98 100644 --- a/docs/qemu-cpu-models.texi +++ b/docs/qemu-cpu-models.texi @@ -200,6 +200,18 @@ Not included by default in any Intel CPU model. Should be explicitly turned on for all Intel CPU models. =20 Note that not all CPU hardware will support this feature. + +@item @code{md-clear} + +Required to confirm the MDS (CVE-2018-12126, CVE-2018-12127, CVE-2018-1213= 0, +CVE-2019-11091) fixes. + +Not included by default in any Intel CPU model. + +Must be explicitly turned on for all Intel CPU models. + +Requires the host CPU microcode to support this feature before it +can be used for guest CPUs. @end table =20 =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976283; cv=none; d=zoho.com; s=zohoarc; b=S61a76i8BdvkcwedhF1svF0J+3DohHOFLnE4TavSKaPni0yvV6oipnwm/WH8fGOvcTq/qpfY3A21fzgKo+dSHIBP6/VSZnq2T8KRxUYRNm1v+P/jGFi0RMxcqokR4j3St43SPYP63OR8BUKOJh3F1x20Xgwu8iY7P/8IaQ670Ic= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976283; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=fXjEzwYQnjWJ0BPhvaaxaF34PyhrRhe9fQ0wuVVuylM=; b=dqPXSLQhqbWYoVpiWP8cwmOjOVfL9ofrlCiU0HveZqBkpo3zMhnpel+lp4g98nHbnxs/VWIbwnhISpkPLNS6ld1yluFTb9Az6nqxENFl3fswaVGmq+kSwF0ZNqbeWlMTKzZxvVnhCd+74FqpAZFnT8/ksVP/A+WbkPdFkGY12+U= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976283717357.7680930007027; Tue, 1 Oct 2019 17:31:23 -0700 (PDT) Received: from localhost ([::1]:49740 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSY6-0003js-EF for importer@patchew.org; Tue, 01 Oct 2019 20:31:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39847) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsD-0001bX-BW for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00024c-FB for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:05 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:7900 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs7-0001Qh-3Q; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlP0n122864; Tue, 1 Oct 2019 19:47:32 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgk4095m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:32 -0400 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlWPS123064; Tue, 1 Oct 2019 19:47:32 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgk4095f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:32 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlOx031629; Tue, 1 Oct 2019 23:47:31 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fb5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:31 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlVJu55443880 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:31 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 437AC2805C; Tue, 1 Oct 2019 23:47:31 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 284CF28059; Tue, 1 Oct 2019 23:47:31 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:31 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 36/97] virtio-pci: fix missing device properties Date: Tue, 1 Oct 2019 18:45:15 -0500 Message-Id: <20191001234616.7825-37-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=4 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NlP0n122864 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Marc-Andr=C3=A9 Lureau Since commit a4ee4c8baa37154 ("virtio: Helper for registering virtio device types"), virtio-gpu-pci, virtio-vga, and virtio-crypto-pci lost some properties: "ioeventfd" and "vectors". This may cause various issues, such as failing migration or invalid properties. Since those VirtioPCI devices do not have a base name, their class are initialized with virtio_pci_generic_base_class_init(). However, if the VirtioPCIDeviceTypeInfo provided a class_init which sets dc->props, the properties were overwritten by virtio_pci_generic_class_init(). Instead, introduce an intermediary base-type to register the generic properties. Fixes: a4ee4c8baa37154f42b4dc6a13fee79268d15238 Cc: qemu-stable@nongnu.org Signed-off-by: Marc-Andr=C3=A9 Lureau Message-Id: <20190625232333.30752-1-marcandre.lureau@redhat.com> (cherry picked from commit 683c1d89efd1eeb111c129a9a91f629b94d90d45) Signed-off-by: Michael Roth --- hw/virtio/virtio-pci.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index cb44e19b67..497092e83c 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1905,13 +1905,6 @@ static void virtio_pci_generic_class_init(ObjectClas= s *klass, void *data) dc->props =3D virtio_pci_generic_properties; } =20 -/* Used when the generic type and the base type is the same */ -static void virtio_pci_generic_base_class_init(ObjectClass *klass, void *d= ata) -{ - virtio_pci_base_class_init(klass, data); - virtio_pci_generic_class_init(klass, NULL); -} - static void virtio_pci_transitional_instance_init(Object *obj) { VirtIOPCIProxy *proxy =3D VIRTIO_PCI(obj); @@ -1930,14 +1923,13 @@ static void virtio_pci_non_transitional_instance_in= it(Object *obj) =20 void virtio_pci_types_register(const VirtioPCIDeviceTypeInfo *t) { + char *base_name =3D NULL; TypeInfo base_type_info =3D { .name =3D t->base_name, .parent =3D t->parent ? t->parent : TYPE_VIRTIO_PCI, .instance_size =3D t->instance_size, .instance_init =3D t->instance_init, .class_size =3D t->class_size, - .class_init =3D virtio_pci_base_class_init, - .class_data =3D (void *)t, .abstract =3D true, }; TypeInfo generic_type_info =3D { @@ -1953,13 +1945,20 @@ void virtio_pci_types_register(const VirtioPCIDevic= eTypeInfo *t) =20 if (!base_type_info.name) { /* No base type -> register a single generic device type */ - base_type_info.name =3D t->generic_name; - base_type_info.class_init =3D virtio_pci_generic_base_class_init; - base_type_info.interfaces =3D generic_type_info.interfaces; - base_type_info.abstract =3D false; - generic_type_info.name =3D NULL; + /* use intermediate %s-base-type to add generic device props */ + base_name =3D g_strdup_printf("%s-base-type", t->generic_name); + base_type_info.name =3D base_name; + base_type_info.class_init =3D virtio_pci_generic_class_init; + + generic_type_info.parent =3D base_name; + generic_type_info.class_init =3D virtio_pci_base_class_init; + generic_type_info.class_data =3D (void *)t; + assert(!t->non_transitional_name); assert(!t->transitional_name); + } else { + base_type_info.class_init =3D virtio_pci_base_class_init; + base_type_info.class_data =3D (void *)t; } =20 type_register(&base_type_info); @@ -1997,6 +1996,7 @@ void virtio_pci_types_register(const VirtioPCIDeviceT= ypeInfo *t) }; type_register(&transitional_type_info); } + g_free(base_name); } =20 /* virtio-pci-bus */ --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975472; cv=none; d=zoho.com; s=zohoarc; b=ojhwZSkOQtfji/76pY0XJiBG4w2NZzCG5lyHCmteoC+zCq6kcPpBfp0b8g4VP3kBzpsvlz3UDtW7frX+juM1vL8qCexLf73+2Bukkn6qBStm9cgAepudb3LJfOuCHlcIPKpJk2IcEGMAYVKkBf15JYppLMWOUgsGc12DnuuDR6w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975472; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Buqc7jEMstHA4wqLAcol6ZjYbGfPeJj0FVP7fyPfLHM=; b=A52kMztC5II4HWKFIKybjN5Oi4xJNBshGjvVp3i5EcrRB/OjHq+Kjv85hFLxVlFWsH1Z58M4gidTK3wV1PK9Q1X5fEajfiszS47I+xcAFrz5wcjRnNDKrbX7tZyoyEp5Wbly3jf3F3U8BYGUsUX1cMyCTk/8EjnbxcSWwWBaG/Y= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975472880914.5938402314976; Tue, 1 Oct 2019 17:17:52 -0700 (PDT) Received: from localhost ([::1]:49634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSL1-0008Bn-87 for importer@patchew.org; Tue, 01 Oct 2019 20:17:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39218) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrq-0001ED-6E for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrm-0001Wd-9i for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:41 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:21910 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrl-0001RW-QJ; Tue, 01 Oct 2019 19:47:37 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQR2037587; Tue, 1 Oct 2019 19:47:33 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b327-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:33 -0400 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlXOV037810; Tue, 1 Oct 2019 19:47:33 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b321-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:33 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkJw027153; Tue, 1 Oct 2019 23:47:32 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bg0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:32 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlVaF53346612 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:31 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA96D28059; Tue, 1 Oct 2019 23:47:31 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9F68228058; Tue, 1 Oct 2019 23:47:31 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:31 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 37/97] usbredir: fix buffer-overflow on vmload Date: Tue, 1 Oct 2019 18:45:16 -0500 Message-Id: <20191001234616.7825-38-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NlQR2037587 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , qemu-stable@nongnu.org, Gerd Hoffmann Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Marc-Andr=C3=A9 Lureau If interface_count is NO_INTERFACE_INFO, let's not access the arrays out-of-bounds. =3D=3D994=3D=3DERROR: AddressSanitizer: heap-buffer-overflow on address 0x6= 25000243930 at pc 0x5642068086a8 bp 0x7f0b6f9ffa50 sp 0x7f0b6f9ffa40 READ of size 1 at 0x625000243930 thread T0 #0 0x5642068086a7 in usbredir_check_bulk_receiving /home/elmarco/src/qe= mu/hw/usb/redirect.c:1503 #1 0x56420681301c in usbredir_post_load /home/elmarco/src/qemu/hw/usb/r= edirect.c:2154 #2 0x5642068a56c2 in vmstate_load_state /home/elmarco/src/qemu/migratio= n/vmstate.c:168 #3 0x56420688e2ac in vmstate_load /home/elmarco/src/qemu/migration/save= vm.c:829 #4 0x5642068980cb in qemu_loadvm_section_start_full /home/elmarco/src/q= emu/migration/savevm.c:2211 #5 0x564206899645 in qemu_loadvm_state_main /home/elmarco/src/qemu/migr= ation/savevm.c:2395 #6 0x5642068998cf in qemu_loadvm_state /home/elmarco/src/qemu/migration= /savevm.c:2467 #7 0x56420685f3e9 in process_incoming_migration_co /home/elmarco/src/qe= mu/migration/migration.c:449 #8 0x564207106c47 in coroutine_trampoline /home/elmarco/src/qemu/util/c= oroutine-ucontext.c:115 #9 0x7f0c0604e37f (/lib64/libc.so.6+0x4d37f) Signed-off-by: Marc-Andr=C3=A9 Lureau Reviewed-by: Liam Merwick Reviewed-by: Li Qiang Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-id: 20190807084048.4258-1-marcandre.lureau@redhat.com Signed-off-by: Gerd Hoffmann (cherry picked from commit 7b84b90966568da0e05655ecaa78c209300aae6e) Signed-off-by: Michael Roth --- hw/usb/redirect.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/usb/redirect.c b/hw/usb/redirect.c index 7cb6b120d4..586c7db025 100644 --- a/hw/usb/redirect.c +++ b/hw/usb/redirect.c @@ -1494,6 +1494,11 @@ static void usbredir_check_bulk_receiving(USBRedirDe= vice *dev) for (i =3D EP2I(USB_DIR_IN); i < MAX_ENDPOINTS; i++) { dev->endpoint[i].bulk_receiving_enabled =3D 0; } + + if (dev->interface_info.interface_count =3D=3D NO_INTERFACE_INFO) { + return; + } + for (i =3D 0; i < dev->interface_info.interface_count; i++) { quirks =3D usb_get_quirks(dev->device_info.vendor_id, dev->device_info.product_id, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976442; cv=none; d=zoho.com; s=zohoarc; b=UCO3zL0ko1j5TxPfX5cK1erCuxxS0Yd8t6Xfhallx4Xao40KGlJkdHSdq1EjJLyNowW5QfWTWcqpFJmybtEMwoN6IYoGYezsOuD+NwZzl2AkNsDZNgLqu7mwPnB9m2QP7lE3QRzmspcn57urUfjv8TGRyfFse1lOTODUsoiYsek= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976442; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=sjpZz1Di8o/QNJabGSvvMUfuLHwKbMemURkcHAxDnNo=; b=ep7pfH6Im1tdBp/jNuSC9TSYKPaSvKv77qmyZViYN699Q49o/768A2T7sAIK8ilwpm9wZjve3uQYHazkgobufeuHYzmOI8lJ5wBrF5CH9eednAyNWdMXbjtbzfZa+cY0Ljcd9D/C36tLxcVV5M4v2he72rp/6B1F4uw7RuRN9og= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976442868987.666863948052; Tue, 1 Oct 2019 17:34:02 -0700 (PDT) Received: from localhost ([::1]:49762 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSaX-0005zA-5S for importer@patchew.org; Tue, 01 Oct 2019 20:33:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39772) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zm-HF for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00023w-9n for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:31202 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs6-0001Rm-V8; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlP46078942; Tue, 1 Oct 2019 19:47:34 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00u2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:34 -0400 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlXRH079208; Tue, 1 Oct 2019 19:47:34 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00tr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:33 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlYh029380; Tue, 1 Oct 2019 23:47:33 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fahc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:33 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlWUG49021242 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:32 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 41CA32805A; Tue, 1 Oct 2019 23:47:32 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2602E28058; Tue, 1 Oct 2019 23:47:32 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:32 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 38/97] virtio-balloon: fix QEMU 4.0 config size migration incompatibility Date: Tue, 1 Oct 2019 18:45:17 -0500 Message-Id: <20191001234616.7825-39-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Stefan Hajnoczi , "Michael S . Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi The virtio-balloon config size changed in QEMU 4.0 even for existing machine types. Migration from QEMU 3.1 to 4.0 can fail in some circumstances with the following error: qemu-system-x86_64: get_pci_config_device: Bad config data: i=3D0x10 read= : a1 device: 1 cmask: ff wmask: c0 w1cmask:0 This happens because the virtio-balloon config size affects the VIRTIO Legacy I/O Memory PCI BAR size. Introduce a qdev property called "qemu-4-0-config-size" and enable it only for the QEMU 4.0 machine types. This way <4.0 machine types use the old size, 4.0 uses the larger size, and >4.0 machine types use the appropriate size depending on enabled virtio-balloon features. Live migration to and from old QEMUs to QEMU 4.1 works again as long as a versioned machine type is specified (do not use just "pc"!). Originally-by: Wolfgang Bumiller Signed-off-by: Stefan Hajnoczi Message-Id: <20190710141440.27635-1-stefanha@redhat.com> Reviewed-by: Dr. David Alan Gilbert Tested-by: Dr. David Alan Gilbert Tested-by: Wolfgang Bumiller Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit 2bbadb08ce272d65e1f78621002008b07d1e0f03) Conflicts: hw/core/machine.c * drop context dep. on 0a71966253c Signed-off-by: Michael Roth --- hw/core/machine.c | 5 ++++- hw/virtio/virtio-balloon.c | 28 +++++++++++++++++++++++++--- include/hw/virtio/virtio-balloon.h | 2 ++ 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index eb34f53a85..55b08f1466 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -24,7 +24,9 @@ #include "hw/pci/pci.h" #include "hw/mem/nvdimm.h" =20 -GlobalProperty hw_compat_4_0[] =3D {}; +GlobalProperty hw_compat_4_0[] =3D { + { "virtio-balloon-device", "qemu-4-0-config-size", "true" }, +}; const size_t hw_compat_4_0_len =3D G_N_ELEMENTS(hw_compat_4_0); =20 GlobalProperty hw_compat_3_1[] =3D { @@ -39,6 +41,7 @@ GlobalProperty hw_compat_3_1[] =3D { { "usb-tablet", "serial", "42" }, { "virtio-blk-device", "discard", "false" }, { "virtio-blk-device", "write-zeroes", "false" }, + { "virtio-balloon-device", "qemu-4-0-config-size", "false" }, }; const size_t hw_compat_3_1_len =3D G_N_ELEMENTS(hw_compat_3_1); =20 diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 2112874055..5579260fd4 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -615,6 +615,22 @@ virtio_balloon_free_page_report_notify(NotifierWithRet= urn *n, void *data) return 0; } =20 +static size_t virtio_balloon_config_size(VirtIOBalloon *s) +{ + uint64_t features =3D s->host_features; + + if (s->qemu_4_0_config_size) { + return sizeof(struct virtio_balloon_config); + } + if (virtio_has_feature(features, VIRTIO_BALLOON_F_PAGE_POISON)) { + return sizeof(struct virtio_balloon_config); + } + if (virtio_has_feature(features, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) { + return offsetof(struct virtio_balloon_config, poison_val); + } + return offsetof(struct virtio_balloon_config, free_page_report_cmd_id); +} + static void virtio_balloon_get_config(VirtIODevice *vdev, uint8_t *config_= data) { VirtIOBalloon *dev =3D VIRTIO_BALLOON(vdev); @@ -635,7 +651,7 @@ static void virtio_balloon_get_config(VirtIODevice *vde= v, uint8_t *config_data) } =20 trace_virtio_balloon_get_config(config.num_pages, config.actual); - memcpy(config_data, &config, sizeof(struct virtio_balloon_config)); + memcpy(config_data, &config, virtio_balloon_config_size(dev)); } =20 static int build_dimm_list(Object *obj, void *opaque) @@ -679,7 +695,7 @@ static void virtio_balloon_set_config(VirtIODevice *vde= v, uint32_t oldactual =3D dev->actual; ram_addr_t vm_ram_size =3D get_current_ram_size(); =20 - memcpy(&config, config_data, sizeof(struct virtio_balloon_config)); + memcpy(&config, config_data, virtio_balloon_config_size(dev)); dev->actual =3D le32_to_cpu(config.actual); if (dev->actual !=3D oldactual) { qapi_event_send_balloon_change(vm_ram_size - @@ -766,7 +782,7 @@ static void virtio_balloon_device_realize(DeviceState *= dev, Error **errp) int ret; =20 virtio_init(vdev, "virtio-balloon", VIRTIO_ID_BALLOON, - sizeof(struct virtio_balloon_config)); + virtio_balloon_config_size(s)); =20 ret =3D qemu_add_balloon_handler(virtio_balloon_to_target, virtio_balloon_stat, s); @@ -897,6 +913,12 @@ static Property virtio_balloon_properties[] =3D { VIRTIO_BALLOON_F_DEFLATE_ON_OOM, false), DEFINE_PROP_BIT("free-page-hint", VirtIOBalloon, host_features, VIRTIO_BALLOON_F_FREE_PAGE_HINT, false), + /* QEMU 4.0 accidentally changed the config size even when free-page-h= int + * is disabled, resulting in QEMU 3.1 migration incompatibility. This + * property retains this quirk for QEMU 4.1 machine types. + */ + DEFINE_PROP_BOOL("qemu-4-0-config-size", VirtIOBalloon, + qemu_4_0_config_size, false), DEFINE_PROP_LINK("iothread", VirtIOBalloon, iothread, TYPE_IOTHREAD, IOThread *), DEFINE_PROP_END_OF_LIST(), diff --git a/include/hw/virtio/virtio-balloon.h b/include/hw/virtio/virtio-= balloon.h index 1afafb12f6..5a99293a45 100644 --- a/include/hw/virtio/virtio-balloon.h +++ b/include/hw/virtio/virtio-balloon.h @@ -71,6 +71,8 @@ typedef struct VirtIOBalloon { int64_t stats_poll_interval; uint32_t host_features; PartiallyBalloonedPage *pbp; + + bool qemu_4_0_config_size; } VirtIOBalloon; =20 #endif --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975821; cv=none; d=zoho.com; s=zohoarc; b=dkNJgfnKRIYJP1KVJ8NZKnpaj0sgg0ExpYsp3sPzh59sYtgVw5LC56kFcswRcma/3yJQ6PfSeBJ3UFLm5HNu3Omj0GbMtF+Mg2mIQilYggfxuaUTRG+Nf6XFPZg14kW3Bu2UF53HFiQbKIvMTpnHD42FqwLsRI2f6aJy2TEWVFA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975821; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=QUNvmHEwGdwTPIEspPizfAEHwpvcp66uKjipQeTk+T0=; b=SQoOKFLR2xe1g0S1xDlSmo9PNhMqwfByrfklKyrw7D/bt6m9KDsJZOqER+JTXSL2Y8tvb/rtm/uk6C1wqXbrM+0IWoZ/hUvydWVsrTYh33yGPhyOmkUDSUEboDykp0FYvJpH3CpX4T2vcYooQ1CllEG4eZjVnGDc2YVAuYJ557M= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975821422107.56926669767529; Tue, 1 Oct 2019 17:23:41 -0700 (PDT) Received: from localhost ([::1]:49688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSQd-0005ee-Up for importer@patchew.org; Tue, 01 Oct 2019 20:23:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39692) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRs9-0001Z3-8q for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs6-00021W-BY for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:00 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:42680 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs6-0001Ri-0U; Tue, 01 Oct 2019 19:47:58 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlP3F179336; Tue, 1 Oct 2019 19:47:34 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncss-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:34 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlXZh179631; Tue, 1 Oct 2019 19:47:33 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncsm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:33 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njoct031648; Tue, 1 Oct 2019 23:47:33 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fb9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:33 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlWvR55050682 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:32 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BE11F28058; Tue, 1 Oct 2019 23:47:32 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A27292805E; Tue, 1 Oct 2019 23:47:32 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:32 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 39/97] docs/interop/bitmaps.rst: Fix typos Date: Tue, 1 Oct 2019 18:45:18 -0500 Message-Id: <20191001234616.7825-40-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Pygments and Sphinx get pickier all the time; Sphinx 2.1+ now catches these errors. Signed-off-by: John Snow Reported-by: Aarushi Mehta Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20190603214653.29369-2-jsnow@redhat.com Signed-off-by: John Snow (cherry picked from commit 575e6226287072bd0d6eb85d9712d280eb29c392) Signed-off-by: Michael Roth --- docs/interop/bitmaps.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst index 510e8809a9..c29ac4a854 100644 --- a/docs/interop/bitmaps.rst +++ b/docs/interop/bitmaps.rst @@ -399,7 +399,7 @@ in any one source bitmap, the target bitmap will mark t= hat segment dirty. "arguments": { "node": "drive0", "target": "new_bitmap", - "bitmaps: [ "bitmap0" ] + "bitmaps": [ "bitmap0" ] } } =20 @@ -1437,7 +1437,7 @@ applied: .. code:: json =20 <- { - "timestamp": {...} + "timestamp": {...}, "data": { "device": "drive0", "type": "backup", --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975071; cv=none; d=zoho.com; s=zohoarc; b=WuXUCj+PHbNmyh8N3iIxJZlDkC+Abz7j/2HocStNcgjrtRMxHMKOiWl00qV4HmLAniui2QLFJSy+6powUNmivvUJpS8qBBvg4pfXLRdPB3f6AosvrdlblQupdtgTERLJaJTzYvPBs8gYRGw/x4Ty52vqlUdKCnOylSIFxV+Mukc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975071; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=60HGMh7yDOTZ9Ogkzm/Rg23yK1dKkMuE8HiK31iFcOI=; b=Uw3q0k5j+RiyUYJTv0aCIyLyWLPSa1AjICTsfJhiUpeKo1ScpUgtDSKEfTsQV5dVldEW3StLTOnk+9OzUnwUnu9hY+/kX2epO7K1wuVhrGJkcqWfifnuml88nc0Ym88ouZSpDMBFVDek9wSPvSqpKGYQrY0Q8XmupbbWmvKVVSg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975071926164.84745095272626; Tue, 1 Oct 2019 17:11:11 -0700 (PDT) Received: from localhost ([::1]:49590 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSEY-0002n4-6V for importer@patchew.org; Tue, 01 Oct 2019 20:11:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39287) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRrr-0001Gr-Sx for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRrm-0001XH-LJ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:47:43 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:36370 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRrl-0001TH-So; Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPGa179294; Tue, 1 Oct 2019 19:47:35 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnctk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:35 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlP6c179311; Tue, 1 Oct 2019 19:47:35 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnct1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:35 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlYk029380; Tue, 1 Oct 2019 23:47:34 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fahk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:34 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlXXT11928134 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:33 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 428122805A; Tue, 1 Oct 2019 23:47:33 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 26E152805C; Tue, 1 Oct 2019 23:47:33 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:33 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 40/97] sphinx: add qmp_lexer Date: Tue, 1 Oct 2019 18:45:19 -0500 Message-Id: <20191001234616.7825-41-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Sphinx, through Pygments, does not like annotated json examples very much. In some versions of Sphinx (1.7), it will render the non-json portions of code blocks in red, but in newer versions (2.0) it will throw an exception and not highlight the block at all. Though we can suppress this warning, it doesn't bring back highlighting on non-strict json blocks. We can alleviate this by creating a custom lexer for QMP examples that allows us to properly highlight these examples in a robust way, keeping our directionality and elision notations. Signed-off-by: Eduardo Habkost Signed-off-by: John Snow Reported-by: Aarushi Mehta Reviewed-by: Peter Maydell Message-id: 20190603214653.29369-3-jsnow@redhat.com Signed-off-by: John Snow (cherry picked from commit cd231e13bdcb8d686b014bef940c7d19c6f1e769) Signed-off-by: Michael Roth --- docs/conf.py | 4 ++-- docs/sphinx/qmp_lexer.py | 43 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 2 deletions(-) create mode 100644 docs/sphinx/qmp_lexer.py diff --git a/docs/conf.py b/docs/conf.py index befbcc6c3e..e46b299b71 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -41,7 +41,7 @@ except NameError: # add these directories to sys.path here. If the directory is relative to = the # documentation root, use an absolute path starting from qemu_docdir. # -# sys.path.insert(0, os.path.join(qemu_docdir, "my_subdir")) +sys.path.insert(0, os.path.join(qemu_docdir, "sphinx")) =20 =20 # -- General configuration ------------------------------------------------ @@ -54,7 +54,7 @@ needs_sphinx =3D '1.3' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions =3D [] +extensions =3D ['qmp_lexer'] =20 # Add any paths that contain templates here, relative to this directory. templates_path =3D ['_templates'] diff --git a/docs/sphinx/qmp_lexer.py b/docs/sphinx/qmp_lexer.py new file mode 100644 index 0000000000..f7e4c0e198 --- /dev/null +++ b/docs/sphinx/qmp_lexer.py @@ -0,0 +1,43 @@ +# QEMU Monitor Protocol Lexer Extension +# +# Copyright (C) 2019, Red Hat Inc. +# +# Authors: +# Eduardo Habkost +# John Snow +# +# This work is licensed under the terms of the GNU GPLv2 or later. +# See the COPYING file in the top-level directory. +"""qmp_lexer is a Sphinx extension that provides a QMP lexer for code bloc= ks.""" + +from pygments.lexer import RegexLexer, DelegatingLexer +from pygments.lexers.data import JsonLexer +from pygments import token +from sphinx import errors + +class QMPExampleMarkersLexer(RegexLexer): + """ + QMPExampleMarkersLexer lexes QMP example annotations. + This lexer adds support for directionality flow and elision indicators. + """ + tokens =3D { + 'root': [ + (r'-> ', token.Generic.Prompt), + (r'<- ', token.Generic.Prompt), + (r' ?\.{3} ?', token.Generic.Prompt), + ] + } + +class QMPExampleLexer(DelegatingLexer): + """QMPExampleLexer lexes annotated QMP examples.""" + def __init__(self, **options): + super(QMPExampleLexer, self).__init__(JsonLexer, QMPExampleMarkers= Lexer, + token.Error, **options) + +def setup(sphinx): + """For use by the Sphinx extensions API.""" + try: + sphinx.require_sphinx('2.1') + sphinx.add_lexer('QMP', QMPExampleLexer) + except errors.VersionRequirementError: + sphinx.add_lexer('QMP', QMPExampleLexer()) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977071; cv=none; d=zoho.com; s=zohoarc; b=PixCVQoYQ6aEQuFzQ8d07P+qGI97E8/IitUvO5AZ4gk44J1EI3eUIC1PhdjhrPpw6zu+jGrKainleT3588sOVvXgVOclLKlW2eaQ2ugXSb3TntAImGV/hHTD3yPrsZxDELoanp5l2UiMYAdoDbcZBJXXZoSDNLAi3rcBgq0Hsk8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977071; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=R1z6ft6+MKpgeYIF9UQbmBJpfOzZXDYNjvynrbUlNCQ=; b=n64d3RHgY4OYAaWpMuGxP3J5z2Ws4khpIEUsWtFBiDxygoBESFcJ8HlL1EjzzzHphwoW9xPcHIsUaOqbOhMRSkOnsH0/Yf/R6sBA7YMBAKBnUIvaYj9MmL/OjDMjqmMKpAZtYOnW7kS6HLgXT5MRRGjeWzdDChDsxkluSaaNNLA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977071076679.4427008661127; Tue, 1 Oct 2019 17:44:31 -0700 (PDT) Received: from localhost ([::1]:49818 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSkn-0005uJ-Dx for importer@patchew.org; Tue, 01 Oct 2019 20:44:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39774) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zn-I0 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00024Q-DV for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:38002 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs7-0001TS-2O; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPAx179362; Tue, 1 Oct 2019 19:47:35 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnctr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:35 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlZpg179670; Tue, 1 Oct 2019 19:47:35 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnctf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:35 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlTB023573; Tue, 1 Oct 2019 23:47:34 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577gya-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:34 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlYtw5767716 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:34 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 52E142805E; Tue, 1 Oct 2019 23:47:34 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2DCDA28058; Tue, 1 Oct 2019 23:47:34 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:34 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 41/97] docs/bitmaps: use QMP lexer instead of json Date: Tue, 1 Oct 2019 18:45:20 -0500 Message-Id: <20191001234616.7825-42-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow The annotated style json we use in QMP documentation is not strict json and depending on the version of Sphinx (2.0+) or Pygments installed, might cause the build to fail. Use the new QMP lexer. Further, some versions of Sphinx can not apply custom lexers to "code" directives and require the use of "code-block" directives instead, so make that change at this time as well. Tested under: - Sphinx 1.3.6 and Pygments 2.4 - Sphinx 1.7.6 and Pygments 2.2 (Fedora 29 packages) - Sphinx 2.0.1 and Pygments 2.4 - Sphinx 3.0.0+/f396b3a783 and Pygments 2.4 (From Sphinx git c4f44bdd) Reported-by: Aarushi Mehta Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Message-id: 20190603214653.29369-4-jsnow@redhat.com Signed-off-by: John Snow (cherry picked from commit a7786bfb0effe0b4b0fc61d8a8cd307c0b739ed7) Signed-off-by: Michael Roth --- docs/interop/bitmaps.rst | 54 ++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst index c29ac4a854..c20bd37a79 100644 --- a/docs/interop/bitmaps.rst +++ b/docs/interop/bitmaps.rst @@ -199,7 +199,7 @@ persistence, and recording state can be adjusted at cre= ation time. =20 to create a new, actively recording persistent bitmap: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "block-dirty-bitmap-add", "arguments": { @@ -220,7 +220,7 @@ persistence, and recording state can be adjusted at cre= ation time. To create a new, disabled (``-recording``), transient bitmap that tracks changes in 32KiB segments: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "block-dirty-bitmap-add", "arguments": { @@ -254,7 +254,7 @@ Deletes a bitmap. Bitmaps that are ``+busy`` cannot be = removed. =20 Remove a bitmap named ``bitmap0`` from node ``drive0``: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "block-dirty-bitmap-remove", "arguments": { @@ -280,7 +280,7 @@ Clears all dirty bits from a bitmap. ``+busy`` bitmaps = cannot be cleared. =20 Clear all dirty bits from bitmap ``bitmap0`` on node ``drive0``: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "block-dirty-bitmap-clear", "arguments": { @@ -309,7 +309,7 @@ begin being recorded. ``+busy`` bitmaps cannot be enabl= ed. =20 To set ``+recording`` on bitmap ``bitmap0`` on node ``drive0``: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "block-dirty-bitmap-enable", "arguments": { @@ -347,7 +347,7 @@ writes to begin being ignored. ``+busy`` bitmaps cannot= be disabled. =20 To set ``-recording`` on bitmap ``bitmap0`` on node ``drive0``: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "block-dirty-bitmap-disable", "arguments": { @@ -393,7 +393,7 @@ in any one source bitmap, the target bitmap will mark t= hat segment dirty. ``drive0``. If ``new_bitmap`` was empty prior to this command, this achie= ves a copy. =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "block-dirty-bitmap-merge", "arguments": { @@ -424,7 +424,7 @@ attached to nodes serving as the root for guest devices. API. This result highlights a bitmap ``bitmap0`` attached to the root nod= e of device ``drive0``. =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "query-block", @@ -562,7 +562,7 @@ new, empty bitmap that records writes from this point i= n time forward. destination. These writes will be recorded in the bitmap accordingly. =20 -.. code:: json +.. code-block:: QMP =20 -> { "execute": "transaction", @@ -650,7 +650,7 @@ Example: Resetting an Incremental Backup Anchor Point If we want to start a new backup chain with an existing bitmap, we can also use a transaction to reset the bitmap while making a new full backup: =20 -.. code:: json +.. code-block:: QMP =20 -> { "execute": "transaction", @@ -730,7 +730,7 @@ Example: First Incremental Backup =20 #. Issue an incremental backup command: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "drive-backup", @@ -788,7 +788,7 @@ Example: Second Incremental Backup #. Issue a new incremental backup command. The only difference here is tha= t we have changed the target image below. =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "drive-backup", @@ -869,7 +869,7 @@ image: #. Issue a new incremental backup command. Apart from the new destination image, there is no difference from the last two examples. =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "drive-backup", @@ -932,7 +932,7 @@ point in time. =20 #. Create a full (anchor) backup for each drive, with accompanying bitmaps: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "transaction", @@ -1018,7 +1018,7 @@ point in time. =20 #. Issue a multi-drive incremental push backup transaction: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "transaction", @@ -1121,7 +1121,7 @@ described above. This example demonstrates the single= -job failure case: =20 #. Attempt to create an incremental backup via QMP: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "drive-backup", @@ -1139,7 +1139,7 @@ described above. This example demonstrates the single= -job failure case: =20 #. Receive a pair of events indicating failure: =20 - .. code:: json + .. code-block:: QMP =20 <- { "timestamp": {...}, @@ -1175,7 +1175,7 @@ described above. This example demonstrates the single= -job failure case: #. Retry the command after fixing the underlying problem, such as freeing up space on the backup volume: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "drive-backup", @@ -1193,7 +1193,7 @@ described above. This example demonstrates the single= -job failure case: =20 #. Receive confirmation that the job completed successfully: =20 - .. code:: json + .. code-block:: QMP =20 <- { "timestamp": {...}, @@ -1233,7 +1233,7 @@ and one succeeds: =20 #. Issue the transaction to start a backup of both drives. =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "transaction", @@ -1267,13 +1267,13 @@ and one succeeds: #. Receive notice that the Transaction was accepted, and jobs were launched: =20 - .. code:: json + .. code-block:: QMP =20 <- { "return": {} } =20 #. Receive notice that the first job has completed: =20 - .. code:: json + .. code-block:: QMP =20 <- { "timestamp": {...}, @@ -1289,7 +1289,7 @@ and one succeeds: =20 #. Receive notice that the second job has failed: =20 - .. code:: json + .. code-block:: QMP =20 <- { "timestamp": {...}, @@ -1365,7 +1365,7 @@ applied: =20 #. Issue the multi-drive incremental backup transaction: =20 - .. code:: json + .. code-block:: QMP =20 -> { "execute": "transaction", @@ -1401,13 +1401,13 @@ applied: =20 #. Receive notice that the Transaction was accepted, and jobs were launche= d: =20 - .. code:: json + .. code-block:: QMP =20 <- { "return": {} } =20 #. Receive notification that the backup job for ``drive1`` has failed: =20 - .. code:: json + .. code-block:: QMP =20 <- { "timestamp": {...}, @@ -1434,7 +1434,7 @@ applied: =20 #. Receive notification that the job for ``drive0`` has been cancelled: =20 - .. code:: json + .. code-block:: QMP =20 <- { "timestamp": {...}, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976574; cv=none; d=zoho.com; s=zohoarc; b=Hoc1AWpn89j0WOlYN+1Hap9raK8twFyF0fXMt8AvtYvwoDE3RA9JYs2vEJi3t6GMRJcvwqEty7c9ERE49AYjLkfcIHAZIJR/cMNeLNIOn8i1SR89omNvVqB64Jrn+oYoH9APeWF9twgQQYrFLSaK+hZFrrLblpH2cE6lidc1Cig= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976574; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=K+jkAO4a9XnF8RtKW9bJUg58GBj7DhDVTSfBEI4UA6M=; b=koIMViUNUFLnbyfv6Obapi/gYE1q0egMFtNOWkPRQwFyVl8zX5Ouy+hlvyPzT3RDMq3hCPk85DZc4X+A64USFRg3iKijSA7npo39f2s7tcjgNE9ryphmImTroH+DAYMqHRs5mr56l+TlyC9UXJFI2E1mJJXfSuH3bglo5tzKBQA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976574187946.8468350033423; Tue, 1 Oct 2019 17:36:14 -0700 (PDT) Received: from localhost ([::1]:49770 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSck-0007gu-CG for importer@patchew.org; Tue, 01 Oct 2019 20:36:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39814) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsC-0001aU-RU for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00024M-DA for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:04 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:32710) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs7-0001Wh-4P; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPcC088155; Tue, 1 Oct 2019 19:47:37 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bv9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:37 -0400 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlaS2088480; Tue, 1 Oct 2019 19:47:36 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bv2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:36 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlYn029380; Tue, 1 Oct 2019 23:47:35 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59faht-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:35 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlYro6488910 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:34 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D40E828059; Tue, 1 Oct 2019 23:47:34 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B719828058; Tue, 1 Oct 2019 23:47:34 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:34 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 42/97] hw/ssi/xilinx_spips: Convert lqspi_read() to read_with_attrs Date: Tue, 1 Oct 2019 18:45:21 -0500 Message-Id: <20191001234616.7825-43-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=813 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NlPcC088155 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Philippe Mathieu-Daud=C3=A9 In the next commit we will implement the write_with_attrs() handler. To avoid using different APIs, convert the read() handler first. Reviewed-by: Francisco Iglesias Tested-by: Francisco Iglesias Signed-off-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Peter Maydell (cherry picked from commit 5937bd50d3841b6ab2592c1ff4233448762a8483) Signed-off-by: Michael Roth --- hw/ssi/xilinx_spips.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/hw/ssi/xilinx_spips.c b/hw/ssi/xilinx_spips.c index 16f88f7402..9c5dd93b21 100644 --- a/hw/ssi/xilinx_spips.c +++ b/hw/ssi/xilinx_spips.c @@ -1199,27 +1199,26 @@ static void lqspi_load_cache(void *opaque, hwaddr a= ddr) } } =20 -static uint64_t -lqspi_read(void *opaque, hwaddr addr, unsigned int size) +static MemTxResult lqspi_read(void *opaque, hwaddr addr, uint64_t *value, + unsigned size, MemTxAttrs attrs) { - XilinxQSPIPS *q =3D opaque; - uint32_t ret; + XilinxQSPIPS *q =3D XILINX_QSPIPS(opaque); =20 if (addr >=3D q->lqspi_cached_addr && addr <=3D q->lqspi_cached_addr + LQSPI_CACHE_SIZE - 4) { uint8_t *retp =3D &q->lqspi_buf[addr - q->lqspi_cached_addr]; - ret =3D cpu_to_le32(*(uint32_t *)retp); - DB_PRINT_L(1, "addr: %08x, data: %08x\n", (unsigned)addr, - (unsigned)ret); - return ret; - } else { - lqspi_load_cache(opaque, addr); - return lqspi_read(opaque, addr, size); + *value =3D cpu_to_le32(*(uint32_t *)retp); + DB_PRINT_L(1, "addr: %08" HWADDR_PRIx ", data: %08" PRIx64 "\n", + addr, *value); + return MEMTX_OK; } + + lqspi_load_cache(opaque, addr); + return lqspi_read(opaque, addr, value, size, attrs); } =20 static const MemoryRegionOps lqspi_ops =3D { - .read =3D lqspi_read, + .read_with_attrs =3D lqspi_read, .endianness =3D DEVICE_NATIVE_ENDIAN, .valid =3D { .min_access_size =3D 1, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569975759; cv=none; d=zoho.com; s=zohoarc; b=PJo20/E9u682ULbRKgQ+9BFjQzDAKjdEQ8CubhN2oXi/3uC/vJwAnZd9gMGCh9KoYn/BNpCLRJgZjo9QjN9MpimFsEZf4LvHYn4fR9IoBBjdXGh/QZkLDSlMDBwjqHPkJ1ckIBcy9VBvIdGj+dQgfaSPOnqzavpqXDIWARGAMtY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569975759; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=zsIUAbnBuo2daNbifDZGjo1qjU1jwVDr90MWdzwYhR0=; b=Xotmra8yicnuWaBgWnGCzd2Ei0SLB1gX7Hfzqk63vHz2kgggWYBM7Xbup+zn/sBK2vACSSgxuzH49C86xLkpXIsJ7G4/TPGsL+c2hWai3iy+jhJpex13mEy66WbWcTO0M0ukPqxQpAt5zvZ3zPFGfPjr4gN92yPfLob48T2MDjw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569975759765333.8839675384015; Tue, 1 Oct 2019 17:22:39 -0700 (PDT) Received: from localhost ([::1]:49686 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSPc-0004ik-Su for importer@patchew.org; Tue, 01 Oct 2019 20:22:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39764) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zk-Fh for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00024G-FK for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:42558 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs6-0001Vk-OB; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlRrQ146346; Tue, 1 Oct 2019 19:47:36 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26dq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:36 -0400 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlamp146538; Tue, 1 Oct 2019 19:47:36 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26dk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:36 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njl1A006380; Tue, 1 Oct 2019 23:47:35 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye82-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:35 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlZgL46006592 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:35 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5FE8A2805A; Tue, 1 Oct 2019 23:47:35 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 43AC128058; Tue, 1 Oct 2019 23:47:35 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:35 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 43/97] hw/ssi/xilinx_spips: Avoid AXI writes to the LQSPI linear memory Date: Tue, 1 Oct 2019 18:45:22 -0500 Message-Id: <20191001234616.7825-44-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0b-001b2d01.pphosted.com id x91NlRrQ146346 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Philippe Mathieu-Daud=C3=A9 Lei Sun found while auditing the code that a CPU write would trigger a NULL pointer dereference. >From UG1085 datasheet [*] AXI writes in this region are ignored and generates an AXI Slave Error (SLVERR). Fix by implementing the write_with_attrs() handler. Return MEMTX_ERROR when the region is accessed (this error maps to an AXI slave error). [*] https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ul= trascale-trm.pdf Reported-by: Lei Sun Reviewed-by: Francisco Iglesias Tested-by: Francisco Iglesias Signed-off-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Peter Maydell (cherry picked from commit 936a236c4e4b1068ade99220260cd04f68eb0212) Signed-off-by: Michael Roth --- hw/ssi/xilinx_spips.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/hw/ssi/xilinx_spips.c b/hw/ssi/xilinx_spips.c index 9c5dd93b21..83ed5ab1e0 100644 --- a/hw/ssi/xilinx_spips.c +++ b/hw/ssi/xilinx_spips.c @@ -1217,8 +1217,24 @@ static MemTxResult lqspi_read(void *opaque, hwaddr a= ddr, uint64_t *value, return lqspi_read(opaque, addr, value, size, attrs); } =20 +static MemTxResult lqspi_write(void *opaque, hwaddr offset, uint64_t value, + unsigned size, MemTxAttrs attrs) +{ + /* + * From UG1085, Chapter 24 (Quad-SPI controllers): + * - Writes are ignored + * - AXI writes generate an external AXI slave error (SLVERR) + */ + qemu_log_mask(LOG_GUEST_ERROR, "%s Unexpected %u-bit access to 0x%" PR= Ix64 + " (value: 0x%" PRIx64 "\n", + __func__, size << 3, offset, value); + + return MEMTX_ERROR; +} + static const MemoryRegionOps lqspi_ops =3D { .read_with_attrs =3D lqspi_read, + .write_with_attrs =3D lqspi_write, .endianness =3D DEVICE_NATIVE_ENDIAN, .valid =3D { .min_access_size =3D 1, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976088; cv=none; d=zoho.com; s=zohoarc; b=ojBHjjq/B2Z1n9VuWARqZ3+KGHbWn3QuypeJimJvs0N/AmVJNYxjW3OB/B8EFzKRB1B8xieJw/+A8gk40S59nLX4chFVPqll36gOGAhg8HpgKEj29PsdL/ItADh5scIuKgbRCEvqVlEdOnX58J/z7+X7PophKArPLhxYP/5QOS8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976088; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=cKfXoRLGV99sIXxEt30/AQksjs3JtlSL6vDl63tHgCg=; b=EC+rWJOwD2umYFDo03aNcQ1vY0LvvI0uod3AWo0Amog5e6NKqbCv2FT4KSyplTqlx+qM0oLGFkhfS1VNdDqZOqr1YVHHfUPaZOu2CMsQmeC6FpIUDG7J5ynp+kYMf2vG89MTB3eJLT54bGK5dHzAaVQESmYRNpZHsawUJK6XfKI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976088913872.5269966515785; Tue, 1 Oct 2019 17:28:08 -0700 (PDT) Received: from localhost ([::1]:49718 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSUw-0000G6-Qh for importer@patchew.org; Tue, 01 Oct 2019 20:28:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39759) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zi-E9 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00023n-A0 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:32242) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs7-0001Xu-KL; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPc9053476; Tue, 1 Oct 2019 19:47:38 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg171669-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:37 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlbic053848; Tue, 1 Oct 2019 19:47:37 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17165s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:37 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkYF031624; Tue, 1 Oct 2019 23:47:36 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fbn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:36 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlZJ046006594 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:35 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC2A52805A; Tue, 1 Oct 2019 23:47:35 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BFD9928058; Tue, 1 Oct 2019 23:47:35 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:35 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 44/97] hw/ssi/xilinx_spips: Avoid out-of-bound access to lqspi_buf[] Date: Tue, 1 Oct 2019 18:45:23 -0500 Message-Id: <20191001234616.7825-45-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=991 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NlPc9053476 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Philippe Mathieu-Daud=C3=A9 Both lqspi_read() and lqspi_load_cache() expect a 32-bit aligned address. >From UG1085 datasheet [*] chapter on 'Quad-SPI Controller': Transfer Size Limitations Because of the 32-bit wide TX, RX, and generic FIFO, all APB/AXI transfers must be an integer multiple of 4-bytes. Shorter transfers are not possible. Set MemoryRegionOps.impl values to force 32-bit accesses, this way we are sure we do not access the lqspi_buf[] array out of bound. [*] https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ul= trascale-trm.pdf Reviewed-by: Francisco Iglesias Tested-by: Francisco Iglesias Signed-off-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Peter Maydell (cherry picked from commit 526668c734e6a07f2fedfd378840a61b70c1cbab) Signed-off-by: Michael Roth --- hw/ssi/xilinx_spips.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/ssi/xilinx_spips.c b/hw/ssi/xilinx_spips.c index 83ed5ab1e0..b649c464fb 100644 --- a/hw/ssi/xilinx_spips.c +++ b/hw/ssi/xilinx_spips.c @@ -1236,6 +1236,10 @@ static const MemoryRegionOps lqspi_ops =3D { .read_with_attrs =3D lqspi_read, .write_with_attrs =3D lqspi_write, .endianness =3D DEVICE_NATIVE_ENDIAN, + .impl =3D { + .min_access_size =3D 4, + .max_access_size =3D 4, + }, .valid =3D { .min_access_size =3D 1, .max_access_size =3D 4 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976739; cv=none; d=zoho.com; s=zohoarc; b=eBT/lRvRUNvdRXpI1Tlt/H9sTkQvD5yrW+iWLjx+K/n6UJrUjuiE0LUrp6JYRbrgUsI1g585DrugjQsOVZmKQYnh1vufnboVV+8F6H58iTW29883kA8nf86eJrLQ9+YaQIQlGRByFXEVJFthoLtaSbWUk/3GVfBK01Z+XAKNUg0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976739; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=cnlFF4AYz3zI/o5DtdujS9hcK6Tk062jy2UrZ3rYTz0=; b=iPn4kbQNkC+GB/lmuBx7I1Rr8ha+E6MPDU75lDNCMt6c+UBrnSffQyMi6fiF0Pzpnn/+LrUNsOt5USPhscRLqwiJQo0gmZeI/836uYpGLNjevB+vPS9/S6Mnfg3G2qMwtRiGVCnKTDQAkhmhKIen0zupXWFfefsBi2wNfKxqM8k= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997673985863.704809578294544; Tue, 1 Oct 2019 17:38:59 -0700 (PDT) Received: from localhost ([::1]:49788 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSfS-0001Zm-CK for importer@patchew.org; Tue, 01 Oct 2019 20:38:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39841) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsD-0001bR-6G for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00024Y-Cc for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:04 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:54290 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs7-0001Xn-7D; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91Nlakk079234; Tue, 1 Oct 2019 19:47:38 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00vq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:38 -0400 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlc1R079267; Tue, 1 Oct 2019 19:47:38 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f00vc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njlt7032690; Tue, 1 Oct 2019 23:47:37 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma02dal.us.ibm.com with ESMTP id 2v9y57qb7k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:37 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlabT46662136 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:36 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 58A662805C; Tue, 1 Oct 2019 23:47:36 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4763928060; Tue, 1 Oct 2019 23:47:36 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:36 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 45/97] ioapic: kvm: Skip route updates for masked pins Date: Tue, 1 Oct 2019 18:45:24 -0500 Message-Id: <20191001234616.7825-46-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=915 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jan Kiszka , qemu-stable@nongnu.org, "Michael S . Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Jan Kiszka Masked entries will not generate interrupt messages, thus do no need to be routed by KVM. This is a cosmetic cleanup, just avoiding warnings of the kind qemu-system-x86_64: vtd_irte_get: detected non-present IRTE (index=3D0, hig= h=3D0xff00, low=3D0x100) if the masked entry happens to reference a non-present IRTE. Cc: qemu-stable@nongnu.org Signed-off-by: Jan Kiszka Message-Id: Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin Reviewed-by: Peter Xu (cherry picked from commit be1927c97e564346cbd409cb17fe611df74b84e5) Signed-off-by: Michael Roth --- hw/intc/ioapic.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c index 9d75f84d3b..194ccb6a3e 100644 --- a/hw/intc/ioapic.c +++ b/hw/intc/ioapic.c @@ -188,9 +188,11 @@ static void ioapic_update_kvm_routes(IOAPICCommonState= *s) MSIMessage msg; struct ioapic_entry_info info; ioapic_entry_parse(s->ioredtbl[i], &info); - msg.address =3D info.addr; - msg.data =3D info.data; - kvm_irqchip_update_msi_route(kvm_state, i, msg, NULL); + if (!info.masked) { + msg.address =3D info.addr; + msg.data =3D info.data; + kvm_irqchip_update_msi_route(kvm_state, i, msg, NULL); + } } kvm_irqchip_commit_routes(kvm_state); } --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976051; cv=none; d=zoho.com; s=zohoarc; b=eXTniEwhdoapmFDxFq06GJX0CUCeKEWjwRfBb5BrUO7oPSMdncNJkESJVD8Ld+oaoYcQFiaXtolePNEEO5CoRmh25GOdhQDklxPUc/PrG/lQJHA2a6JkbN+zlRgqym0ulnWpceUjeibh2eilH9qM7ThZwbdw8Wk6rmQUHMn2ehU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976051; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=HRQtByMWdwk2VEw+gBn/oADwEd0DBMNlRIYyq8YOjvY=; b=jSG/pFu5/GCv9tevJIO3foJ7UB5dn+x9XA4HbXSx7SPQjN3aAbWjEZDC9357Q2Cq6KPWRp/iGa4z03FoL6meaHBjxAq5qTsmFaQvxvcZleqaMIBmAzs6CsLWocVlpr36Tecpjq0MOyqhqh8YvaHMZv/ERYWRh+6kAk9KOYBPQNE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976051252272.9238387132199; Tue, 1 Oct 2019 17:27:31 -0700 (PDT) Received: from localhost ([::1]:49710 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSUM-0008ES-1J for importer@patchew.org; Tue, 01 Oct 2019 20:27:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39755) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsB-0001Zd-C6 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00023r-Eb for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:03 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:5808 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs7-0001ZE-La; Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlO8v146269; Tue, 1 Oct 2019 19:47:39 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26en-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:38 -0400 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlcD3146606; Tue, 1 Oct 2019 19:47:38 -0400 Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26eb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:38 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlTE022434; Tue, 1 Oct 2019 23:47:37 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma04dal.us.ibm.com with ESMTP id 2v9y57yd6q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:37 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlaVZ51052934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:37 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DE3002805C; Tue, 1 Oct 2019 23:47:36 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C295D28058; Tue, 1 Oct 2019 23:47:36 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:36 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 46/97] i386/acpi: show PCI Express bus on pxb-pcie expanders Date: Tue, 1 Oct 2019 18:45:25 -0500 Message-Id: <20191001234616.7825-47-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=13 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=990 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Evgeny Yakovlev , "Michael S . Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Evgeny Yakovlev Show PCIe host bridge PNP id with PCI host bridge as a compatible id when expanding a pcie bus. Cc: qemu-stable@nongnu.org Signed-off-by: Evgeny Yakovlev Message-Id: <1563526469-15588-1-git-send-email-wrfsh@yandex-team.ru> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit ee4b0c8686f781987879508d7c6dd605b5435bac) Signed-off-by: Michael Roth --- hw/i386/acpi-build.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index 416da318ae..ede27ab3c4 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -1895,10 +1895,13 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, scope =3D aml_scope("\\_SB"); dev =3D aml_device("PC%.02X", bus_num); aml_append(dev, aml_name_decl("_UID", aml_int(bus_num))); - aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A03"))); aml_append(dev, aml_name_decl("_BBN", aml_int(bus_num))); if (pci_bus_is_express(bus)) { + aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08"= ))); + aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03"= ))); aml_append(dev, build_q35_osc_method()); + } else { + aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A03"= ))); } =20 if (numa_node !=3D NUMA_NODE_UNASSIGNED) { --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976558; cv=none; d=zoho.com; s=zohoarc; b=bj1Q/AHVKADBo22DEFhQlxEX7guAoRcdmm2raukngizBz0dTOltEsUz2BjYeWPFJ69NGMMeuLb9tf/YvOH5cxrhb8vgWSt9zFsY8/S4zjhLL4BwJY/k4bkDUETwCpY4DXL04oUMk+xKPaCvXvhkG5Zb+WKJUWuxietzktZDOPyI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976558; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=+OFUa34bSfKgZaNIJQ1ZRPQRDx9dttYhRYlOEFK4FMM=; b=cC4ohTaksvulBhV1qV1ZMe89sUsRuWhmucU9NtjCOmEYPoRhc1latadqQcrCMY0KJ4gUcHxiK/ndPz5tc1qEtTIelIhTOo2rVK/qTwjhBhIUkryop7iqxFDnpvdwamFF1lSaRiPEZ0LhXaGtqnIa/qE7ALpTWumqyHS+/Ycgxz4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976558173318.0073303181234; Tue, 1 Oct 2019 17:35:58 -0700 (PDT) Received: from localhost ([::1]:49768 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFScW-0007N4-1e for importer@patchew.org; Tue, 01 Oct 2019 20:35:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39883) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsE-0001cw-34 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-000250-Iu for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:05 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:44758) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs9-0001b9-7d; Tue, 01 Oct 2019 19:48:01 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPW4088125; Tue, 1 Oct 2019 19:47:40 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bw7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:39 -0400 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlZPj088456; Tue, 1 Oct 2019 19:47:39 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bvt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:39 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njl1C006380; Tue, 1 Oct 2019 23:47:38 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye88-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:37 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlbm350725262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:37 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 661182805C; Tue, 1 Oct 2019 23:47:37 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4B3A128065; Tue, 1 Oct 2019 23:47:37 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:37 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 47/97] virtio-balloon: Fix wrong sign extension of PFNs Date: Tue, 1 Oct 2019 18:45:26 -0500 Message-Id: <20191001234616.7825-48-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , qemu-stable@nongnu.org, David Hildenbrand Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand If we directly cast from int to uint64_t, we will first sign-extend to an int64_t, which is wrong. We actually want to treat the PFNs like unsigned values. As far as I can see, this dates back to the initial virtio-balloon commit, but wasn't triggered as fairly big guests would be required. Cc: qemu-stable@nongnu.org Reported-by: Michael S. Tsirkin Signed-off-by: David Hildenbrand Message-Id: <20190722134108.22151-2-david@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin Reviewed-by: David Gibson (cherry picked from commit ffa207d08253ffffb3993a1dbe09e40af4fc91f1) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 5579260fd4..49194f5638 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -343,8 +343,8 @@ static void virtio_balloon_handle_output(VirtIODevice *= vdev, VirtQueue *vq) } =20 while (iov_to_buf(elem->out_sg, elem->out_num, offset, &pfn, 4) = =3D=3D 4) { + unsigned int p =3D virtio_ldl_p(vdev, &pfn); hwaddr pa; - int p =3D virtio_ldl_p(vdev, &pfn); =20 pa =3D (hwaddr) p << VIRTIO_BALLOON_PFN_SHIFT; offset +=3D 4; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976939; cv=none; d=zoho.com; s=zohoarc; b=UGeuvdUIAMj+r+HeHaYL21SRMy2Tk87HTZI1e4JfbSrOILPUwLMjqbhtUJgOHzpgC6PwxORuwUakl/4qbh62r1p/Wvx+UsCl8silOL9N7N03icuEWiy3OdNvkzcrPPdT0GPeZm/VxMIbR3UEdDtS7IpODiR7mYKGOSVSgn5SFDM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976939; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=DM4x9HX5O2VlKDo2w0Qxs2WzgVGaAtc6oS6CyxxN5Lo=; b=iudkRW1gXj1Olnwfm74iWh9bHizav76ETF5Hqul501w1lKD1fD/BdIuH2tUy3/d1Zf1/W4G0bwViVfUfBYbO6nZHCBcU7tfg6MwM2YmOq30uQAZrNppovF7Nx/4AY6clPWJ9T8qzZx3x/+uCDXR43bVuHDEI/vKY7CPHXBLf98U= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976939396120.19403645867828; Tue, 1 Oct 2019 17:42:19 -0700 (PDT) Received: from localhost ([::1]:49802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSic-0003bW-IM for importer@patchew.org; Tue, 01 Oct 2019 20:42:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39832) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsD-0001b8-0e for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRs9-00024l-FN for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:04 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:41130) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs9-0001b2-0E; Tue, 01 Oct 2019 19:48:01 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOLa053207; Tue, 1 Oct 2019 19:47:40 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg171676-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:39 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NldHN053903; Tue, 1 Oct 2019 19:47:39 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17166r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:39 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlP8031629; Tue, 1 Oct 2019 23:47:38 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fbu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:38 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlc1l43188636 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:38 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E2A9128058; Tue, 1 Oct 2019 23:47:37 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C777E28059; Tue, 1 Oct 2019 23:47:37 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:37 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 48/97] virtio-balloon: Fix QEMU crashes on pagesize > BALLOON_PAGE_SIZE Date: Tue, 1 Oct 2019 18:45:27 -0500 Message-Id: <20191001234616.7825-49-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , qemu-stable@nongnu.org, David Hildenbrand Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand We are using the wrong functions to set/clear bits, effectively touching multiple bits, writing out of range of the bitmap, resulting in memory corruptions. We have to use set_bit()/clear_bit() instead. Can easily be reproduced by starting a qemu guest on hugetlbfs memory, inflating the balloon. QEMU crashes. This never could have worked properly - especially, also pages would have been discarded when the first sub-page would be inflated (the whole bitmap would be set). While testing I realized, that on hugetlbfs it is pretty much impossible to discard a page - the guest just frees the 4k sub-pages in random order most of the time. I was only able to discard a hugepage a handful of times - so I hope that now works correctly. Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < hos= t page size") Fixes: b27b32391404 ("virtio-balloon: Fix possible guest memory corruption = with inflates & deflates") Cc: qemu-stable@nongnu.org #v4.0.0 Acked-by: David Gibson Signed-off-by: David Hildenbrand Message-Id: <20190722134108.22151-3-david@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit 483f13524bb2a08b7ff6a7560b846564ed3b0c33) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 49194f5638..038cf1c23a 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -94,9 +94,8 @@ static void balloon_inflate_page(VirtIOBalloon *balloon, balloon->pbp->base =3D host_page_base; } =20 - bitmap_set(balloon->pbp->bitmap, - (ram_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, - subpages); + set_bit((ram_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, + balloon->pbp->bitmap); =20 if (bitmap_full(balloon->pbp->bitmap, subpages)) { /* We've accumulated a full host page, we can actually discard @@ -140,9 +139,8 @@ static void balloon_deflate_page(VirtIOBalloon *balloon, * for a guest to do this in practice, but handle it anyway, * since getting it wrong could mean discarding memory the * guest is still using. */ - bitmap_clear(balloon->pbp->bitmap, - (ram_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, - subpages); + clear_bit((ram_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, + balloon->pbp->bitmap); =20 if (bitmap_empty(balloon->pbp->bitmap, subpages)) { g_free(balloon->pbp); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976025; cv=none; d=zoho.com; s=zohoarc; b=IgnNySde30Gl+VETa+COfp1wk787CczUGV2iad38alzVOq4oThibIXi5GKeGQmkQ7+/j33MkD5nAW5bKeABJA8FhWxO8ky5pbAmpD2qQOKKPls5M5sD5ZvEYlUX7CDkAYtMasY7z3/TDLzpSJ6ZetMqc/mR0CQK1k1smYJc5yj0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976025; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=r2YWzdd5OzdqtzMZY1+Xw1it2s2ZUeoJbtmWt6JTHoc=; b=ggaBmew9PmVIvhFiVniyV5e0B5beCepG5CoPYK5vrmzumjIU3Dh0fYsQZRj+hAIM69Oe1dR18PkYCC0rJ90SNsTaJw8AORasxpG8Z2rECenIbwIZ8Dwl77oapxngp/OZ2rKBQwWjgzuPr8s71vgisaG8Vl6/EWbHvB7GekHxC1s= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976025473587.4223152884102; Tue, 1 Oct 2019 17:27:05 -0700 (PDT) Received: from localhost ([::1]:49708 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSTv-0007pl-67 for importer@patchew.org; Tue, 01 Oct 2019 20:27:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39822) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsC-0001aq-Ss for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsA-00025J-7s for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:04 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:9544 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRs9-0001aX-Ma; Tue, 01 Oct 2019 19:48:02 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPpq146298; Tue, 1 Oct 2019 19:47:39 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26ey-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:39 -0400 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nldlf146627; Tue, 1 Oct 2019 19:47:39 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26eq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:39 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk0G006371; Tue, 1 Oct 2019 23:47:38 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye8b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:38 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlcW051052938 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:38 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 69C7028058; Tue, 1 Oct 2019 23:47:38 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4DC2A28059; Tue, 1 Oct 2019 23:47:38 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:38 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 49/97] virtio-balloon: Simplify deflate with pbp Date: Tue, 1 Oct 2019 18:45:28 -0500 Message-Id: <20191001234616.7825-50-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , qemu-stable@nongnu.org, David Hildenbrand Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand Let's simplify this - the case we are optimizing for is very hard to trigger and not worth the effort. If we're switching from inflation to deflation, let's reset the pbp. Acked-by: David Gibson Signed-off-by: David Hildenbrand Message-Id: <20190722134108.22151-4-david@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit 2ffc49eea1bbd454913a88a0ad872c2649b36950) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 26 +++++--------------------- 1 file changed, 5 insertions(+), 21 deletions(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 038cf1c23a..4f42d3f2b0 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -117,7 +117,7 @@ static void balloon_deflate_page(VirtIOBalloon *balloon, void *addr =3D memory_region_get_ram_ptr(mr) + offset; RAMBlock *rb; size_t rb_page_size; - ram_addr_t ram_offset, host_page_base; + ram_addr_t ram_offset; void *host_addr; int ret; =20 @@ -125,27 +125,11 @@ static void balloon_deflate_page(VirtIOBalloon *ballo= on, * host address? */ rb =3D qemu_ram_block_from_host(addr, false, &ram_offset); rb_page_size =3D qemu_ram_pagesize(rb); - host_page_base =3D ram_offset & ~(rb_page_size - 1); - - if (balloon->pbp - && rb =3D=3D balloon->pbp->rb - && host_page_base =3D=3D balloon->pbp->base) { - int subpages =3D rb_page_size / BALLOON_PAGE_SIZE; =20 - /* - * This means the guest has asked to discard some of the 4kiB - * subpages of a host page, but then changed its mind and - * asked to keep them after all. It's exceedingly unlikely - * for a guest to do this in practice, but handle it anyway, - * since getting it wrong could mean discarding memory the - * guest is still using. */ - clear_bit((ram_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, - balloon->pbp->bitmap); - - if (bitmap_empty(balloon->pbp->bitmap, subpages)) { - g_free(balloon->pbp); - balloon->pbp =3D NULL; - } + if (balloon->pbp) { + /* Let's play safe and always reset the pbp on deflation requests.= */ + g_free(balloon->pbp); + balloon->pbp =3D NULL; } =20 host_addr =3D (void *)((uintptr_t)addr & ~(rb_page_size - 1)); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977297; cv=none; d=zoho.com; s=zohoarc; b=DVD+0bTMdZ2EciylzdLJJnx14wvFmDmtqY90sZzawiDewfTjJ1UDie+iAOCc6dcI4PskfOgbuw4oMfoYXC0syXi472+ubF70P269LxL2ix9JlT3cM++OJQzFxBY2LpNU+90Hb9RlFm13IukLvSOLphnQP8cX77lHTUEHUheoNJo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977297; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=L38CF15wWxxDDk3F8RbslJyBf4fFHqjCQfp4Ys14e+4=; b=lXCLJQffgYlJE804s00TJ71Yr+BRPA9gnoDf9hRvSeSnWBZl38o3Hgs0JZ5FOXj+aRzhAHCr5wwURML4fSQH1rNi5Kuh1TX0lmcbb92FcLncP1LjQwdI73nzrs+XeIj5Fsp2oGqxmDUAAJDZdQJp0w7U5agqKMaX/YqloB+gomI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977297170662.362053124109; Tue, 1 Oct 2019 17:48:17 -0700 (PDT) Received: from localhost ([::1]:49848 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSoR-0001dJ-Vq for importer@patchew.org; Tue, 01 Oct 2019 20:48:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39904) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsE-0001df-Mb for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsB-000274-FL for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:06 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:3240) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsA-0001dh-S6; Tue, 01 Oct 2019 19:48:03 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPJA088120; Tue, 1 Oct 2019 19:47:41 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bxe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:41 -0400 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlWuq088382; Tue, 1 Oct 2019 19:47:41 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6bwh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:41 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjmlK023578; Tue, 1 Oct 2019 23:47:39 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577gyj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:39 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nldda40304954 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:39 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E70A928059; Tue, 1 Oct 2019 23:47:38 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CAA692805A; Tue, 1 Oct 2019 23:47:38 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:38 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 50/97] virtio-balloon: Better names for offset variables in inflate/deflate code Date: Tue, 1 Oct 2019 18:45:29 -0500 Message-Id: <20191001234616.7825-51-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , qemu-stable@nongnu.org, David Hildenbrand Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand "host_page_base" is really confusing, let's make this clearer, also rename the other offsets to indicate to which base they apply. offset -> mr_offset ram_offset -> rb_offset host_page_base -> rb_aligned_offset While at it, use QEMU_ALIGN_DOWN() instead of a handcrafted computation and move the computation to the place where it is needed. Acked-by: David Gibson Signed-off-by: David Hildenbrand Message-Id: <20190722134108.22151-5-david@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit e6129b271b9dccca22c84870e313c315f2c70063) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 4f42d3f2b0..76b4c58206 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -41,24 +41,23 @@ struct PartiallyBalloonedPage { }; =20 static void balloon_inflate_page(VirtIOBalloon *balloon, - MemoryRegion *mr, hwaddr offset) + MemoryRegion *mr, hwaddr mr_offset) { - void *addr =3D memory_region_get_ram_ptr(mr) + offset; + void *addr =3D memory_region_get_ram_ptr(mr) + mr_offset; + ram_addr_t rb_offset, rb_aligned_offset; RAMBlock *rb; size_t rb_page_size; int subpages; - ram_addr_t ram_offset, host_page_base; =20 /* XXX is there a better way to get to the RAMBlock than via a * host address? */ - rb =3D qemu_ram_block_from_host(addr, false, &ram_offset); + rb =3D qemu_ram_block_from_host(addr, false, &rb_offset); rb_page_size =3D qemu_ram_pagesize(rb); - host_page_base =3D ram_offset & ~(rb_page_size - 1); =20 if (rb_page_size =3D=3D BALLOON_PAGE_SIZE) { /* Easy case */ =20 - ram_block_discard_range(rb, ram_offset, rb_page_size); + ram_block_discard_range(rb, rb_offset, rb_page_size); /* We ignore errors from ram_block_discard_range(), because it * has already reported them, and failing to discard a balloon * page is not fatal */ @@ -74,11 +73,12 @@ static void balloon_inflate_page(VirtIOBalloon *balloon, warn_report_once( "Balloon used with backing page size > 4kiB, this may not be reliable"); =20 + rb_aligned_offset =3D QEMU_ALIGN_DOWN(rb_offset, rb_page_size); subpages =3D rb_page_size / BALLOON_PAGE_SIZE; =20 if (balloon->pbp && (rb !=3D balloon->pbp->rb - || host_page_base !=3D balloon->pbp->base)) { + || rb_aligned_offset !=3D balloon->pbp->base)) { /* We've partially ballooned part of a host page, but now * we're trying to balloon part of a different one. Too hard, * give up on the old partial page */ @@ -91,10 +91,10 @@ static void balloon_inflate_page(VirtIOBalloon *balloon, size_t bitlen =3D BITS_TO_LONGS(subpages) * sizeof(unsigned long); balloon->pbp =3D g_malloc0(sizeof(PartiallyBalloonedPage) + bitlen= ); balloon->pbp->rb =3D rb; - balloon->pbp->base =3D host_page_base; + balloon->pbp->base =3D rb_aligned_offset; } =20 - set_bit((ram_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, + set_bit((rb_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, balloon->pbp->bitmap); =20 if (bitmap_full(balloon->pbp->bitmap, subpages)) { @@ -112,18 +112,18 @@ static void balloon_inflate_page(VirtIOBalloon *ballo= on, } =20 static void balloon_deflate_page(VirtIOBalloon *balloon, - MemoryRegion *mr, hwaddr offset) + MemoryRegion *mr, hwaddr mr_offset) { - void *addr =3D memory_region_get_ram_ptr(mr) + offset; + void *addr =3D memory_region_get_ram_ptr(mr) + mr_offset; + ram_addr_t rb_offset; RAMBlock *rb; size_t rb_page_size; - ram_addr_t ram_offset; void *host_addr; int ret; =20 /* XXX is there a better way to get to the RAMBlock than via a * host address? */ - rb =3D qemu_ram_block_from_host(addr, false, &ram_offset); + rb =3D qemu_ram_block_from_host(addr, false, &rb_offset); rb_page_size =3D qemu_ram_pagesize(rb); =20 if (balloon->pbp) { --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977541; cv=none; d=zoho.com; s=zohoarc; b=Y7GUzdSpMz9ZvTjCSVaimEHMmPP5UZ+Bmq+QZDLv8e84TEujVJdd2sj89TDtCAS1plTcM6Cm94Qoeh2scs1Yf7nRMcz8S4+0dm+f0u47LiL4DP2POrbe/5i4UF4L+2fkukXn6jIVX/B+Ys9NLZ9c7TSdeK7PjrY6N1BEd4WBgw4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977541; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=LYsaG0MXY6VgBC8aV4GHyxhhepEM7qgCSV4v28h/r+Y=; b=f9PpuLJfRLr+M6DCfegR65V+wuvc2yAZ9Uxjxrh861vwkLVfgjbbTrkD6DsEx+DvfdCESaHSRH+E1MAUVDmIwQdAGUSwUb6Lq/FxA5c0tCJ+L9d1BSnA7LsnFLdbwB+tpxm+ijHwK6aOUaeuT3wRcf+o5geyhvjnE5MUM+S8+Fc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15699775411291012.3564583981488; Tue, 1 Oct 2019 17:52:21 -0700 (PDT) Received: from localhost ([::1]:49886 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSsL-0005SL-NW for importer@patchew.org; Tue, 01 Oct 2019 20:52:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39905) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsE-0001dj-N4 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsB-00027A-Hc for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:06 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:46892) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsA-0001e9-Uc; Tue, 01 Oct 2019 19:48:03 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPhN074797; Tue, 1 Oct 2019 19:47:42 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpy2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:41 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlfl1075660; Tue, 1 Oct 2019 19:47:41 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vpxf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:41 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njod0031648; Tue, 1 Oct 2019 23:47:40 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fc0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:40 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nldob53477664 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:40 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E063B28059; Tue, 1 Oct 2019 23:47:39 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C5BA52805A; Tue, 1 Oct 2019 23:47:39 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:39 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 51/97] virtio-balloon: Rework pbp tracking data Date: Tue, 1 Oct 2019 18:45:30 -0500 Message-Id: <20191001234616.7825-52-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , qemu-stable@nongnu.org, David Hildenbrand Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand Using the address of a RAMBlock to test for a matching pbp is not really safe. Instead, let's use the guest physical address of the base page along with the page size (via the number of subpages). Also, let's allocate the bitmap separately. This makes the code easier to read and maintain - we can reuse bitmap_new(). Prepare the code to move the PBP out of the device. Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < hos= t page size") Fixes: b27b32391404 ("virtio-balloon: Fix possible guest memory corruption = with inflates & deflates") Cc: qemu-stable@nongnu.org #v4.0.0 Signed-off-by: David Hildenbrand Message-Id: <20190722134108.22151-6-david@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit 1c5cfc2b7153dd72bf4b8ddc456408eb2b9b66d8) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 69 +++++++++++++++++++++++++------------- 1 file changed, 46 insertions(+), 23 deletions(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 76b4c58206..49999d0bbe 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -35,16 +35,44 @@ #define BALLOON_PAGE_SIZE (1 << VIRTIO_BALLOON_PFN_SHIFT) =20 struct PartiallyBalloonedPage { - RAMBlock *rb; - ram_addr_t base; - unsigned long bitmap[]; + ram_addr_t base_gpa; + long subpages; + unsigned long *bitmap; }; =20 +static void virtio_balloon_pbp_free(PartiallyBalloonedPage *pbp) +{ + if (!pbp) { + return; + } + g_free(pbp->bitmap); + g_free(pbp); +} + +static PartiallyBalloonedPage *virtio_balloon_pbp_alloc(ram_addr_t base_gp= a, + long subpages) +{ + PartiallyBalloonedPage *pbp =3D g_new0(PartiallyBalloonedPage, 1); + + pbp->base_gpa =3D base_gpa; + pbp->subpages =3D subpages; + pbp->bitmap =3D bitmap_new(subpages); + + return pbp; +} + +static bool virtio_balloon_pbp_matches(PartiallyBalloonedPage *pbp, + ram_addr_t base_gpa, long subpages) +{ + return pbp->subpages =3D=3D subpages && pbp->base_gpa =3D=3D base_gpa; +} + static void balloon_inflate_page(VirtIOBalloon *balloon, MemoryRegion *mr, hwaddr mr_offset) { void *addr =3D memory_region_get_ram_ptr(mr) + mr_offset; - ram_addr_t rb_offset, rb_aligned_offset; + ram_addr_t rb_offset, rb_aligned_offset, base_gpa; + PartiallyBalloonedPage **pbp =3D &balloon->pbp; RAMBlock *rb; size_t rb_page_size; int subpages; @@ -75,39 +103,34 @@ static void balloon_inflate_page(VirtIOBalloon *balloo= n, =20 rb_aligned_offset =3D QEMU_ALIGN_DOWN(rb_offset, rb_page_size); subpages =3D rb_page_size / BALLOON_PAGE_SIZE; + base_gpa =3D memory_region_get_ram_addr(mr) + mr_offset - + (rb_offset - rb_aligned_offset); =20 - if (balloon->pbp - && (rb !=3D balloon->pbp->rb - || rb_aligned_offset !=3D balloon->pbp->base)) { + if (*pbp && !virtio_balloon_pbp_matches(*pbp, base_gpa, subpages)) { /* We've partially ballooned part of a host page, but now * we're trying to balloon part of a different one. Too hard, * give up on the old partial page */ - g_free(balloon->pbp); - balloon->pbp =3D NULL; + virtio_balloon_pbp_free(*pbp); + *pbp =3D NULL; } =20 - if (!balloon->pbp) { - /* Starting on a new host page */ - size_t bitlen =3D BITS_TO_LONGS(subpages) * sizeof(unsigned long); - balloon->pbp =3D g_malloc0(sizeof(PartiallyBalloonedPage) + bitlen= ); - balloon->pbp->rb =3D rb; - balloon->pbp->base =3D rb_aligned_offset; + if (!*pbp) { + *pbp =3D virtio_balloon_pbp_alloc(base_gpa, subpages); } =20 - set_bit((rb_offset - balloon->pbp->base) / BALLOON_PAGE_SIZE, - balloon->pbp->bitmap); + set_bit((rb_offset - rb_aligned_offset) / BALLOON_PAGE_SIZE, + (*pbp)->bitmap); =20 - if (bitmap_full(balloon->pbp->bitmap, subpages)) { + if (bitmap_full((*pbp)->bitmap, subpages)) { /* We've accumulated a full host page, we can actually discard * it now */ =20 - ram_block_discard_range(rb, balloon->pbp->base, rb_page_size); + ram_block_discard_range(rb, rb_aligned_offset, rb_page_size); /* We ignore errors from ram_block_discard_range(), because it * has already reported them, and failing to discard a balloon * page is not fatal */ - - g_free(balloon->pbp); - balloon->pbp =3D NULL; + virtio_balloon_pbp_free(*pbp); + *pbp =3D NULL; } } =20 @@ -128,7 +151,7 @@ static void balloon_deflate_page(VirtIOBalloon *balloon, =20 if (balloon->pbp) { /* Let's play safe and always reset the pbp on deflation requests.= */ - g_free(balloon->pbp); + virtio_balloon_pbp_free(balloon->pbp); balloon->pbp =3D NULL; } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977222; cv=none; d=zoho.com; s=zohoarc; b=hPBz0G0hJSg2oh18Fd52tE4bmPP8u3yfKdTunVSvPgeOyaCLNAmRuUs09yjiSWuMaHOgq4wuzUO7FE/3SF+5R5x9mUhJasIeO1pwK2zKvsPpr7mwFEA4mCes+eDVgTrY8gns0FLDKfy8S/uilAdhQvjP4EYuS9iXGMzoMvHxL5w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977222; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=M+3I8GKLSkMzcdWlmA4dk0jstKBwFTH3PgvPwfYwXZE=; b=XdiZdGhFDOxK/pPcl4suD5t27w1Bk0dChWEY0cl/NZ/3ZRxqVKmuAvTprg5CO+gAbCptn1yREYWZcNujP8nJ203cAmJEqYL58U0aG5N84fB3bhWDAzOHABmwPhTAouxN4+SVwMBoGeksBjYaAkkYk6eE8maobSOFs3NJqjUDXns= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977222950898.2569299330431; Tue, 1 Oct 2019 17:47:02 -0700 (PDT) Received: from localhost ([::1]:49844 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSnE-0008UX-Cj for importer@patchew.org; Tue, 01 Oct 2019 20:47:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39980) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsG-0001fh-8l for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsB-00027Y-Ke for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:07 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:20080 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsB-0001fC-9F; Tue, 01 Oct 2019 19:48:03 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPOb179355; Tue, 1 Oct 2019 19:47:42 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncwc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:42 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlgWk180245; Tue, 1 Oct 2019 19:47:42 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncw2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:42 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlU5029385; Tue, 1 Oct 2019 23:47:41 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fajb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:41 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NleBa41288190 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:40 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6884428058; Tue, 1 Oct 2019 23:47:40 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4C54B2805A; Tue, 1 Oct 2019 23:47:40 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:40 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 52/97] virtio-balloon: Use temporary PBP only Date: Tue, 1 Oct 2019 18:45:31 -0500 Message-Id: <20191001234616.7825-53-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=4 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , qemu-stable@nongnu.org, David Hildenbrand Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand We still have multiple issues in the current code - The PBP is not freed during unrealize() - The PBP is not reset on device resets: After a reset, the PBP is stale. - We are not indicating VIRTIO_BALLOON_F_MUST_TELL_HOST, therefore guests (esp. legacy guests) will reuse pages without deflating, turning the PBP stale. Adding that would require compat handling. Instead, let's use the PBP only temporarily, when processing one bulk of inflation requests. This will keep guest_page_size > 4k working (with Linux guests). There is nothing to do for deflation requests anymore. The pbp is only used for a limited amount of time. Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < hos= t page size") Cc: qemu-stable@nongnu.org #v4.0.0 Suggested-by: Michael S. Tsirkin Signed-off-by: David Hildenbrand Message-Id: <20190722134108.22151-7-david@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin Acked-by: David Gibson (cherry picked from commit a8cd64d488325f3be5c4ddec4bf07efb3b8c7330) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 21 +++++++++------------ include/hw/virtio/virtio-balloon.h | 3 --- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 49999d0bbe..bd54e302de 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -34,11 +34,11 @@ =20 #define BALLOON_PAGE_SIZE (1 << VIRTIO_BALLOON_PFN_SHIFT) =20 -struct PartiallyBalloonedPage { +typedef struct PartiallyBalloonedPage { ram_addr_t base_gpa; long subpages; unsigned long *bitmap; -}; +} PartiallyBalloonedPage; =20 static void virtio_balloon_pbp_free(PartiallyBalloonedPage *pbp) { @@ -68,11 +68,11 @@ static bool virtio_balloon_pbp_matches(PartiallyBalloon= edPage *pbp, } =20 static void balloon_inflate_page(VirtIOBalloon *balloon, - MemoryRegion *mr, hwaddr mr_offset) + MemoryRegion *mr, hwaddr mr_offset, + PartiallyBalloonedPage **pbp) { void *addr =3D memory_region_get_ram_ptr(mr) + mr_offset; ram_addr_t rb_offset, rb_aligned_offset, base_gpa; - PartiallyBalloonedPage **pbp =3D &balloon->pbp; RAMBlock *rb; size_t rb_page_size; int subpages; @@ -149,12 +149,6 @@ static void balloon_deflate_page(VirtIOBalloon *balloo= n, rb =3D qemu_ram_block_from_host(addr, false, &rb_offset); rb_page_size =3D qemu_ram_pagesize(rb); =20 - if (balloon->pbp) { - /* Let's play safe and always reset the pbp on deflation requests.= */ - virtio_balloon_pbp_free(balloon->pbp); - balloon->pbp =3D NULL; - } - host_addr =3D (void *)((uintptr_t)addr & ~(rb_page_size - 1)); =20 /* When a page is deflated, we hint the whole host page it lives @@ -336,6 +330,7 @@ static void balloon_stats_set_poll_interval(Object *obj= , Visitor *v, static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq) { VirtIOBalloon *s =3D VIRTIO_BALLOON(vdev); + PartiallyBalloonedPage *pbp =3D NULL; VirtQueueElement *elem; MemoryRegionSection section; =20 @@ -344,7 +339,7 @@ static void virtio_balloon_handle_output(VirtIODevice *= vdev, VirtQueue *vq) uint32_t pfn; elem =3D virtqueue_pop(vq, sizeof(VirtQueueElement)); if (!elem) { - return; + break; } =20 while (iov_to_buf(elem->out_sg, elem->out_num, offset, &pfn, 4) = =3D=3D 4) { @@ -373,7 +368,7 @@ static void virtio_balloon_handle_output(VirtIODevice *= vdev, VirtQueue *vq) if (!qemu_balloon_is_inhibited()) { if (vq =3D=3D s->ivq) { balloon_inflate_page(s, section.mr, - section.offset_within_region); + section.offset_within_region, &pb= p); } else if (vq =3D=3D s->dvq) { balloon_deflate_page(s, section.mr, section.offset_wit= hin_region); } else { @@ -387,6 +382,8 @@ static void virtio_balloon_handle_output(VirtIODevice *= vdev, VirtQueue *vq) virtio_notify(vdev, vq); g_free(elem); } + + virtio_balloon_pbp_free(pbp); } =20 static void virtio_balloon_receive_stats(VirtIODevice *vdev, VirtQueue *vq) diff --git a/include/hw/virtio/virtio-balloon.h b/include/hw/virtio/virtio-= balloon.h index 5a99293a45..d1c968d237 100644 --- a/include/hw/virtio/virtio-balloon.h +++ b/include/hw/virtio/virtio-balloon.h @@ -33,8 +33,6 @@ typedef struct virtio_balloon_stat_modern { uint64_t val; } VirtIOBalloonStatModern; =20 -typedef struct PartiallyBalloonedPage PartiallyBalloonedPage; - enum virtio_balloon_free_page_report_status { FREE_PAGE_REPORT_S_STOP =3D 0, FREE_PAGE_REPORT_S_REQUESTED =3D 1, @@ -70,7 +68,6 @@ typedef struct VirtIOBalloon { int64_t stats_last_update; int64_t stats_poll_interval; uint32_t host_features; - PartiallyBalloonedPage *pbp; =20 bool qemu_4_0_config_size; } VirtIOBalloon; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569976987; cv=none; d=zoho.com; s=zohoarc; b=n1dV11ReXJmFCOiI+4YIaF0jddmikPE50czDznpJZuIA5TzOojmq5UBPOYwiWMmDXAu7tvD342u58P5EJpe2Qt7j0Nx5enMi1hx4eyNHTwHyqB51BNz03m8dZLaDRSiR1W7nZa36S2TlmAQplhqcaI94z6iOl8mIYZ/kjONNSBs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569976987; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=4zy8y6WaLJn1lkQP4CFOB+fsEpGyQUketjQt5G+dIdI=; b=KG6I980lF8jCGAsmd2bnjBY8akZMZfOUo5yK4L1OBRmCHZPWSodDOXK1kzS5fcLzPfB2Cw9RVWWJnv/YFNyCjFKevrjJI35+WrqMvosg7FbyZi1utkZj4XT/XmJsTLoJKYWbMsAoF7RSAcrVYfD0xdKpbk1akH1GUB0dACtm6Kk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569976987416899.9238727723305; Tue, 1 Oct 2019 17:43:07 -0700 (PDT) Received: from localhost ([::1]:49810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSjP-0004B2-CN for importer@patchew.org; Tue, 01 Oct 2019 20:43:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39974) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsF-0001fW-Rm for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsC-00028k-FU for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:07 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:9752) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsC-0001gF-0x; Tue, 01 Oct 2019 19:48:04 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlO0f053445; Tue, 1 Oct 2019 19:47:43 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17168u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:42 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlgOm053994; Tue, 1 Oct 2019 19:47:42 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg17168d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:42 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkK4027153; Tue, 1 Oct 2019 23:47:41 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bh2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:41 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NleA941287940 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:41 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DD49928058; Tue, 1 Oct 2019 23:47:40 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C16CC2805A; Tue, 1 Oct 2019 23:47:40 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:40 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 53/97] virtio-balloon: don't track subpages for the PBP Date: Tue, 1 Oct 2019 18:45:32 -0500 Message-Id: <20191001234616.7825-54-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , qemu-stable@nongnu.org, David Hildenbrand Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand As ramblocks cannot get removed/readded while we are processing a bulk of inflation requests, there is no more need to track the page size in form of the number of subpages. Suggested-by: David Gibson Signed-off-by: David Hildenbrand Message-Id: <20190725113638.4702-8-david@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit 9a7ca8a7c920360db9dcaf616ca6f1440c025043) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index bd54e302de..4a5a22ca3d 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -36,7 +36,6 @@ =20 typedef struct PartiallyBalloonedPage { ram_addr_t base_gpa; - long subpages; unsigned long *bitmap; } PartiallyBalloonedPage; =20 @@ -55,16 +54,15 @@ static PartiallyBalloonedPage *virtio_balloon_pbp_alloc= (ram_addr_t base_gpa, PartiallyBalloonedPage *pbp =3D g_new0(PartiallyBalloonedPage, 1); =20 pbp->base_gpa =3D base_gpa; - pbp->subpages =3D subpages; pbp->bitmap =3D bitmap_new(subpages); =20 return pbp; } =20 static bool virtio_balloon_pbp_matches(PartiallyBalloonedPage *pbp, - ram_addr_t base_gpa, long subpages) + ram_addr_t base_gpa) { - return pbp->subpages =3D=3D subpages && pbp->base_gpa =3D=3D base_gpa; + return pbp->base_gpa =3D=3D base_gpa; } =20 static void balloon_inflate_page(VirtIOBalloon *balloon, @@ -106,7 +104,7 @@ static void balloon_inflate_page(VirtIOBalloon *balloon, base_gpa =3D memory_region_get_ram_addr(mr) + mr_offset - (rb_offset - rb_aligned_offset); =20 - if (*pbp && !virtio_balloon_pbp_matches(*pbp, base_gpa, subpages)) { + if (*pbp && !virtio_balloon_pbp_matches(*pbp, base_gpa)) { /* We've partially ballooned part of a host page, but now * we're trying to balloon part of a different one. Too hard, * give up on the old partial page */ --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977067; cv=none; d=zoho.com; s=zohoarc; b=NK8WUYGg6EAl/RdezZt6PhiB4W6nusZUZYsp1r8oa9RY2owJ0/VO+TjDWGmIQHR0ruGYMKCZLSBpz9CT+YFT2MNYIuwpcCuY2rO/bP7aliR9kOYffnJVE/435TY5o7gb6v8V3OIhynzRE14Cr/WIgnPXeW2f7IrsU8qmPdJgcPI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977067; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=166DGSdoTuLeN0zAwM4isBLo7SjbYbdagoz99KE3yoE=; b=ia6LPITeHepptDbfraobLqe880ijhEJO5t8YfyHWMHFAt/rJ7PI5utOyi6g8CVrWay/5GMS/5pbkyCBbzAneFSTQMppnmFRFy8RBSMxnDjHyFVEXVUSvbB7EH7ns1FrOy9AleF6uiIOqETRMPYKsKe/v0EatVPJ45SSi3NOLRpM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977067836261.19138630760096; Tue, 1 Oct 2019 17:44:27 -0700 (PDT) Received: from localhost ([::1]:49814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSkj-0005nV-Qn for importer@patchew.org; Tue, 01 Oct 2019 20:44:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40240) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsL-0001nr-7P for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsE-0002DR-UF for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:12 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:28616) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsE-0001hx-Eo; Tue, 01 Oct 2019 19:48:06 -0400 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlMkj193517; Tue, 1 Oct 2019 19:47:44 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsg6u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:43 -0400 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlMnL193520; Tue, 1 Oct 2019 19:47:42 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsg6m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:42 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njm5w023587; Tue, 1 Oct 2019 23:47:41 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577gyn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:41 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlfS514090956 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:41 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 62A432805A; Tue, 1 Oct 2019 23:47:41 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 46F442805E; Tue, 1 Oct 2019 23:47:41 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:41 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 54/97] virtio-balloon: free pbp more aggressively Date: Tue, 1 Oct 2019 18:45:33 -0500 Message-Id: <20191001234616.7825-55-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Hildenbrand , qemu-stable@nongnu.org, "Michael S. Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Michael S. Tsirkin" Previous patches switched to a temporary pbp but that does not go far enough: after device uses a buffer, guest is free to reuse it, so tracking the page and freeing it later is wrong. Free and reset the pbp after we push each element. Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < hos= t page size") Cc: qemu-stable@nongnu.org #v4.0.0 Cc: David Hildenbrand Signed-off-by: Michael S. Tsirkin (cherry picked from commit 1b47b37c33ec01ae1efc527f4c97f97f93723bc4) Signed-off-by: Michael Roth --- hw/virtio/virtio-balloon.c | 37 ++++++++++++++++--------------------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index 4a5a22ca3d..adde97fe4b 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -41,22 +41,19 @@ typedef struct PartiallyBalloonedPage { =20 static void virtio_balloon_pbp_free(PartiallyBalloonedPage *pbp) { - if (!pbp) { + if (!pbp->bitmap) { return; } g_free(pbp->bitmap); - g_free(pbp); + pbp->bitmap =3D NULL; } =20 -static PartiallyBalloonedPage *virtio_balloon_pbp_alloc(ram_addr_t base_gp= a, - long subpages) +static void virtio_balloon_pbp_alloc(PartiallyBalloonedPage *pbp, + ram_addr_t base_gpa, + long subpages) { - PartiallyBalloonedPage *pbp =3D g_new0(PartiallyBalloonedPage, 1); - pbp->base_gpa =3D base_gpa; pbp->bitmap =3D bitmap_new(subpages); - - return pbp; } =20 static bool virtio_balloon_pbp_matches(PartiallyBalloonedPage *pbp, @@ -67,7 +64,7 @@ static bool virtio_balloon_pbp_matches(PartiallyBallooned= Page *pbp, =20 static void balloon_inflate_page(VirtIOBalloon *balloon, MemoryRegion *mr, hwaddr mr_offset, - PartiallyBalloonedPage **pbp) + PartiallyBalloonedPage *pbp) { void *addr =3D memory_region_get_ram_ptr(mr) + mr_offset; ram_addr_t rb_offset, rb_aligned_offset, base_gpa; @@ -104,22 +101,21 @@ static void balloon_inflate_page(VirtIOBalloon *ballo= on, base_gpa =3D memory_region_get_ram_addr(mr) + mr_offset - (rb_offset - rb_aligned_offset); =20 - if (*pbp && !virtio_balloon_pbp_matches(*pbp, base_gpa)) { + if (pbp->bitmap && !virtio_balloon_pbp_matches(pbp, base_gpa)) { /* We've partially ballooned part of a host page, but now * we're trying to balloon part of a different one. Too hard, * give up on the old partial page */ - virtio_balloon_pbp_free(*pbp); - *pbp =3D NULL; + virtio_balloon_pbp_free(pbp); } =20 - if (!*pbp) { - *pbp =3D virtio_balloon_pbp_alloc(base_gpa, subpages); + if (!pbp->bitmap) { + virtio_balloon_pbp_alloc(pbp, base_gpa, subpages); } =20 set_bit((rb_offset - rb_aligned_offset) / BALLOON_PAGE_SIZE, - (*pbp)->bitmap); + pbp->bitmap); =20 - if (bitmap_full((*pbp)->bitmap, subpages)) { + if (bitmap_full(pbp->bitmap, subpages)) { /* We've accumulated a full host page, we can actually discard * it now */ =20 @@ -127,8 +123,7 @@ static void balloon_inflate_page(VirtIOBalloon *balloon, /* We ignore errors from ram_block_discard_range(), because it * has already reported them, and failing to discard a balloon * page is not fatal */ - virtio_balloon_pbp_free(*pbp); - *pbp =3D NULL; + virtio_balloon_pbp_free(pbp); } } =20 @@ -328,13 +323,14 @@ static void balloon_stats_set_poll_interval(Object *o= bj, Visitor *v, static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq) { VirtIOBalloon *s =3D VIRTIO_BALLOON(vdev); - PartiallyBalloonedPage *pbp =3D NULL; VirtQueueElement *elem; MemoryRegionSection section; =20 for (;;) { + PartiallyBalloonedPage pbp =3D {}; size_t offset =3D 0; uint32_t pfn; + elem =3D virtqueue_pop(vq, sizeof(VirtQueueElement)); if (!elem) { break; @@ -379,9 +375,8 @@ static void virtio_balloon_handle_output(VirtIODevice *= vdev, VirtQueue *vq) virtqueue_push(vq, elem, offset); virtio_notify(vdev, vq); g_free(elem); + virtio_balloon_pbp_free(&pbp); } - - virtio_balloon_pbp_free(pbp); } =20 static void virtio_balloon_receive_stats(VirtIODevice *vdev, VirtQueue *vq) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977345; cv=none; d=zoho.com; s=zohoarc; b=QfDJsTtTLyfxhHUc57c9uXr5FLY7FFYIGOJL67UrNbRF5dW9otZNvuaRYS0VZIgKH84ahXCpubxSFcgMSIgQ2DkPUawxi0ZO3Om25xu7ELSBYRrdIXbpTtfIIk6c9sFT0XlDBMgKZXjhb2vdC9FI8YuF9B6ISvUjAc4E19OI1C0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977345; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ngpezeVp/bHCWbakDi/OzglXYM30po8SmsEzUIsbSZI=; b=NEpVQV2MjruGAjm9R4tkB5wNYOrfAFRuuAli8xytSfOMep1ycSnizkWjAwXAReF2WSePQLjr1oke7vccn2DK5KDnIQeswOdPfv+YrnRD/lHtwriJfL5/V3fM8jnwwr4/9uap//WpniWrZy+R7yCJ6H83V6R6jRPGcUA5zFBN9dM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977345859134.7919967138523; Tue, 1 Oct 2019 17:49:05 -0700 (PDT) Received: from localhost ([::1]:49854 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSp9-00024F-1L for importer@patchew.org; Tue, 01 Oct 2019 20:48:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40371) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsP-0001rp-SI for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsJ-0002LN-5Q for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:13 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:8602 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsB-0001g3-Iv; Tue, 01 Oct 2019 19:48:03 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPS3037480; Tue, 1 Oct 2019 19:47:43 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b35p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:42 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njod1031648; Tue, 1 Oct 2019 23:47:42 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fc5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:42 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlfpi34996640 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:41 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC75E2805A; Tue, 1 Oct 2019 23:47:41 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C14AE28059; Tue, 1 Oct 2019 23:47:41 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:41 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 55/97] i386/acpi: fix gint overflow in crs_range_compare Date: Tue, 1 Oct 2019 18:45:34 -0500 Message-Id: <20191001234616.7825-56-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=62 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Evgeny Yakovlev Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Evgeny Yakovlev When very large regions (32GB sized in our case, PCI pass-through of GPUs) are compared substraction result does not fit into gint. As a result crs_replace_with_free_ranges does not get sorted ranges and incorrectly computes PCI64 free space regions. Which then makes linux guest complain about device and PCI64 hole intersection and device becomes unusable. Fix that by returning exactly fitting ranges. Also fix indentation of an entire crs_replace_with_free_ranges to make checkpatch happy. Cc: qemu-stable@nongnu.org Signed-off-by: Evgeny Yakovlev Message-Id: <1563466463-26012-1-git-send-email-wrfsh@yandex-team.ru> Signed-off-by: Evgeny Yakovlev (cherry picked from commit 21e2acd583126db94f6d881005cd58e835160582) Signed-off-by: Michael Roth --- hw/i386/acpi-build.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index ede27ab3c4..bf59c475be 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -743,10 +743,16 @@ static void crs_range_set_free(CrsRangeSet *range_set) =20 static gint crs_range_compare(gconstpointer a, gconstpointer b) { - CrsRangeEntry *entry_a =3D *(CrsRangeEntry **)a; - CrsRangeEntry *entry_b =3D *(CrsRangeEntry **)b; + CrsRangeEntry *entry_a =3D *(CrsRangeEntry **)a; + CrsRangeEntry *entry_b =3D *(CrsRangeEntry **)b; =20 - return (int64_t)entry_a->base - (int64_t)entry_b->base; + if (entry_a->base < entry_b->base) { + return -1; + } else if (entry_a->base > entry_b->base) { + return 1; + } else { + return 0; + } } =20 /* --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977932; cv=none; d=zoho.com; s=zohoarc; b=ZdHg8uZ0ejOo7szNLhWl5t3wClEbhPK7rCdDBugrTj1o/nsTWV1c2MK0GqpJZGeXBlhwprn9mOn63t9nNQOQ9ux1Pf3DFC8JBJVzxF0YaW/dKFn6ZZ6NErtRDSl27c8X1A2Rkd5fJLvznF3sFdziyxLP1yu9fWIloJHUKQYLPyE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977932; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=tqo/UuX+khAgU+JC4XDkUkJvVgzX86wzf7/lDI4Bc00=; b=knsCnOB8ISp3mzmXeLm0Ljg1bkOY1A09p6BbKGqoe55jv1N3roiObc3eLfE82JD1rzvPRC9gXyPneREsRxyJjftISawLibyumIYOf45sbfw7gsf01tcOV6lwKtcSITaddYNUySOmLxLEKDxATw7Idf/V+TD9JFeLpbEP7+cEY0Q= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977932040317.1299762375194; Tue, 1 Oct 2019 17:58:52 -0700 (PDT) Received: from localhost ([::1]:49944 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSyg-0003F9-AB for importer@patchew.org; Tue, 01 Oct 2019 20:58:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40281) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsM-0001p8-6g for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsI-0002Ja-3i for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:13 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:25424) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsC-0001h5-3c; Tue, 01 Oct 2019 19:48:04 -0400 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlMwt193511; Tue, 1 Oct 2019 19:47:43 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsg6y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:43 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjmlN023578; Tue, 1 Oct 2019 23:47:42 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577gyw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:42 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlgtE46727432 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:42 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 66B482805C; Tue, 1 Oct 2019 23:47:42 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 498732805E; Tue, 1 Oct 2019 23:47:42 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:42 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 56/97] tpm: Exit in reset when backend indicates failure Date: Tue, 1 Oct 2019 18:45:35 -0500 Message-Id: <20191001234616.7825-57-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-001b2d01.pphosted.com id x91NlMwt193511 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Stefan Berger , qemu-stable@nongnu.org, Stefan Berger Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Stefan Berger Exit() in the frontend reset function when the backend indicates intialization failure. Signed-off-by: Stefan Berger Reviewed-by: Marc-Andr=C3=A9 Lureau (cherry picked from commit bcfd16fe26d6bb6eabfd2dfb46b9fda59d5493db) Signed-off-by: Michael Roth --- hw/tpm/tpm_crb.c | 4 +++- hw/tpm/tpm_tis.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/hw/tpm/tpm_crb.c b/hw/tpm/tpm_crb.c index 3087acc4ab..55ea74f3ba 100644 --- a/hw/tpm/tpm_crb.c +++ b/hw/tpm/tpm_crb.c @@ -273,7 +273,9 @@ static void tpm_crb_reset(void *dev) s->be_buffer_size =3D MIN(tpm_backend_get_buffer_size(s->tpmbe), CRB_CTRL_CMD_SIZE); =20 - tpm_backend_startup_tpm(s->tpmbe, s->be_buffer_size); + if (tpm_backend_startup_tpm(s->tpmbe, s->be_buffer_size) < 0) { + exit(1); + } } =20 static void tpm_crb_realize(DeviceState *dev, Error **errp) diff --git a/hw/tpm/tpm_tis.c b/hw/tpm/tpm_tis.c index fd183e8deb..4203da1008 100644 --- a/hw/tpm/tpm_tis.c +++ b/hw/tpm/tpm_tis.c @@ -909,7 +909,9 @@ static void tpm_tis_reset(DeviceState *dev) s->rw_offset =3D 0; } =20 - tpm_backend_startup_tpm(s->be_driver, s->be_buffer_size); + if (tpm_backend_startup_tpm(s->be_driver, s->be_buffer_size) < 0) { + exit(1); + } } =20 /* persistent state handling */ --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980915; cv=none; d=zoho.com; s=zohoarc; b=SvyeeCSiyKDd99RgQ2NkJ3ebXh1al7I0YEHgmg+mI/JX48iiB9gwdn1b4PpULzG8jc4+pfyd4YZa9WciY61FoRT9AuYpeg1Al3lk9m1I0dLxWe11C1Jl2nIKGBQaLBPzzLJ4ZXuwLQniaEsiyexZHZG9BpIcpf04X3HKh5sH108= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980915; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ODyAhiiJoxY/Bt/jyczo1T/hBtmxb67gn5Jar3PLVHg=; b=ZTHlvoMaYJYEo2fMF433vHVh/6gEX461z2UEICEsjX//QwD0flBuOOApWpoVbh+cVoCLypPS9nEEzi8PaWOVXpOn+CIccHv9/zHeCNzu9D5WcKem12tA4QlA2k1fOWfuPomjbaOft78y7ZccD+S3WNGIG46lQ2k68HgSXoh8rpg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569980915105884.6347889827024; Tue, 1 Oct 2019 18:48:35 -0700 (PDT) Received: from localhost ([::1]:50372 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTkn-0001zz-4p for importer@patchew.org; Tue, 01 Oct 2019 21:48:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40700) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsd-000232-An for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002ZW-Lv for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:30 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:12646 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsE-0001im-E6; Tue, 01 Oct 2019 19:48:06 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlUAk179530; Tue, 1 Oct 2019 19:47:44 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpncx6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:44 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlKq027161; Tue, 1 Oct 2019 23:47:43 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bhc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:43 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlggO45023722 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:43 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E2E6228059; Tue, 1 Oct 2019 23:47:42 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C695C2805A; Tue, 1 Oct 2019 23:47:42 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:42 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 57/97] tpm_emulator: Translate TPM error codes to strings Date: Tue, 1 Oct 2019 18:45:36 -0500 Message-Id: <20191001234616.7825-58-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0b-001b2d01.pphosted.com id x91NlUAk179530 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Stefan Berger , qemu-stable@nongnu.org, Stefan Berger Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Stefan Berger Implement a function to translate TPM error codes to strings so that at least the most common error codes can be translated to human readable strings. Signed-off-by: Stefan Berger Reviewed-by: Marc-Andr=C3=A9 Lureau (cherry picked from commit 7e095e84ba0b7c0a1ac45bc6824dace2fd352e56) Signed-off-by: Michael Roth --- hw/tpm/tpm_emulator.c | 60 +++++++++++++++++++++++++++++++++++-------- hw/tpm/tpm_int.h | 13 ++++++++++ 2 files changed, 63 insertions(+), 10 deletions(-) diff --git a/hw/tpm/tpm_emulator.c b/hw/tpm/tpm_emulator.c index 70f4b10284..58894dc320 100644 --- a/hw/tpm/tpm_emulator.c +++ b/hw/tpm/tpm_emulator.c @@ -81,6 +81,40 @@ typedef struct TPMEmulator { TPMBlobBuffers state_blobs; } TPMEmulator; =20 +struct tpm_error { + uint32_t tpm_result; + const char *string; +}; + +static const struct tpm_error tpm_errors[] =3D { + /* TPM 1.2 error codes */ + { TPM_BAD_PARAMETER , "a parameter is bad" }, + { TPM_FAIL , "operation failed" }, + { TPM_KEYNOTFOUND , "key could not be found" }, + { TPM_BAD_PARAM_SIZE , "bad parameter size"}, + { TPM_ENCRYPT_ERROR , "encryption error" }, + { TPM_DECRYPT_ERROR , "decryption error" }, + { TPM_BAD_KEY_PROPERTY, "bad key property" }, + { TPM_BAD_MODE , "bad (encryption) mode" }, + { TPM_BAD_VERSION , "bad version identifier" }, + { TPM_BAD_LOCALITY , "bad locality" }, + /* TPM 2 error codes */ + { TPM_RC_FAILURE , "operation failed" }, + { TPM_RC_LOCALITY , "bad locality" }, + { TPM_RC_INSUFFICIENT, "insufficient amount of data" }, +}; + +static const char *tpm_emulator_strerror(uint32_t tpm_result) +{ + size_t i; + + for (i =3D 0; i < ARRAY_SIZE(tpm_errors); i++) { + if (tpm_errors[i].tpm_result =3D=3D tpm_result) { + return tpm_errors[i].string; + } + } + return ""; +} =20 static int tpm_emulator_ctrlcmd(TPMEmulator *tpm, unsigned long cmd, void = *msg, size_t msg_len_in, size_t msg_len_out) @@ -263,7 +297,8 @@ static int tpm_emulator_stop_tpm(TPMBackend *tb) =20 res =3D be32_to_cpu(res); if (res) { - error_report("tpm-emulator: TPM result for CMD_STOP: 0x%x", res); + error_report("tpm-emulator: TPM result for CMD_STOP: 0x%x %s", res, + tpm_emulator_strerror(res)); return -1; } =20 @@ -292,8 +327,9 @@ static int tpm_emulator_set_buffer_size(TPMBackend *tb, =20 psbs.u.resp.tpm_result =3D be32_to_cpu(psbs.u.resp.tpm_result); if (psbs.u.resp.tpm_result !=3D 0) { - error_report("tpm-emulator: TPM result for set buffer size : 0x%x", - psbs.u.resp.tpm_result); + error_report("tpm-emulator: TPM result for set buffer size : 0x%x = %s", + psbs.u.resp.tpm_result, + tpm_emulator_strerror(psbs.u.resp.tpm_result)); return -1; } =20 @@ -338,7 +374,8 @@ static int tpm_emulator_startup_tpm_resume(TPMBackend *= tb, size_t buffersize, =20 res =3D be32_to_cpu(init.u.resp.tpm_result); if (res) { - error_report("tpm-emulator: TPM result for CMD_INIT: 0x%x", res); + error_report("tpm-emulator: TPM result for CMD_INIT: 0x%x %s", res, + tpm_emulator_strerror(res)); goto err_exit; } return 0; @@ -398,8 +435,9 @@ static int tpm_emulator_reset_tpm_established_flag(TPMB= ackend *tb, =20 res =3D be32_to_cpu(reset_est.u.resp.tpm_result); if (res) { - error_report("tpm-emulator: TPM result for rest establixhed flag: = 0x%x", - res); + error_report( + "tpm-emulator: TPM result for rest established flag: 0x%x %s", + res, tpm_emulator_strerror(res)); return -1; } =20 @@ -637,7 +675,8 @@ static int tpm_emulator_get_state_blob(TPMEmulator *tpm= _emu, res =3D be32_to_cpu(pgs.u.resp.tpm_result); if (res !=3D 0 && (res & 0x800) =3D=3D 0) { error_report("tpm-emulator: Getting the stateblob (type %d) failed= " - "with a TPM error 0x%x", type, res); + "with a TPM error 0x%x %s", type, res, + tpm_emulator_strerror(res)); return -1; } =20 @@ -757,7 +796,8 @@ static int tpm_emulator_set_state_blob(TPMEmulator *tpm= _emu, tpm_result =3D be32_to_cpu(pss.u.resp.tpm_result); if (tpm_result !=3D 0) { error_report("tpm-emulator: Setting the stateblob (type %d) failed= " - "with a TPM error 0x%x", type, tpm_result); + "with a TPM error 0x%x %s", type, tpm_result, + tpm_emulator_strerror(tpm_result)); return -1; } =20 @@ -887,8 +927,8 @@ static void tpm_emulator_shutdown(TPMEmulator *tpm_emu) error_report("tpm-emulator: Could not cleanly shutdown the TPM: %s= ", strerror(errno)); } else if (res !=3D 0) { - error_report("tpm-emulator: TPM result for sutdown: 0x%x", - be32_to_cpu(res)); + error_report("tpm-emulator: TPM result for shutdown: 0x%x %s", + be32_to_cpu(res), tpm_emulator_strerror(be32_to_cpu(r= es))); } } =20 diff --git a/hw/tpm/tpm_int.h b/hw/tpm/tpm_int.h index a4c77fbd7e..3fb28a9d6c 100644 --- a/hw/tpm/tpm_int.h +++ b/hw/tpm/tpm_int.h @@ -39,7 +39,16 @@ struct tpm_resp_hdr { #define TPM_TAG_RSP_AUTH1_COMMAND 0xc5 #define TPM_TAG_RSP_AUTH2_COMMAND 0xc6 =20 +#define TPM_BAD_PARAMETER 3 #define TPM_FAIL 9 +#define TPM_KEYNOTFOUND 13 +#define TPM_BAD_PARAM_SIZE 25 +#define TPM_ENCRYPT_ERROR 32 +#define TPM_DECRYPT_ERROR 33 +#define TPM_BAD_KEY_PROPERTY 40 +#define TPM_BAD_MODE 44 +#define TPM_BAD_VERSION 46 +#define TPM_BAD_LOCALITY 61 =20 #define TPM_ORD_ContinueSelfTest 0x53 #define TPM_ORD_GetTicks 0xf1 @@ -59,4 +68,8 @@ struct tpm_resp_hdr { =20 #define TPM2_PT_MAX_COMMAND_SIZE 0x11e =20 +#define TPM_RC_INSUFFICIENT 0x9a +#define TPM_RC_FAILURE 0x101 +#define TPM_RC_LOCALITY 0x907 + #endif /* TPM_TPM_INT_H */ --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977713; cv=none; d=zoho.com; s=zohoarc; b=AMjemmAolYG4PPPSpVgC0ulqAzA701JqWQz5kuFqj1RgsU4dzt160OSLYeIFn6xnsFSUKgaLEaJGO5OdJXW8CUEoTFaerkLu/Nn+6T748BQ1coG51eGbW1BImvruzE0g177wBnNJEgpT0tAK8RZzEsG/c5GjdNVtpqBdzwS4vv8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977713; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=DV6Z5XoO9nVP3sQ5jI2sTYGBIyAJjlSJHBlJjyJNwgk=; b=fvg0euh0RhnM5wEmNU6ltGMwkCRElfwKi8zZN9nDlIPi6RVm17eAxztCJGfVI0JHcH8RRC3NjU2d3jIQfT1wiLJhNFVGVeCCdKCEbtr05rSZwMsokyQO7YrLmrA5G3R4TM350vNqgfzP3Omj6qX482GxupYeT+57MvS1lWy5M+g= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977713662433.51494938887663; Tue, 1 Oct 2019 17:55:13 -0700 (PDT) Received: from localhost ([::1]:49908 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSvA-0007k7-7v for importer@patchew.org; Tue, 01 Oct 2019 20:55:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40426) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tX-Cw for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsK-0002N5-3K for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:18732 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001pm-CV; Tue, 01 Oct 2019 19:48:11 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPS4037480; Tue, 1 Oct 2019 19:47:45 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b36k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:44 -0400 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlio5038437; Tue, 1 Oct 2019 19:47:44 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b36a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:44 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlJa006377; Tue, 1 Oct 2019 23:47:43 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye8n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:43 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlh6O47382828 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:43 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 67A0F2805C; Tue, 1 Oct 2019 23:47:43 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4C13E28058; Tue, 1 Oct 2019 23:47:43 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:43 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 58/97] block/backup: simplify backup_incremental_init_copy_bitmap Date: Tue, 1 Oct 2019 18:45:37 -0500 Message-Id: <20191001234616.7825-59-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sementsov-Ogievskiy , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Simplify backup_incremental_init_copy_bitmap using the function bdrv_dirty_bitmap_next_dirty_area. Note: move to job->len instead of bitmap size: it should not matter but less code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20190429090842.57910-2-vsementsov@virtuozzo.com Signed-off-by: Max Reitz (cherry picked from commit c2da3413c021398152e98022261bb1643276a2fe) *prereq for 4a5b91ca Signed-off-by: Michael Roth --- block/backup.c | 40 ++++++++++++---------------------------- 1 file changed, 12 insertions(+), 28 deletions(-) diff --git a/block/backup.c b/block/backup.c index 9988753249..d9f5db18ac 100644 --- a/block/backup.c +++ b/block/backup.c @@ -403,43 +403,27 @@ static int coroutine_fn backup_run_incremental(Backup= BlockJob *job) /* init copy_bitmap from sync_bitmap */ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) { - BdrvDirtyBitmapIter *dbi; - int64_t offset; - int64_t end =3D DIV_ROUND_UP(bdrv_dirty_bitmap_size(job->sync_bitmap), - job->cluster_size); - - dbi =3D bdrv_dirty_iter_new(job->sync_bitmap); - while ((offset =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { - int64_t cluster =3D offset / job->cluster_size; - int64_t next_cluster; - - offset +=3D bdrv_dirty_bitmap_granularity(job->sync_bitmap); - if (offset >=3D bdrv_dirty_bitmap_size(job->sync_bitmap)) { - hbitmap_set(job->copy_bitmap, cluster, end - cluster); - break; - } + uint64_t offset =3D 0; + uint64_t bytes =3D job->len; =20 - offset =3D bdrv_dirty_bitmap_next_zero(job->sync_bitmap, offset, - UINT64_MAX); - if (offset =3D=3D -1) { - hbitmap_set(job->copy_bitmap, cluster, end - cluster); - break; - } + while (bdrv_dirty_bitmap_next_dirty_area(job->sync_bitmap, + &offset, &bytes)) + { + uint64_t cluster =3D offset / job->cluster_size; + uint64_t end_cluster =3D DIV_ROUND_UP(offset + bytes, job->cluster= _size); =20 - next_cluster =3D DIV_ROUND_UP(offset, job->cluster_size); - hbitmap_set(job->copy_bitmap, cluster, next_cluster - cluster); - if (next_cluster >=3D end) { + hbitmap_set(job->copy_bitmap, cluster, end_cluster - cluster); + + offset =3D end_cluster * job->cluster_size; + if (offset >=3D job->len) { break; } - - bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size); + bytes =3D job->len - offset; } =20 /* TODO job_progress_set_remaining() would make more sense */ job_progress_update(&job->common.job, job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); - - bdrv_dirty_iter_free(dbi); } =20 static int coroutine_fn backup_run(Job *job, Error **errp) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979389; cv=none; d=zoho.com; s=zohoarc; b=Y8UZpc0rueaXkl/sftlV6PsxjoEltBc/GUlTv84IFPvWgB5Ap5JYPwO7goZ318+dZU1OcisABh3zGhlymFVkYk9FAPAUh6iVxSlsVLeU2bvOFLI4OLeMVi0ZsgsowyvRKcgLr9rZQBfUOCs+XvRw2ngG50QCFcHu3mUtHlDNxXM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979389; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=tsYMge33ranQ+nLRcwGV/tp4/csJqDwsUkuC447qRZY=; b=d/38s6n2sSfzuq4VeOlGT4VAh2/7xMr27SI9+wXrYLM4/lIzzvpbsU+WQuAKGVogR74yafxG8Utx76nchHQjY75iZK7UqqjPOUV6HabOpOR4Sn5aaPFHlZHh3GKLpc8CX/Ui2uJdn1nx/H84url6fRxVjM9nBQRVIYOQb6nxaiE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979389946120.73546178784534; Tue, 1 Oct 2019 18:23:09 -0700 (PDT) Received: from localhost ([::1]:50142 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTMC-0000ds-4q for importer@patchew.org; Tue, 01 Oct 2019 21:23:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40414) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tN-Dl for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsJ-0002MQ-MU for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:31446) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001q0-7g; Tue, 01 Oct 2019 19:48:11 -0400 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlMkk193517; Tue, 1 Oct 2019 19:47:45 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsg7y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:45 -0400 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NliBU194129; Tue, 1 Oct 2019 19:47:45 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsg7q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:44 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njl1G006380; Tue, 1 Oct 2019 23:47:44 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye8p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:44 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nli2653674454 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:44 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ECE6828058; Tue, 1 Oct 2019 23:47:43 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C781F28059; Tue, 1 Oct 2019 23:47:43 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:43 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 59/97] block/backup: move to copy_bitmap with granularity Date: Tue, 1 Oct 2019 18:45:38 -0500 Message-Id: <20191001234616.7825-60-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sementsov-Ogievskiy , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy We are going to share this bitmap between backup and backup-top filter driver, so let's share something more meaningful. It also simplifies some calculations. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20190429090842.57910-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz (cherry picked from commit a8389e315ef71913ae99cf8f3b1f89e84631f599) *prereq for 4a5b91ca Signed-off-by: Michael Roth --- block/backup.c | 48 +++++++++++++++++++++++------------------------- 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/block/backup.c b/block/backup.c index d9f5db18ac..510fc54f98 100644 --- a/block/backup.c +++ b/block/backup.c @@ -113,7 +113,8 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, int read_flags =3D is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 - hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); + assert(QEMU_IS_ALIGNED(start, job->cluster_size)); + hbitmap_reset(job->copy_bitmap, start, job->cluster_size); nbytes =3D MIN(job->cluster_size, job->len - start); if (!*bounce_buffer) { *bounce_buffer =3D blk_blockalign(blk, job->cluster_size); @@ -147,7 +148,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, =20 return nbytes; fail: - hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); + hbitmap_set(job->copy_bitmap, start, job->cluster_size); return ret; =20 } @@ -167,16 +168,15 @@ static int coroutine_fn backup_cow_with_offload(Backu= pBlockJob *job, int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); + assert(QEMU_IS_ALIGNED(start, job->cluster_size)); nbytes =3D MIN(job->copy_range_size, end - start); nr_clusters =3D DIV_ROUND_UP(nbytes, job->cluster_size); - hbitmap_reset(job->copy_bitmap, start / job->cluster_size, - nr_clusters); + hbitmap_reset(job->copy_bitmap, start, job->cluster_size * nr_clusters= ); ret =3D blk_co_copy_range(blk, start, job->target, start, nbytes, read_flags, write_flags); if (ret < 0) { trace_backup_do_cow_copy_range_fail(job, start, ret); - hbitmap_set(job->copy_bitmap, start / job->cluster_size, - nr_clusters); + hbitmap_set(job->copy_bitmap, start, job->cluster_size * nr_cluste= rs); return ret; } =20 @@ -204,7 +204,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *j= ob, cow_request_begin(&cow_request, job, start, end); =20 while (start < end) { - if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) { + if (!hbitmap_get(job->copy_bitmap, start)) { trace_backup_do_cow_skip(job, start); start +=3D job->cluster_size; continue; /* already copied */ @@ -300,6 +300,11 @@ static void backup_clean(Job *job) assert(s->target); blk_unref(s->target); s->target =3D NULL; + + if (s->copy_bitmap) { + hbitmap_free(s->copy_bitmap); + s->copy_bitmap =3D NULL; + } } =20 static void backup_attached_aio_context(BlockJob *job, AioContext *aio_con= text) @@ -312,7 +317,6 @@ static void backup_attached_aio_context(BlockJob *job, = AioContext *aio_context) void backup_do_checkpoint(BlockJob *job, Error **errp) { BackupBlockJob *backup_job =3D container_of(job, BackupBlockJob, commo= n); - int64_t len; =20 assert(block_job_driver(job) =3D=3D &backup_job_driver); =20 @@ -322,8 +326,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) return; } =20 - len =3D DIV_ROUND_UP(backup_job->len, backup_job->cluster_size); - hbitmap_set(backup_job->copy_bitmap, 0, len); + hbitmap_set(backup_job->copy_bitmap, 0, backup_job->len); } =20 static void backup_drain(BlockJob *job) @@ -378,16 +381,16 @@ static int coroutine_fn backup_run_incremental(Backup= BlockJob *job) { int ret; bool error_is_read; - int64_t cluster; + int64_t offset; HBitmapIter hbi; =20 hbitmap_iter_init(&hbi, job->copy_bitmap, 0); - while ((cluster =3D hbitmap_iter_next(&hbi)) !=3D -1) { + while ((offset =3D hbitmap_iter_next(&hbi)) !=3D -1) { do { if (yield_and_check(job)) { return 0; } - ret =3D backup_do_cow(job, cluster * job->cluster_size, + ret =3D backup_do_cow(job, offset, job->cluster_size, &error_is_read, false); if (ret < 0 && backup_error_action(job, error_is_read, -ret) = =3D=3D BLOCK_ERROR_ACTION_REPORT) @@ -409,12 +412,9 @@ static void backup_incremental_init_copy_bitmap(Backup= BlockJob *job) while (bdrv_dirty_bitmap_next_dirty_area(job->sync_bitmap, &offset, &bytes)) { - uint64_t cluster =3D offset / job->cluster_size; - uint64_t end_cluster =3D DIV_ROUND_UP(offset + bytes, job->cluster= _size); + hbitmap_set(job->copy_bitmap, offset, bytes); =20 - hbitmap_set(job->copy_bitmap, cluster, end_cluster - cluster); - - offset =3D end_cluster * job->cluster_size; + offset +=3D bytes; if (offset >=3D job->len) { break; } @@ -423,30 +423,27 @@ static void backup_incremental_init_copy_bitmap(Backu= pBlockJob *job) =20 /* TODO job_progress_set_remaining() would make more sense */ job_progress_update(&job->common.job, - job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); + job->len - hbitmap_count(job->copy_bitmap)); } =20 static int coroutine_fn backup_run(Job *job, Error **errp) { BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); BlockDriverState *bs =3D blk_bs(s->common.blk); - int64_t offset, nb_clusters; + int64_t offset; int ret =3D 0; =20 QLIST_INIT(&s->inflight_reqs); qemu_co_rwlock_init(&s->flush_rwlock); =20 - nb_clusters =3D DIV_ROUND_UP(s->len, s->cluster_size); job_progress_set_remaining(job, s->len); =20 - s->copy_bitmap =3D hbitmap_alloc(nb_clusters, 0); if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { backup_incremental_init_copy_bitmap(s); } else { - hbitmap_set(s->copy_bitmap, 0, nb_clusters); + hbitmap_set(s->copy_bitmap, 0, s->len); } =20 - s->before_write.notify =3D backup_before_write_notify; bdrv_add_before_write_notifier(bs, &s->before_write); =20 @@ -527,7 +524,6 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) /* wait until pending backup_do_cow() calls have completed */ qemu_co_rwlock_wrlock(&s->flush_rwlock); qemu_co_rwlock_unlock(&s->flush_rwlock); - hbitmap_free(s->copy_bitmap); =20 return ret; } @@ -678,6 +674,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, } else { job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); } + + job->copy_bitmap =3D hbitmap_alloc(len, ctz32(job->cluster_size)); job->use_copy_range =3D true; job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), blk_get_max_transfer(job->target)); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979166; cv=none; d=zoho.com; s=zohoarc; b=Dop4sU0k87eIc0mTiO5UBzQ1ZoDT+vu2TvTdkakiucXVaZfpWYr1WYhHEg0hNd2akmjPbmDdTr4OGxPjxT1ClxKQL4hkQIKaK5Z+W4PJgytI29nqD+CS7BQOA26gh7lXssfYy6c6Iz3nKrkEN+ESCg4HGi1jY/Xhv3OVExj29wE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979166; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=YQL3Fs7mbC1hI7RGH+s268cqQ0zN70sYMGGdqy6CMaU=; b=FnQGBU9W0vae/hlQfMiSKsOpsvCwLIdfjat41fhuzW8p4aLkiXymBSEPLJBtnxm2DT1eEPMwAOsWtZZWSqQgwk5JjIBv3xXJKErGoTCec3xWyr8iaYFm1JPBlHiG8eZPu6cX8e6d7hUf2MZeA/w9jlN9Hi63fSTXckROyaIzp3E= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979166206112.84432914343586; Tue, 1 Oct 2019 18:19:26 -0700 (PDT) Received: from localhost ([::1]:50114 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTIa-0004zC-IV for importer@patchew.org; Tue, 01 Oct 2019 21:19:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40443) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tu-JN for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsL-0002Nw-5J for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:20116) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001pz-Bo; Tue, 01 Oct 2019 19:48:11 -0400 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlMwx193511; Tue, 1 Oct 2019 19:47:46 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsg8e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:46 -0400 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlO7L193565; Tue, 1 Oct 2019 19:47:46 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsg7x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:46 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkYJ031624; Tue, 1 Oct 2019 23:47:44 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fc9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:44 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NliIp47972612 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:44 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6C7F52805E; Tue, 1 Oct 2019 23:47:44 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 507472805A; Tue, 1 Oct 2019 23:47:44 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:44 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 60/97] block/backup: refactor and tolerate unallocated cluster skipping Date: Tue, 1 Oct 2019 18:45:39 -0500 Message-Id: <20191001234616.7825-61-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sementsov-Ogievskiy , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Split allocation checking to separate function and reduce nesting. Consider bdrv_is_allocated() fail as allocated area, as copying more than needed is not wrong (and we do it anyway) and seems better than fail the whole job. And, most probably we will fail on the next read, if there are real problem with source. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20190429090842.57910-4-vsementsov@virtuozzo.com Signed-off-by: Max Reitz (cherry picked from commit 9eb5a248f3e50c1f034bc6ff4b2f25c8c56515a5) *prereq for 110571be4e Signed-off-by: Michael Roth --- block/backup.c | 60 +++++++++++++++++++------------------------------- 1 file changed, 23 insertions(+), 37 deletions(-) diff --git a/block/backup.c b/block/backup.c index 510fc54f98..298e85f1a9 100644 --- a/block/backup.c +++ b/block/backup.c @@ -377,6 +377,22 @@ static bool coroutine_fn yield_and_check(BackupBlockJo= b *job) return false; } =20 +static bool bdrv_is_unallocated_range(BlockDriverState *bs, + int64_t offset, int64_t bytes) +{ + int64_t end =3D offset + bytes; + + while (offset < end && !bdrv_is_allocated(bs, offset, bytes, &bytes)) { + if (bytes =3D=3D 0) { + return true; + } + offset +=3D bytes; + bytes =3D end - offset; + } + + return offset >=3D end; +} + static int coroutine_fn backup_run_incremental(BackupBlockJob *job) { int ret; @@ -462,49 +478,19 @@ static int coroutine_fn backup_run(Job *job, Error **= errp) for (offset =3D 0; offset < s->len; offset +=3D s->cluster_size) { bool error_is_read; - int alloced =3D 0; =20 if (yield_and_check(s)) { break; } =20 - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP) { - int i; - int64_t n; - - /* Check to see if these blocks are already in the - * backing file. */ - - for (i =3D 0; i < s->cluster_size;) { - /* bdrv_is_allocated() only returns true/false based - * on the first set of sectors it comes across that - * are are all in the same state. - * For that reason we must verify each sector in the - * backup cluster length. We end up copying more than - * needed but at some point that is always the case. */ - alloced =3D - bdrv_is_allocated(bs, offset + i, - s->cluster_size - i, &n); - i +=3D n; - - if (alloced || n =3D=3D 0) { - break; - } - } - - /* If the above loop never found any sectors that are in - * the topmost image, skip this backup. */ - if (alloced =3D=3D 0) { - continue; - } - } - /* FULL sync mode we copy the whole drive. */ - if (alloced < 0) { - ret =3D alloced; - } else { - ret =3D backup_do_cow(s, offset, s->cluster_size, - &error_is_read, false); + if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP && + bdrv_is_unallocated_range(bs, offset, s->cluster_size)) + { + continue; } + + ret =3D backup_do_cow(s, offset, s->cluster_size, + &error_is_read, false); if (ret < 0) { /* Depending on error action, fail now or retry cluster */ BlockErrorAction action =3D --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979916; cv=none; d=zoho.com; s=zohoarc; b=SZsFVqJTAJgzVeK3ULCFXpRIV6vo2OyNAJu6NmrZgfExCgAMKKlyE3vPZBo6BoYPKlmCc7JRFxYp2bMDQJ2saooDxenws32BoKve+NAt2+XrgogVXlmRno84DKpweAuibRrnNYRoJZ5pZ7VyccSTqNyYOgBI9TTe2itiyYIXqU8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979916; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=AEO0byAPaCYaTzyvIkoI/SMrEOZaYuTnSX3Vi+hy2m4=; b=fZaw7mHPP+opFB8XHC3gPtHT4hKCy4ve+XPQrltGAp9YSlSCPYeFoTaNYdZX+qXtycG3mwcujOMEVwpn05dvNTnsfyjYwA/rHrcRdn5jLLBamKqvKvNzzVPM9r82dd6SIdjpkGoKmldQjuSDwxtSwowWmACmm3F9e6O8Ddk+ZeM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997991656197.91038265389227; Tue, 1 Oct 2019 18:31:56 -0700 (PDT) Received: from localhost ([::1]:50204 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTUc-0000YK-GB for importer@patchew.org; Tue, 01 Oct 2019 21:31:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41530) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsw-0002JN-OR for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRst-00038f-AR for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:48 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:14740 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsq-0002XJ-WC; Tue, 01 Oct 2019 19:48:46 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NmMYk132876; Tue, 1 Oct 2019 19:48:23 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4dec6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:23 -0400 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NmMLQ132902; Tue, 1 Oct 2019 19:48:22 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4de60-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:22 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlKs027161; Tue, 1 Oct 2019 23:47:46 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bhh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:46 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlj7E55443938 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:45 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 701E328059; Tue, 1 Oct 2019 23:47:45 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 546D528058; Tue, 1 Oct 2019 23:47:45 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:45 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 61/97] block/backup: unify different modes code path Date: Tue, 1 Oct 2019 18:45:40 -0500 Message-Id: <20191001234616.7825-62-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sementsov-Ogievskiy , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Do full, top and incremental mode copying all in one place. This unifies the code path and helps further improvements. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20190429090842.57910-5-vsementsov@virtuozzo.com Signed-off-by: Max Reitz (cherry picked from commit c334e897d08eea1f5a3a95f6a2208afe6757c103) *prereq for 110571be4e Signed-off-by: Michael Roth --- block/backup.c | 43 ++++++++++--------------------------------- 1 file changed, 10 insertions(+), 33 deletions(-) diff --git a/block/backup.c b/block/backup.c index 298e85f1a9..b54386b699 100644 --- a/block/backup.c +++ b/block/backup.c @@ -393,15 +393,23 @@ static bool bdrv_is_unallocated_range(BlockDriverStat= e *bs, return offset >=3D end; } =20 -static int coroutine_fn backup_run_incremental(BackupBlockJob *job) +static int coroutine_fn backup_loop(BackupBlockJob *job) { int ret; bool error_is_read; int64_t offset; HBitmapIter hbi; + BlockDriverState *bs =3D blk_bs(job->common.blk); =20 hbitmap_iter_init(&hbi, job->copy_bitmap, 0); while ((offset =3D hbitmap_iter_next(&hbi)) !=3D -1) { + if (job->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP && + bdrv_is_unallocated_range(bs, offset, job->cluster_size)) + { + hbitmap_reset(job->copy_bitmap, offset, job->cluster_size); + continue; + } + do { if (yield_and_check(job)) { return 0; @@ -446,7 +454,6 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) { BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); BlockDriverState *bs =3D blk_bs(s->common.blk); - int64_t offset; int ret =3D 0; =20 QLIST_INIT(&s->inflight_reqs); @@ -471,38 +478,8 @@ static int coroutine_fn backup_run(Job *job, Error **e= rrp) * notify callback service CoW requests. */ job_yield(job); } - } else if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { - ret =3D backup_run_incremental(s); } else { - /* Both FULL and TOP SYNC_MODE's require copying.. */ - for (offset =3D 0; offset < s->len; - offset +=3D s->cluster_size) { - bool error_is_read; - - if (yield_and_check(s)) { - break; - } - - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP && - bdrv_is_unallocated_range(bs, offset, s->cluster_size)) - { - continue; - } - - ret =3D backup_do_cow(s, offset, s->cluster_size, - &error_is_read, false); - if (ret < 0) { - /* Depending on error action, fail now or retry cluster */ - BlockErrorAction action =3D - backup_error_action(s, error_is_read, -ret); - if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { - break; - } else { - offset -=3D s->cluster_size; - continue; - } - } - } + ret =3D backup_loop(s); } =20 notifier_with_return_remove(&s->before_write); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977632; cv=none; d=zoho.com; s=zohoarc; b=hw9+GyBdGEkDIJbA7qE/Ldfql5W9uk5HTwlyJqq7Aaopxxmn3DweL3d2j397pGHyAwn25YgWFoq5/WGSsKyfepKmXOwW3mQSpGOyiq34By5lBIDj2443n8Qs6p2p6sr1QQeC8Z/BS8jXmIALea3orkjU+1D9JZIZjXXC/bT5zKA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977632; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=JXChs/vINX8oCz7hDx/1BMlITJOhxEnXM5V6Jf48mns=; b=Qda3GdLXmtBWJ+mk6OtrCCh4pCUdyvAW/ysWx7La4MKoAbuYj6XCkiv+E9I8la8z9hXX+tz4KLfvZ3h1Ac1EP6GqoK7Pmue+bFPQ2B7TJS9beRZEkBJq3alNWRrG9F7BbwAEmP6MSfb5aU62cz5Muzv+dWI2iIOuz2L4GzENEB8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977632821977.3747712932173; Tue, 1 Oct 2019 17:53:52 -0700 (PDT) Received: from localhost ([::1]:49892 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFStr-0006kv-9V for importer@patchew.org; Tue, 01 Oct 2019 20:53:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40413) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tK-B8 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsK-0002Mz-2d for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:33786 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001pR-An; Tue, 01 Oct 2019 19:48:11 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPqW122859; Tue, 1 Oct 2019 19:47:48 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgk409bd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:48 -0400 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NllFl123402; Tue, 1 Oct 2019 19:47:47 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgk409b5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:47 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlZ1029380; Tue, 1 Oct 2019 23:47:46 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fajv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:46 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlkbF41746912 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:46 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EC6512805A; Tue, 1 Oct 2019 23:47:45 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D0F2928058; Tue, 1 Oct 2019 23:47:45 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:45 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 62/97] block/backup: refactor: split out backup_calculate_cluster_size Date: Tue, 1 Oct 2019 18:45:41 -0500 Message-Id: <20191001234616.7825-63-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sementsov-Ogievskiy , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Split out cluster_size calculation. Move copy-bitmap creation above block-job creation, as we are going to share it with upcoming backup-top filter, which also should be created before actual block job creation. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190429090842.57910-6-vsementsov@virtuozzo.com [mreitz: Dropped a paragraph from the commit message that was left over from a previous version] Signed-off-by: Max Reitz (cherry picked from commit ae6b12fa4cf7d54add35531c790aaf2bd6d833f3) *prereq for 110571be4e Signed-off-by: Michael Roth --- block/backup.c | 82 ++++++++++++++++++++++++++++++++------------------ 1 file changed, 52 insertions(+), 30 deletions(-) diff --git a/block/backup.c b/block/backup.c index b54386b699..d1b94a6dbe 100644 --- a/block/backup.c +++ b/block/backup.c @@ -507,6 +507,42 @@ static const BlockJobDriver backup_job_driver =3D { .drain =3D backup_drain, }; =20 +static int64_t backup_calculate_cluster_size(BlockDriverState *target, + Error **errp) +{ + int ret; + BlockDriverInfo bdi; + + /* + * If there is no backing file on the target, we cannot rely on COW if= our + * backup cluster size is smaller than the target cluster size. Even f= or + * targets with a backing file, try to avoid COW if possible. + */ + ret =3D bdrv_get_info(target, &bdi); + if (ret =3D=3D -ENOTSUP && !target->backing) { + /* Cluster size is not defined */ + warn_report("The target block device doesn't provide " + "information about the block size and it doesn't have = a " + "backing file. The default block size of %u bytes is " + "used. If the actual block size of the target exceeds " + "this default, the backup may be unusable", + BACKUP_CLUSTER_SIZE_DEFAULT); + return BACKUP_CLUSTER_SIZE_DEFAULT; + } else if (ret < 0 && !target->backing) { + error_setg_errno(errp, -ret, + "Couldn't determine the cluster size of the target image, " + "which has no backing file"); + error_append_hint(errp, + "Aborting, since this may create an unusable destination image= \n"); + return ret; + } else if (ret < 0 && target->backing) { + /* Not fatal; just trudge on ahead. */ + return BACKUP_CLUSTER_SIZE_DEFAULT; + } + + return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size); +} + BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, @@ -518,9 +554,10 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, JobTxn *txn, Error **errp) { int64_t len; - BlockDriverInfo bdi; BackupBlockJob *job =3D NULL; int ret; + int64_t cluster_size; + HBitmap *copy_bitmap =3D NULL; =20 assert(bs); assert(target); @@ -582,6 +619,13 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, goto error; } =20 + cluster_size =3D backup_calculate_cluster_size(target, errp); + if (cluster_size < 0) { + goto error; + } + + copy_bitmap =3D hbitmap_alloc(len, ctz32(cluster_size)); + /* job->len is fixed, so we can't allow resize */ job =3D block_job_create(job_id, &backup_job_driver, txn, bs, BLK_PERM_CONSISTENT_READ, @@ -610,35 +654,9 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, =20 /* Detect image-fleecing (and similar) schemes */ job->serialize_target_writes =3D bdrv_chain_contains(target, bs); - - /* If there is no backing file on the target, we cannot rely on COW if= our - * backup cluster size is smaller than the target cluster size. Even f= or - * targets with a backing file, try to avoid COW if possible. */ - ret =3D bdrv_get_info(target, &bdi); - if (ret =3D=3D -ENOTSUP && !target->backing) { - /* Cluster size is not defined */ - warn_report("The target block device doesn't provide " - "information about the block size and it doesn't have = a " - "backing file. The default block size of %u bytes is " - "used. If the actual block size of the target exceeds " - "this default, the backup may be unusable", - BACKUP_CLUSTER_SIZE_DEFAULT); - job->cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; - } else if (ret < 0 && !target->backing) { - error_setg_errno(errp, -ret, - "Couldn't determine the cluster size of the target image, " - "which has no backing file"); - error_append_hint(errp, - "Aborting, since this may create an unusable destination image= \n"); - goto error; - } else if (ret < 0 && target->backing) { - /* Not fatal; just trudge on ahead. */ - job->cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; - } else { - job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); - } - - job->copy_bitmap =3D hbitmap_alloc(len, ctz32(job->cluster_size)); + job->cluster_size =3D cluster_size; + job->copy_bitmap =3D copy_bitmap; + copy_bitmap =3D NULL; job->use_copy_range =3D true; job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), blk_get_max_transfer(job->target)); @@ -654,6 +672,10 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, return &job->common; =20 error: + if (copy_bitmap) { + assert(!job || !job->copy_bitmap); + hbitmap_free(copy_bitmap); + } if (sync_bitmap) { bdrv_reclaim_dirty_bitmap(bs, sync_bitmap, NULL); } --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569982082; cv=none; d=zoho.com; s=zohoarc; b=I4GTRhb/p5HV88MzmItfuWenB8qbLDhedHJaH/oCKKnS1aJXEjwZ3YwJrlz9hAyJz21cb4vynRg/LxBPoSMvyvslBPfslwJM5vaiY1SSTWl/+KEYa2kbAGpujxppH8+ls5X3kr+jAz/W2s2kcJjGSlA054o98zND0Kk6pv3jqII= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569982082; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=gVzpyZbRPd6WF5nongSuJhNEcZiVq5GuRyGllBBIwz0=; b=SPUQmL5SKabXXlDLcAu6/EfyodWRvCVnFj6xiJPQkaY8pqyNLuDyN9OCHRoeNM6Me2eMet7ocvv10awjE98Gvn27vCu/ymzWccYNokKy1TBfrt9UCdZBSaYemR3+0ruPPYAaoE46fhwnd61I+dIkhtZt6YMdJbOlOyNm3T4HL7w= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569982082052971.5567228209884; Tue, 1 Oct 2019 19:08:02 -0700 (PDT) Received: from localhost ([::1]:50570 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFU3b-00050x-2j for importer@patchew.org; Tue, 01 Oct 2019 22:08:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41630) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsz-0002MH-3r for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsx-0003Ci-3N for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:52 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:32100 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsv-0002fZ-5A; Tue, 01 Oct 2019 19:48:50 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NmMfp132829; Tue, 1 Oct 2019 19:48:27 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4ded0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:26 -0400 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NmQaS133438; Tue, 1 Oct 2019 19:48:26 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4de64-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:25 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkEN027150; Tue, 1 Oct 2019 23:47:47 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bhn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:47 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlkwx41746920 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:46 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 733F428058; Tue, 1 Oct 2019 23:47:46 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 57AD52805A; Tue, 1 Oct 2019 23:47:46 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:46 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 63/97] backup: Copy only dirty areas Date: Tue, 1 Oct 2019 18:45:42 -0500 Message-Id: <20191001234616.7825-64-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz The backup job must only copy areas that the copy_bitmap reports as dirty. This is always the case when using traditional non-offloading backup, because it copies each cluster separately. When offloading the copy operation, we sometimes copy more than one cluster at a time, but we only check whether the first one is dirty. Therefore, whenever copy offloading is possible, the backup job currently produces wrong output when the guest writes to an area of which an inner part has already been backed up, because that inner part will be re-copied. Fixes: 9ded4a0114968e98b41494fc035ba14f84cdf700 Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20190801173900.23851-2-mreitz@redhat.com Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz (cherry picked from commit 4a5b91ca024fc6fd87021c54655af76a35f2ef1e) Signed-off-by: Michael Roth --- block/backup.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/block/backup.c b/block/backup.c index d1b94a6dbe..f67c208cf0 100644 --- a/block/backup.c +++ b/block/backup.c @@ -204,22 +204,31 @@ static int coroutine_fn backup_do_cow(BackupBlockJob = *job, cow_request_begin(&cow_request, job, start, end); =20 while (start < end) { + int64_t dirty_end; + if (!hbitmap_get(job->copy_bitmap, start)) { trace_backup_do_cow_skip(job, start); start +=3D job->cluster_size; continue; /* already copied */ } =20 + dirty_end =3D hbitmap_next_zero(job->copy_bitmap, start, (end - st= art)); + if (dirty_end < 0) { + dirty_end =3D end; + } + trace_backup_do_cow_process(job, start); =20 if (job->use_copy_range) { - ret =3D backup_cow_with_offload(job, start, end, is_write_noti= fier); + ret =3D backup_cow_with_offload(job, start, dirty_end, + is_write_notifier); if (ret < 0) { job->use_copy_range =3D false; } } if (!job->use_copy_range) { - ret =3D backup_cow_with_bounce_buffer(job, start, end, is_writ= e_notifier, + ret =3D backup_cow_with_bounce_buffer(job, start, dirty_end, + is_write_notifier, error_is_read, &bounce_buf= fer); } if (ret < 0) { --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978028; cv=none; d=zoho.com; s=zohoarc; b=IPW7ocvdp0QIx2rNUZiHB0v/McIL703V09YymtKhZGFa1hbk53TiK7w6ei/JVAUuXmxDsAUsbgN73yIC3wovoxbrM9slPgzPqT/en+9iyyyA9wiLs2pd/N17OHE4qWPu5BeW8qHDuIhMhbmkoAQZMIuxxnn3+nnSeCr3YDF7gzM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978028; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=nvC5UwpvMku2puCyYowWnl8EiQFT81KMf7F8WQ+/7cE=; b=iIkHYEdRDLaUGnKkJKg/ljltM75I9sHRGdevzoEcQVqIFoJbltD1q2D+U2K3/dU9yqVD+LOhLEhz8EtGs1rUGFYKFqoEXsoseODPB3Z7AbzbSnwMl1njsRvO9NI2LreKZ4+EPIeexhlJahHoxEoDnKDerq1Ru4eJwUETcEmGqXM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978028180525.8845496919831; Tue, 1 Oct 2019 18:00:28 -0700 (PDT) Received: from localhost ([::1]:49948 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT0C-0004Cz-6m for importer@patchew.org; Tue, 01 Oct 2019 21:00:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40420) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tS-AJ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsL-0002No-49 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:29302 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001pc-DT; Tue, 01 Oct 2019 19:48:12 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOtP146257; Tue, 1 Oct 2019 19:47:48 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26j4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:48 -0400 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlamr146538; Tue, 1 Oct 2019 19:47:48 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26hw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:47 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlPF031629; Tue, 1 Oct 2019 23:47:47 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fcj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:47 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NllEk47382866 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:47 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F1BDD2805A; Tue, 1 Oct 2019 23:47:46 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D584128058; Tue, 1 Oct 2019 23:47:46 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:46 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 64/97] iotests: Test backup job with two guest writes Date: Tue, 1 Oct 2019 18:45:43 -0500 Message-Id: <20191001234616.7825-65-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Perform two guest writes to not yet backed up areas of an image, where the former touches an inner area of the latter. Before HEAD^, copy offloading broke this in two ways: (1) The target image differs from the reference image (what the source was when the backup started). (2) But you will not see that in the failing output, because the job offset is reported as being greater than the job length. This is because one cluster is copied twice, and thus accounted for twice, but of course the job length does not increase. Signed-off-by: Max Reitz Message-id: 20190801173900.23851-3-mreitz@redhat.com Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz (cherry picked from commit 5f594a2e99f19ca0f7744d333bcd556f5976b78f) Signed-off-by: Michael Roth --- tests/qemu-iotests/056 | 39 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/056.out | 4 ++-- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index 3df323984d..e5ac25127b 100755 --- a/tests/qemu-iotests/056 +++ b/tests/qemu-iotests/056 @@ -133,6 +133,7 @@ class BackupTest(iotests.QMPTestCase): self.vm =3D iotests.VM() self.test_img =3D img_create('test') self.dest_img =3D img_create('dest') + self.ref_img =3D img_create('ref') self.vm.add_drive(self.test_img) self.vm.launch() =20 @@ -140,6 +141,7 @@ class BackupTest(iotests.QMPTestCase): self.vm.shutdown() try_remove(self.test_img) try_remove(self.dest_img) + try_remove(self.ref_img) =20 def hmp_io_writes(self, drive, patterns): for pattern in patterns: @@ -177,6 +179,43 @@ class BackupTest(iotests.QMPTestCase): self.assert_qmp(event, 'data/error', qerror) return False =20 + def test_overlapping_writes(self): + # Write something to back up + self.hmp_io_writes('drive0', [('42', '0M', '2M')]) + + # Create a reference backup + self.qmp_backup_and_wait(device=3D'drive0', format=3Diotests.imgfm= t, + sync=3D'full', target=3Dself.ref_img, + auto_dismiss=3DFalse) + res =3D self.vm.qmp('block-job-dismiss', id=3D'drive0') + self.assert_qmp(res, 'return', {}) + + # Now to the test backup: We simulate the following guest + # writes: + # (1) [1M + 64k, 1M + 128k): Afterwards, everything in that + # area should be in the target image, and we must not copy + # it again (because the source image has changed now) + # (64k is the job's cluster size) + # (2) [1M, 2M): The backup job must not get overeager. It + # must copy [1M, 1M + 64k) and [1M + 128k, 2M) separately, + # but not the area in between. + + self.qmp_backup(device=3D'drive0', format=3Diotests.imgfmt, sync= =3D'full', + target=3Dself.dest_img, speed=3D1, auto_dismiss=3D= False) + + self.hmp_io_writes('drive0', [('23', '%ik' % (1024 + 64), '64k'), + ('66', '1M', '1M')]) + + # Let the job complete + res =3D self.vm.qmp('block-job-set-speed', device=3D'drive0', spee= d=3D0) + self.assert_qmp(res, 'return', {}) + self.qmp_backup_wait('drive0') + res =3D self.vm.qmp('block-job-dismiss', id=3D'drive0') + self.assert_qmp(res, 'return', {}) + + self.assertTrue(iotests.compare_images(self.ref_img, self.dest_img= ), + 'target image does not match reference image') + def test_dismiss_false(self): res =3D self.vm.qmp('query-block-jobs') self.assert_qmp(res, 'return', []) diff --git a/tests/qemu-iotests/056.out b/tests/qemu-iotests/056.out index dae404e278..36376bed87 100644 --- a/tests/qemu-iotests/056.out +++ b/tests/qemu-iotests/056.out @@ -1,5 +1,5 @@ -......... +.......... ---------------------------------------------------------------------- -Ran 9 tests +Ran 10 tests =20 OK --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977218; cv=none; d=zoho.com; s=zohoarc; b=dbt+hSylyMZ8lFBzByW2kQBuNSiKw5OixFaSFSAVyZsKPIVORcRDvfa1dXPhqQW0WFpj0fG6+wB+9yYpaGp2KGNCk76z8lqDKuVqzNE90XtSCGGbxHY3pQOv4ASe/5Lh61HKU8nyKg4VxTpU3WNcWfzLc0WYOvn4e3d1NIwSSko= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977218; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=tfKdeadiHCsMAXP5DBk7VRvSJSuB48ykD4i/i5hNR/Q=; b=RFKMO642Szgw7PdWgfpDyJMmHm2zAXR1pnc+o3oCHlVDmLhb6Ij4gVUN9q8M+el+ixhAmGJFjtHiZW+bQw9JHIyImC510Bh8JN3M4XdQxWJsWP8HJgexVeH0Gic4T1VEmjFVUm694DXNH3wwjsi5Po18EQx0Vc2dKnxGNMPobzo= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977218098470.35932363062204; Tue, 1 Oct 2019 17:46:58 -0700 (PDT) Received: from localhost ([::1]:49842 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSnA-0008Of-GK for importer@patchew.org; Tue, 01 Oct 2019 20:46:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40421) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tT-BD for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsL-0002Nf-6O for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:39154 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001qq-DU; Tue, 01 Oct 2019 19:48:11 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlRrW146346; Tue, 1 Oct 2019 19:47:49 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26ja-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:48 -0400 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlmed146835; Tue, 1 Oct 2019 19:47:48 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26j5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:48 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjmlR023578; Tue, 1 Oct 2019 23:47:48 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577h0b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:47 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NllO449742276 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:47 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 786E42805C; Tue, 1 Oct 2019 23:47:47 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5D5C528059; Tue, 1 Oct 2019 23:47:47 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:47 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 65/97] util/hbitmap: update orig_size on truncate Date: Tue, 1 Oct 2019 18:45:44 -0500 Message-Id: <20191001234616.7825-66-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=843 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sementsov-Ogievskiy , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Without this, hbitmap_next_zero and hbitmap_next_dirty_area are broken after truncate. So, orig_size is broken since it's introduction in 76d570dc495c56bb. Fixes: 76d570dc495c56bb Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190805120120.23585-1-vsementsov@virtuozzo.com Reviewed-by: Max Reitz Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz (cherry picked from commit 4e4de222799634d8159ee7b9303b9e1b45c6be2c) Signed-off-by: Michael Roth --- util/hbitmap.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/util/hbitmap.c b/util/hbitmap.c index 7905212a8b..bcc0acdc6a 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -53,7 +53,9 @@ */ =20 struct HBitmap { - /* Size of the bitmap, as requested in hbitmap_alloc. */ + /* + * Size of the bitmap, as requested in hbitmap_alloc or in hbitmap_tru= ncate. + */ uint64_t orig_size; =20 /* Number of total bits in the bottom level. */ @@ -732,6 +734,8 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) uint64_t num_elements =3D size; uint64_t old; =20 + hb->orig_size =3D size; + /* Size comes in as logical elements, adjust for granularity. */ size =3D (size + (1ULL << hb->granularity) - 1) >> hb->granularity; assert(size <=3D ((uint64_t)1 << HBITMAP_LOG_MAX_SIZE)); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977463; cv=none; d=zoho.com; s=zohoarc; b=LLxe+bjd8Rd18PpInvOTC6J4qOHXV+PHkqgYrvrZkyIm9a5ZgECtV7Uj63Au6EaUtu+N4nA2218qSX/FnkFqVpwqC4LDpjnZxi8RMhEzo01XP/SfqPxsLuRzOF7OT+NbxhiYY8bKU/k0KOPtXBABsvgSXJwTZnZt59xd5keu/Cw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977463; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=9BZJF+jn9bvF5mCLQW/RDJuKW6ZVxqUUG/FF2Rs+JFo=; b=lvKl6oYqw5vOR3t2EXLcAQ9KDZdY0zvrBEnKJ4y10j+PSO/tUkmOqoT1Galka92YwSOUHGRBQxsCHO5tK0uRH+aUQj5sTZUEgTrS6yFuZGDQT3UTnTZAXfE3o+8wnIUvpfR55TqBiPXuFyMtk9uonzxoiv3x7mhK/YPmlBocB2U= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977463823768.9247280308387; Tue, 1 Oct 2019 17:51:03 -0700 (PDT) Received: from localhost ([::1]:49870 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSr5-0003uO-E2 for importer@patchew.org; Tue, 01 Oct 2019 20:50:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40417) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tQ-AQ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsL-0002O7-5W for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:56160 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001rk-Ba; Tue, 01 Oct 2019 19:48:11 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOgJ098927; Tue, 1 Oct 2019 19:47:49 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v2y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:49 -0400 Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlnhw099831; Tue, 1 Oct 2019 19:47:49 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v2q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:49 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlrD001420; Tue, 1 Oct 2019 23:47:48 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma03wdc.us.ibm.com with ESMTP id 2v9y56yf88-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:48 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlmTC11076576 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:48 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0256C2805A; Tue, 1 Oct 2019 23:47:48 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DA63228059; Tue, 1 Oct 2019 23:47:47 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:47 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 66/97] iotests: Test incremental backup after truncation Date: Tue, 1 Oct 2019 18:45:45 -0500 Message-Id: <20191001234616.7825-67-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=838 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Message-id: 20190805152840.32190-1-mreitz@redhat.com Signed-off-by: Max Reitz (cherry picked from commit 8a9cb864086269af14bbd13f395472703cf99f8c) Signed-off-by: Michael Roth --- tests/qemu-iotests/124 | 38 ++++++++++++++++++++++++++++++++++---- tests/qemu-iotests/124.out | 4 ++-- 2 files changed, 36 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 80b356f7bb..3440f54781 100755 --- a/tests/qemu-iotests/124 +++ b/tests/qemu-iotests/124 @@ -212,25 +212,28 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): return bitmap =20 =20 - def prepare_backup(self, bitmap=3DNone, parent=3DNone): + def prepare_backup(self, bitmap=3DNone, parent=3DNone, **kwargs): if bitmap is None: bitmap =3D self.bitmaps[-1] if parent is None: parent, _ =3D bitmap.last_target() =20 target, _ =3D bitmap.new_target() - self.img_create(target, bitmap.drive['fmt'], parent=3Dparent) + self.img_create(target, bitmap.drive['fmt'], parent=3Dparent, + **kwargs) return target =20 =20 def create_incremental(self, bitmap=3DNone, parent=3DNone, - parentFormat=3DNone, validate=3DTrue): + parentFormat=3DNone, validate=3DTrue, + target=3DNone): if bitmap is None: bitmap =3D self.bitmaps[-1] if parent is None: parent, _ =3D bitmap.last_target() =20 - target =3D self.prepare_backup(bitmap, parent) + if target is None: + target =3D self.prepare_backup(bitmap, parent) res =3D self.do_qmp_backup(job_id=3Dbitmap.drive['id'], device=3Dbitmap.drive['id'], sync=3D'incremental', bitmap=3Dbitmap.nam= e, @@ -572,6 +575,33 @@ class TestIncrementalBackup(TestIncrementalBackupBase): 'bitmap0', self.drives[0], granularity=3D64000) =20 + def test_growing_before_backup(self): + ''' + Test: Add a bitmap, truncate the image, write past the old + end, do a backup. + + Incremental backup should not ignore dirty bits past the old + image end. + ''' + self.assert_no_active_block_jobs() + + self.create_anchor_backup() + + self.add_bitmap('bitmap0', self.drives[0]) + + res =3D self.vm.qmp('block_resize', device=3Dself.drives[0]['id'], + size=3D(65 * 1048576)) + self.assert_qmp(res, 'return', {}) + + # Dirty the image past the old end + self.vm.hmp_qemu_io(self.drives[0]['id'], 'write 64M 64k') + + target =3D self.prepare_backup(size=3D'65M') + self.create_incremental(target=3Dtarget) + + self.vm.shutdown() + self.check_backups() + =20 class TestIncrementalBackupBlkdebug(TestIncrementalBackupBase): '''Incremental backup tests that utilize a BlkDebug filter on drive0.'= '' diff --git a/tests/qemu-iotests/124.out b/tests/qemu-iotests/124.out index 281b69efea..fa16b5ccef 100644 --- a/tests/qemu-iotests/124.out +++ b/tests/qemu-iotests/124.out @@ -1,5 +1,5 @@ -............ +............. ---------------------------------------------------------------------- -Ran 12 tests +Ran 13 tests =20 OK --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978860; cv=none; d=zoho.com; s=zohoarc; b=YO/3wDsce156qh2W9ViL3RhQl+F2fpO8obvi4in+Xp0cNx4JYw6TkeVCcTKsyUwan1MROYTV7sr2IvKtx8qzVeOI2uioah2XEiFGJez/kRLBJGcSG1Gg4Q9vJT1+1U8v7OR9KwGjW9XdGzD/7p1vjEwhB2l+B0UZJgSbiFy6AiU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978860; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=14bHccR0Kmt+YQZYDfljzeM768CZJ0wcd60RwEZeXtg=; b=YwhHcBA6D2+AntbrQQvOxY+3RdmbiUPNLL+kjocv+HE0Vd1RDDenmlZSkZiMCyfSlMmberitsRGOV+TCKe+yMiy3nUMKUaOb61UU/E2ST1aVpOraYImaVDJWKLzrG48kSq91CHmQa+CAlEekww3kYVV0jcEGrVyWmPzmZNyKtbM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156997886029253.84858339584139; Tue, 1 Oct 2019 18:14:20 -0700 (PDT) Received: from localhost ([::1]:50080 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTDd-0000cE-TO for importer@patchew.org; Tue, 01 Oct 2019 21:14:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40419) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tR-BR for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsL-0002O2-5R for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:34234) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsJ-0001tL-FO; Tue, 01 Oct 2019 19:48:12 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPuK053509; Tue, 1 Oct 2019 19:47:50 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716bk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:50 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nla2J053831; Tue, 1 Oct 2019 19:47:50 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716b7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:50 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njofd032175; Tue, 1 Oct 2019 23:47:50 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma01wdc.us.ibm.com with ESMTP id 2v9y57fgc4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:50 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlmlR40173988 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:48 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 82EDD28059; Tue, 1 Oct 2019 23:47:48 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 66A4C2805A; Tue, 1 Oct 2019 23:47:48 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:48 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 67/97] mirror: Only mirror granularity-aligned chunks Date: Tue, 1 Oct 2019 18:45:46 -0500 Message-Id: <20191001234616.7825-68-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz In write-blocking mode, all writes to the top node directly go to the target. We must only mirror chunks of data that are aligned to the job's granularity, because that is how the dirty bitmap works. Therefore, the request alignment for writes must be the job's granularity (in write-blocking mode). Unfortunately, this forces all reads and writes to have the same granularity (we only need this alignment for writes to the target, not the source), but that is something to be fixed another time. Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20190805153308.2657-1-mreitz@redhat.com Reviewed-by: Vladimir Sementsov-Ogievskiy Fixes: d06107ade0ce74dc39739bac80de84b51ec18546 Signed-off-by: Max Reitz (cherry picked from commit 9adc1cb49af8d4e54f57980b1eed5c0a4b2dafa6) Signed-off-by: Michael Roth --- block/mirror.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index ff15cfb197..062dc42867 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1477,6 +1477,15 @@ static void bdrv_mirror_top_child_perm(BlockDriverSt= ate *bs, BdrvChild *c, *nshared =3D BLK_PERM_ALL; } =20 +static void bdrv_mirror_top_refresh_limits(BlockDriverState *bs, Error **e= rrp) +{ + MirrorBDSOpaque *s =3D bs->opaque; + + if (s && s->job && s->job->copy_mode =3D=3D MIRROR_COPY_MODE_WRITE_BLO= CKING) { + bs->bl.request_alignment =3D s->job->granularity; + } +} + /* Dummy node that provides consistent read to its users without requiring= it * from its backing file and that allows writes on the backing file chain.= */ static BlockDriver bdrv_mirror_top =3D { @@ -1489,6 +1498,7 @@ static BlockDriver bdrv_mirror_top =3D { .bdrv_co_block_status =3D bdrv_co_block_status_from_backing, .bdrv_refresh_filename =3D bdrv_mirror_top_refresh_filename, .bdrv_child_perm =3D bdrv_mirror_top_child_perm, + .bdrv_refresh_limits =3D bdrv_mirror_top_refresh_limits, }; =20 static void mirror_start_job(const char *job_id, BlockDriverState *bs, @@ -1627,6 +1637,25 @@ static void mirror_start_job(const char *job_id, Blo= ckDriverState *bs, s->should_complete =3D true; } =20 + /* + * Must be called before we start tracking writes, but after + * + * ((MirrorBlockJob *) + * ((MirrorBDSOpaque *) + * mirror_top_bs->opaque + * )->job + * )->copy_mode + * + * has the correct value. + * (We start tracking writes as of the following + * bdrv_create_dirty_bitmap() call.) + */ + bdrv_refresh_limits(mirror_top_bs, &local_err); + if (local_err) { + error_propagate(errp, local_err); + goto fail; + } + s->dirty_bitmap =3D bdrv_create_dirty_bitmap(bs, granularity, NULL, er= rp); if (!s->dirty_bitmap) { goto fail; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978581; cv=none; d=zoho.com; s=zohoarc; b=dh3YuP/N7w/ko8SeYHUy3ZwzsekxvHBTCjS44I3WBwjCXeLEY3Al0XM7QVf/5jV2uWFFADVP/YMSwDvyb+Cx4qnRXj4bnDmLQGRE/xWO3LGbpzJyaEOFwk0fjSviD9N9k4t65QIxYKpBGx1fuNI+rfJya4xDm8c+D0SpqgnET+s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978581; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=qkIW/t5Du9sNW6RFi/jDPcTwtDMmBEL//xzQy0G4w3k=; b=cnAgOv90aK95XZukxVvdJyVEJZrjQAw9HD6ACNh9hwFbJEiFe0typFxtUYb/hOueT2xdUHmYVFtc4g5oL6VXyFaKEzoEd9EUnx7VcJQwbjf7gng4Q+4c9QBOLNnxL1C4mHcqf5J5aObyiOEBy3zSnd2TzrrjMHvuGRx7Kaj9vXw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978581918508.62052655254274; Tue, 1 Oct 2019 18:09:41 -0700 (PDT) Received: from localhost ([::1]:50036 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT9A-0004SL-4Z for importer@patchew.org; Tue, 01 Oct 2019 21:09:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40424) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001tW-Do for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsL-0002PB-MM for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:33204) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsK-0001uH-7Z; Tue, 01 Oct 2019 19:48:13 -0400 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlRGL086518; Tue, 1 Oct 2019 19:47:51 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat8108-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:51 -0400 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nloh2087187; Tue, 1 Oct 2019 19:47:50 -0400 Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat80yx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:50 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlTH022434; Tue, 1 Oct 2019 23:47:49 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma04dal.us.ibm.com with ESMTP id 2v9y57yd7a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:49 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlnOD35520934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:49 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0CF6528058; Tue, 1 Oct 2019 23:47:49 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E501A2805A; Tue, 1 Oct 2019 23:47:48 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:48 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 68/97] iotests: Test unaligned blocking mirror write Date: Tue, 1 Oct 2019 18:45:47 -0500 Message-Id: <20191001234616.7825-69-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Message-id: 20190805113526.20319-1-mreitz@redhat.com Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz (cherry picked from commit 19ba4651fe2d17cc49adae29acbb4a8cc29db8d1) Signed-off-by: Michael Roth --- tests/qemu-iotests/151 | 25 +++++++++++++++++++++++++ tests/qemu-iotests/151.out | 4 ++-- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151 index 1bb74d67c4..ad7359fc8d 100755 --- a/tests/qemu-iotests/151 +++ b/tests/qemu-iotests/151 @@ -114,6 +114,31 @@ class TestActiveMirror(iotests.QMPTestCase): def testActiveIOFlushed(self): self.doActiveIO(True) =20 + def testUnalignedActiveIO(self): + # Fill the source image + result =3D self.vm.hmp_qemu_io('source', 'write -P 1 0 2M') + + # Start the block job (very slowly) + result =3D self.vm.qmp('blockdev-mirror', + job_id=3D'mirror', + filter_node_name=3D'mirror-node', + device=3D'source-node', + target=3D'target-node', + sync=3D'full', + copy_mode=3D'write-blocking', + buf_size=3D(1048576 // 4), + speed=3D1) + self.assert_qmp(result, 'return', {}) + + # Start an unaligned request to a dirty area + result =3D self.vm.hmp_qemu_io('source', 'write -P 2 %i 1' % (1048= 576 + 42)) + + # Let the job finish + result =3D self.vm.qmp('block-job-set-speed', device=3D'mirror', s= peed=3D0) + self.assert_qmp(result, 'return', {}) + self.complete_and_wait(drive=3D'mirror') + + self.potential_writes_in_flight =3D False =20 =20 if __name__ =3D=3D '__main__': diff --git a/tests/qemu-iotests/151.out b/tests/qemu-iotests/151.out index fbc63e62f8..8d7e996700 100644 --- a/tests/qemu-iotests/151.out +++ b/tests/qemu-iotests/151.out @@ -1,5 +1,5 @@ -.. +... ---------------------------------------------------------------------- -Ran 2 tests +Ran 3 tests =20 OK --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978569; cv=none; d=zoho.com; s=zohoarc; b=hmWWbOvKUwxp+Tw1QkefpHse68ut79c3QtRSssVFBYaMevDMVNkISTY61tX02P9QoEys+c1rRVFdfPyFAqKjcxxY/MyPHt7wJcCMbHogwclOW9+nIPfqxpT9mEoDWhg3ZUWuX3IOgK3agXn9dkokZJu6V79Ojch13/fQioEq7uw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978569; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=iS+GT9pZ+F7kVKfRvtKHpGMASq6D2oG3w4ElRc/DCSE=; b=hvPbiDCyi7nPy2ELygCkLDJL0mOZNPGF6Fe/AP8k6IepaHIr6NScQ6iHONMgeHbBLrX7cncB0N30yXDqzLakNjm3WcY8KDCkOEw/iUlhH7RyNHkkaSbjeq4mw85lnjGBt5GVyhJBPnWE05S806gyX9Jf7JoVVvevSlZ2oJNGTww= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978569788980.439078338943; Tue, 1 Oct 2019 18:09:29 -0700 (PDT) Received: from localhost ([::1]:50034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT8x-0004C2-Ue for importer@patchew.org; Tue, 01 Oct 2019 21:09:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40528) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsY-0001wd-9I for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002YB-9s for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:25 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:2488) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsM-0001xf-Dv; Tue, 01 Oct 2019 19:48:17 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPbZ088119; Tue, 1 Oct 2019 19:47:53 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6c28-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:53 -0400 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlnTS089206; Tue, 1 Oct 2019 19:47:51 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6c1u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:51 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlJd006377; Tue, 1 Oct 2019 23:47:50 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye90-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:50 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlnjQ49742088 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:49 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7E6282805E; Tue, 1 Oct 2019 23:47:49 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6C5592805C; Tue, 1 Oct 2019 23:47:49 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:49 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 69/97] block/backup: disable copy_range for compressed backup Date: Tue, 1 Oct 2019 18:45:48 -0500 Message-Id: <20191001234616.7825-70-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Sementsov-Ogievskiy , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Enabled by default copy_range ignores compress option. It's definitely unexpected for user. It's broken since introduction of copy_range usage in backup in 9ded4a011496. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190730163251.755248-3-vsementsov@virtuozzo.com Reviewed-by: John Snow Reviewed-by: Max Reitz Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz (cherry picked from commit 110571be4e70ac015628e76d2731f96dd8d1998c) Signed-off-by: Michael Roth --- block/backup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/backup.c b/block/backup.c index f67c208cf0..381659d5ef 100644 --- a/block/backup.c +++ b/block/backup.c @@ -666,7 +666,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, job->cluster_size =3D cluster_size; job->copy_bitmap =3D copy_bitmap; copy_bitmap =3D NULL; - job->use_copy_range =3D true; + job->use_copy_range =3D !compress; /* compression isn't supported for = it */ job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), blk_get_max_transfer(job->target)); job->copy_range_size =3D MAX(job->cluster_size, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978313; cv=none; d=zoho.com; s=zohoarc; b=GO7KbRj+7LfSWdNjuXBhOsZ2oSjAM7OCv8zxFgFCwp4aVDMGoN4hyCnvTb+F5ZQ/R5yW/dKVZpQ/pJKTQxMnzWhaMYTPLukQWzxJNYt1HKV/u8Uhuixa3vnvFTAwfeARmrixJhTkxym6t4eZ6D0HI2x+ti7gefvKMCNrKWdQf00= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978313; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=rQOL05npL036jCwf+qDsDhqvezEJzevcF13BUKcVgMI=; b=UkU6C9nzpHp8MfwbD/ejponY12RVwpybN2wXGL2GxfqOnTeupqxtqpCa+c1ynkYPeN6HRbPDhMzsl5NNh1CpifDzJhxUw9ysLS+5p3tW09t1ONk9wnm4xsQn3WPrpULKFvCdK1pcRbvPWV1KR29OGmL/0rr10rLAYxZ+d88fK4k= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978313321973.0055275824003; Tue, 1 Oct 2019 18:05:13 -0700 (PDT) Received: from localhost ([::1]:49994 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT4l-0008MS-VL for importer@patchew.org; Tue, 01 Oct 2019 21:05:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40429) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsW-0001ta-FP for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsL-0002PQ-Oy for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:17 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:52394 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsK-0001uk-6I; Tue, 01 Oct 2019 19:48:13 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOhv098919; Tue, 1 Oct 2019 19:47:51 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v3m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:51 -0400 Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlpld099899; Tue, 1 Oct 2019 19:47:51 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v3c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:51 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlXK001425; Tue, 1 Oct 2019 23:47:50 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma03wdc.us.ibm.com with ESMTP id 2v9y56yf8g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:50 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NloGF46596384 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:50 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 112812805A; Tue, 1 Oct 2019 23:47:50 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E99EB28058; Tue, 1 Oct 2019 23:47:49 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:49 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 70/97] Revert "ide/ahci: Check for -ECANCELED in aio callbacks" Date: Tue, 1 Oct 2019 18:45:49 -0500 Message-Id: <20191001234616.7825-71-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow This reverts commit 0d910cfeaf2076b116b4517166d5deb0fea76394. It's not correct to just ignore an error code in a callback; we need to handle that error and possible report failure to the guest so that they don't wait indefinitely for an operation that will now never finish. This ought to help cases reported by Nutanix where iSCSI returns a legitimate -ECANCELED for certain operations which should be propagated normally. Reported-by: Shaju Abraham Signed-off-by: John Snow Message-id: 20190729223605.7163-1-jsnow@redhat.com Signed-off-by: John Snow (cherry picked from commit 8ec41c4265714255d5a138f8b538faf3583dcff6) Signed-off-by: Michael Roth --- hw/ide/ahci.c | 3 --- hw/ide/core.c | 14 -------------- 2 files changed, 17 deletions(-) diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c index d700ca973b..bda9583c8b 100644 --- a/hw/ide/ahci.c +++ b/hw/ide/ahci.c @@ -1022,9 +1022,6 @@ static void ncq_cb(void *opaque, int ret) IDEState *ide_state =3D &ncq_tfs->drive->port.ifs[0]; =20 ncq_tfs->aiocb =3D NULL; - if (ret =3D=3D -ECANCELED) { - return; - } =20 if (ret < 0) { bool is_read =3D ncq_tfs->cmd =3D=3D READ_FPDMA_QUEUED; diff --git a/hw/ide/core.c b/hw/ide/core.c index 6afadf894f..8e1624f7ce 100644 --- a/hw/ide/core.c +++ b/hw/ide/core.c @@ -722,9 +722,6 @@ static void ide_sector_read_cb(void *opaque, int ret) s->pio_aiocb =3D NULL; s->status &=3D ~BUSY_STAT; =20 - if (ret =3D=3D -ECANCELED) { - return; - } if (ret !=3D 0) { if (ide_handle_rw_error(s, -ret, IDE_RETRY_PIO | IDE_RETRY_READ)) { @@ -840,10 +837,6 @@ static void ide_dma_cb(void *opaque, int ret) uint64_t offset; bool stay_active =3D false; =20 - if (ret =3D=3D -ECANCELED) { - return; - } - if (ret =3D=3D -EINVAL) { ide_dma_error(s); return; @@ -975,10 +968,6 @@ static void ide_sector_write_cb(void *opaque, int ret) IDEState *s =3D opaque; int n; =20 - if (ret =3D=3D -ECANCELED) { - return; - } - s->pio_aiocb =3D NULL; s->status &=3D ~BUSY_STAT; =20 @@ -1058,9 +1047,6 @@ static void ide_flush_cb(void *opaque, int ret) =20 s->pio_aiocb =3D NULL; =20 - if (ret =3D=3D -ECANCELED) { - return; - } if (ret < 0) { /* XXX: What sector number to set here? */ if (ide_handle_rw_error(s, -ret, IDE_RETRY_FLUSH)) { --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979256; cv=none; d=zoho.com; s=zohoarc; b=n0CmdT+0eDAKmP+NtlTiANKSSQx+bNWqQcF1E12lVu/N8Hm3C8wCPH1jJ2QK7BvsMVIEQ/DlBJvQTRCnzopzB2ZShRBbmMhBsepdL/Yqm5A0nvktJHI3nSIwkwqJ+oh3891fD37aMGdoM0O7gwOub1OR8XR6Jk+Yrnkam5KvlHs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979256; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ojyt//EbfgO25cFpo6h573ksDzknDvXfzSW6QDCPTm4=; b=VGoVT7FmEJqCh88Fs7W8JkcxkS7zHqdEY4g+CzHy5Oyj2HqkWFygdNPkYZJEZlLbOMmpNsL08Wb6E+YnpMI3BtDpTwZfk3nLhH9lMjNZdz/3L8WIlTJEFlM/+n2TNhG8u35U2ZsP8w5HRfUpJ8oXLNntOtH3ZMfTP1ZznXIiP4M= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979256300495.8060951262132; Tue, 1 Oct 2019 18:20:56 -0700 (PDT) Received: from localhost ([::1]:50122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTK2-0006sF-8K for importer@patchew.org; Tue, 01 Oct 2019 21:20:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41336) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsq-0002GM-VY for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsn-00031g-1a for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:42 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:34206) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsi-0002PX-VL; Tue, 01 Oct 2019 19:48:38 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlUD2053657; Tue, 1 Oct 2019 19:47:53 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716ch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:52 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlqhQ054271; Tue, 1 Oct 2019 19:47:52 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716c6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:52 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlC4023568; Tue, 1 Oct 2019 23:47:51 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577h0n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:51 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlpDJ50725314 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:51 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0BE2E28058; Tue, 1 Oct 2019 23:47:51 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E44C42805A; Tue, 1 Oct 2019 23:47:50 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:50 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 71/97] qcow2: Fix the calculation of the maximum L2 cache size Date: Tue, 1 Oct 2019 18:45:50 -0500 Message-Id: <20191001234616.7825-72-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Alberto Garcia , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia The size of the qcow2 L2 cache defaults to 32 MB, which can be easily larger than the maximum amount of L2 metadata that the image can have. For example: with 64 KB clusters the user would need a qcow2 image with a virtual size of 256 GB in order to have 32 MB of L2 metadata. Because of that, since commit b749562d9822d14ef69c9eaa5f85903010b86c30 we forbid the L2 cache to become larger than the maximum amount of L2 metadata for the image, calculated using this formula: uint64_t max_l2_cache =3D virtual_disk_size / (s->cluster_size / 8); The problem with this formula is that the result should be rounded up to the cluster size because an L2 table on disk always takes one full cluster. For example, a 1280 MB qcow2 image with 64 KB clusters needs exactly 160 KB of L2 metadata, but we need 192 KB on disk (3 clusters) even if the last 32 KB of those are not going to be used. However QEMU rounds the numbers down and only creates 2 cache tables (128 KB), which is not enough for the image. A quick test doing 4KB random writes on a 1280 MB image gives me around 500 IOPS, while with the correct cache size I get 16K IOPS. Cc: qemu-stable@nongnu.org Signed-off-by: Alberto Garcia Signed-off-by: Kevin Wolf (cherry picked from commit b70d08205b2e4044c529eefc21df2c8ab61b473b) Signed-off-by: Michael Roth --- block/qcow2.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index 840f289a48..c80f48a02b 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -831,7 +831,11 @@ static void read_cache_sizes(BlockDriverState *bs, Qem= uOpts *opts, bool l2_cache_entry_size_set; int min_refcount_cache =3D MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; uint64_t virtual_disk_size =3D bs->total_sectors * BDRV_SECTOR_SIZE; - uint64_t max_l2_cache =3D virtual_disk_size / (s->cluster_size / 8); + uint64_t max_l2_entries =3D DIV_ROUND_UP(virtual_disk_size, s->cluster= _size); + /* An L2 table is always one cluster in size so the max cache size + * should be a multiple of the cluster size. */ + uint64_t max_l2_cache =3D ROUND_UP(max_l2_entries * sizeof(uint64_t), + s->cluster_size); =20 combined_cache_size_set =3D qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE); l2_cache_size_set =3D qemu_opt_get(opts, QCOW2_OPT_L2_CACHE_SIZE); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977908; cv=none; d=zoho.com; s=zohoarc; b=D6IwEOgm42WX5XifTL1Vdck3rPA3CpmbqXy/BDopn1nJS8TQJqiFDP9AWme8RIDVlinfJI1wHFY8wXFTlHIRJVFUGEt+7fIiCqCPU+ceRbyULm3gUhq+LzZ243Ir40H8ApRnj+dNvlUn2hb7wA4uaarK5BZcEtOhL3tMep7iJ30= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977908; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=UGvrWKUZktgMwEroPZL7V4SZlNgs6VBrt//+w1kvqBI=; b=VDTp9gCJWZfk0amJG1UvgKF5+6u8OEavBwFBBOcUY/PYCQUJXmRQYjnceE7NYbYmd/msXNnffw29PjYdTAu+p0j7OdcwOzD697ivNeLk4FErLH4EX4weaoYhK8BEHbclGZIRN5emrUCO6sTAaNhwr4//LZiHCbuBoMpHNpKYljg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977908376938.6700305080974; Tue, 1 Oct 2019 17:58:28 -0700 (PDT) Received: from localhost ([::1]:49942 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSyI-0002k3-AN for importer@patchew.org; Tue, 01 Oct 2019 20:58:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40543) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsY-0001xQ-Pr for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002Yp-DX for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:26 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:54934) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsO-0001y3-DT; Tue, 01 Oct 2019 19:48:19 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlWwg135052; Tue, 1 Oct 2019 19:47:53 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwberj9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:53 -0400 Received: from m0098404.ppops.net (m0098404.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlrPW138284; Tue, 1 Oct 2019 19:47:53 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwberj0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:53 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlmW032693; Tue, 1 Oct 2019 23:47:52 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02dal.us.ibm.com with ESMTP id 2v9y57qb8c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:52 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlp6d53674250 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:51 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 878E328065; Tue, 1 Oct 2019 23:47:51 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6C1E22805C; Tue, 1 Oct 2019 23:47:51 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:51 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 72/97] dma-helpers: ensure AIO callback is invoked after cancellation Date: Tue, 1 Oct 2019 18:45:51 -0500 Message-Id: <20191001234616.7825-73-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , John Snow , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini dma_aio_cancel unschedules the BH if there is one, which corresponds to the reschedule_dma case of dma_blk_cb. This can stall the DMA permanently, because dma_complete will never get invoked and therefore nobody will ever invoke the original AIO callback in dbs->common.cb. Fix this by invoking the callback (which is ensured to happen after a bdrv_aio_cancel_async, or done manually in the dbs->bh case), and add assertions to check that the DMA state machine is indeed waiting for dma_complete or reschedule_dma, but never both. Reported-by: John Snow Signed-off-by: Paolo Bonzini Message-id: 20190729213416.1972-1-pbonzini@redhat.com Signed-off-by: John Snow (cherry picked from commit 539343c0a47e19d5dd64d846d64d084d9793681f) Signed-off-by: Michael Roth --- dma-helpers.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/dma-helpers.c b/dma-helpers.c index 2d7e02d35e..d3871dc61e 100644 --- a/dma-helpers.c +++ b/dma-helpers.c @@ -90,6 +90,7 @@ static void reschedule_dma(void *opaque) { DMAAIOCB *dbs =3D (DMAAIOCB *)opaque; =20 + assert(!dbs->acb && dbs->bh); qemu_bh_delete(dbs->bh); dbs->bh =3D NULL; dma_blk_cb(dbs, 0); @@ -111,15 +112,12 @@ static void dma_complete(DMAAIOCB *dbs, int ret) { trace_dma_complete(dbs, ret, dbs->common.cb); =20 + assert(!dbs->acb && !dbs->bh); dma_blk_unmap(dbs); if (dbs->common.cb) { dbs->common.cb(dbs->common.opaque, ret); } qemu_iovec_destroy(&dbs->iov); - if (dbs->bh) { - qemu_bh_delete(dbs->bh); - dbs->bh =3D NULL; - } qemu_aio_unref(dbs); } =20 @@ -179,14 +177,21 @@ static void dma_aio_cancel(BlockAIOCB *acb) =20 trace_dma_aio_cancel(dbs); =20 + assert(!(dbs->acb && dbs->bh)); if (dbs->acb) { + /* This will invoke dma_blk_cb. */ blk_aio_cancel_async(dbs->acb); + return; } + if (dbs->bh) { cpu_unregister_map_client(dbs->bh); qemu_bh_delete(dbs->bh); dbs->bh =3D NULL; } + if (dbs->common.cb) { + dbs->common.cb(dbs->common.opaque, -ECANCELED); + } } =20 static AioContext *dma_get_aio_context(BlockAIOCB *acb) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979862; cv=none; d=zoho.com; s=zohoarc; b=X5tmK20XSO4vKTx9TZjLXHysobrzEfql0O2jN+Hw/6yvcGd6AEyFrK/qBxNWmgsY0XD8iVzNHl5h1Hwa4vzSKnYLiIwF1KV1hm251In5LFS7KAR6BOTNpd3n7T9gIymsv3LHRdsqovrX9HmK7kXOSsZcpD3DOhrp+8lZLTMqj1w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979862; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=dKTaeJ3AmYxRhHCmywjO26Wnln0EwXo8PT2H1XHjLKg=; b=g93jCUrXXdUTnza6Kppb+beyKQsmTrlE0MvXcT5SEVMBXx5wgvzyu1ZQQ68Zk0ia47+iq+7nlr0COfiwJn67L1z8jLamXX/xTmKP0m/yoCsuchLnemOYIWd7DZnahkwgy7YMPnhB2sjaeAe8HquTAqZl3SfaqQgp3PrSRhTKZg8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979862565786.1927627116664; Tue, 1 Oct 2019 18:31:02 -0700 (PDT) Received: from localhost ([::1]:50196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTTo-0007oo-4d for importer@patchew.org; Tue, 01 Oct 2019 21:31:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41339) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsq-0002GS-VN for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsi-0002wP-I5 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:42 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:8688) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsO-0001z8-GK; Tue, 01 Oct 2019 19:48:19 -0400 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlSJr086592; Tue, 1 Oct 2019 19:47:54 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat810x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:53 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlZ2029380; Tue, 1 Oct 2019 23:47:52 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fak2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:52 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlqxN36831538 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:52 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 12F442805E; Tue, 1 Oct 2019 23:47:52 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EA8F728058; Tue, 1 Oct 2019 23:47:51 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:51 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 73/97] target/arm: Don't abort on M-profile exception return in linux-user mode Date: Tue, 1 Oct 2019 18:45:52 -0500 Message-Id: <20191001234616.7825-74-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Peter Maydell An attempt to do an exception-return (branch to one of the magic addresses) in linux-user mode for M-profile should behave like a normal branch, because linux-user mode is always going to be in 'handler' mode. This used to work, but we broke it when we added support for the M-profile security extension in commit d02a8698d7ae2bfed. In that commit we allowed even handler-mode calls to magic return values to be checked for and dealt with by causing an EXCP_EXCEPTION_EXIT exception to be taken, because this is needed for the FNC_RETURN return-from-non-secure-function-call handling. For system mode we added a check in do_v7m_exception_exit() to make any spurious calls from Handler mode behave correctly, but forgot that linux-user mode would also be affected. How an attempted return-from-non-secure-function-call in linux-user mode should be handled is not clear -- on real hardware it would result in return to secure code (not to the Linux kernel) which could then handle the error in any way it chose. For QEMU we take the simple approach of treating this erroneous return the same way it would be handled on a CPU without the security extensions -- treat it as a normal branch. The upshot of all this is that for linux-user mode we should never do any of the bx_excret magic, so the code change is simple. This ought to be a weird corner case that only affects broken guest code (because Linux user processes should never be attempting to do exception returns or NS function returns), except that the code that assigns addresses in RAM for the process and stack in our linux-user code does not attempt to avoid this magic address range, so legitimate code attempting to return to a trampoline routine on the stack can fall into this case. This change fixes those programs, but we should also look at restricting the range of memory we use for M-profile linux-user guests to the area that would be real RAM in hardware. Cc: qemu-stable@nongnu.org Reported-by: Christophe Lyon Reviewed-by: Richard Henderson Signed-off-by: Peter Maydell Message-id: 20190822131534.16602-1-peter.maydell@linaro.org Fixes: https://bugs.launchpad.net/qemu/+bug/1840922 Signed-off-by: Peter Maydell (cherry picked from commit 5e5584c89f36b302c666bc6db535fd3f7ff35ad2) Signed-off-by: Michael Roth --- target/arm/translate.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/target/arm/translate.c b/target/arm/translate.c index d408e4d7ef..d9d4e765ca 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -964,10 +964,27 @@ static inline void gen_bx(DisasContext *s, TCGv_i32 v= ar) store_cpu_field(var, thumb); } =20 -/* Set PC and Thumb state from var. var is marked as dead. +/* + * Set PC and Thumb state from var. var is marked as dead. * For M-profile CPUs, include logic to detect exception-return * branches and handle them. This is needed for Thumb POP/LDM to PC, LDR t= o PC, * and BX reg, and no others, and happens only for code in Handler mode. + * The Security Extension also requires us to check for the FNC_RETURN + * which signals a function return from non-secure state; this can happen + * in both Handler and Thread mode. + * To avoid having to do multiple comparisons in inline generated code, + * we make the check we do here loose, so it will match for EXC_RETURN + * in Thread mode. For system emulation do_v7m_exception_exit() checks + * for these spurious cases and returns without doing anything (giving + * the same behaviour as for a branch to a non-magic address). + * + * In linux-user mode it is unclear what the right behaviour for an + * attempted FNC_RETURN should be, because in real hardware this will go + * directly to Secure code (ie not the Linux kernel) which will then treat + * the error in any way it chooses. For QEMU we opt to make the FNC_RETURN + * attempt behave the way it would on a CPU without the security extension, + * which is to say "like a normal branch". That means we can simply treat + * all branches as normal with no magic address behaviour. */ static inline void gen_bx_excret(DisasContext *s, TCGv_i32 var) { @@ -975,10 +992,12 @@ static inline void gen_bx_excret(DisasContext *s, TCG= v_i32 var) * s->base.is_jmp that we need to do the rest of the work later. */ gen_bx(s, var); +#ifndef CONFIG_USER_ONLY if (arm_dc_feature(s, ARM_FEATURE_M_SECURITY) || (s->v7m_handler_mode && arm_dc_feature(s, ARM_FEATURE_M))) { s->base.is_jmp =3D DISAS_BX_EXCRET; } +#endif } =20 static inline void gen_bx_excret_final_code(DisasContext *s) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569981803; cv=none; d=zoho.com; s=zohoarc; b=gew6fZkCsSoaGozy+9Y0/mTsVo9Ir4zbnFUd7Lyrp2CyIHRMMdMwfLLd3BIlF78HU8yPK70PlGgzKjVZmKA71yhGqKsfD9Luad5j8EGkQ9K//wBVJeElp0PKAg7Hw1jIBPjpNsBHsxweVIfsDLL2d6zHidXjepy1NGy2TAdbmfA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569981803; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=emNSpvJJDBm1nkvadsZPVsTG4uZxd0n4z2KWrQY2R6g=; b=QT7vrfqj44c2wx5Pv4WFp9opYMoCFVkKX2zfgYWcdiSKVRDv1h7J8nUg5EBPEXngmto7reuD7oIEToTXv79R/WPc8WpdMy9ar88DYKGStbqSjg/GuIHSGpaSfmRQY8dsnCWBsZmEDN7sS4y4ghV9TSfRJbFN0falVVav9gfTE5M= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569981803750876.3148987509691; Tue, 1 Oct 2019 19:03:23 -0700 (PDT) Received: from localhost ([::1]:50516 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTz6-0008Hm-Bw for importer@patchew.org; Tue, 01 Oct 2019 22:03:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41277) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsp-0002E9-4n for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsi-0002vs-ES for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:10352) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsO-0001zF-HH; Tue, 01 Oct 2019 19:48:19 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQrj107943; Tue, 1 Oct 2019 19:47:54 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurpb9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:54 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkKH027153; Tue, 1 Oct 2019 23:47:53 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bj8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:53 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlqXn36831542 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:52 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8C54C28058; Tue, 1 Oct 2019 23:47:52 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7033E28059; Tue, 1 Oct 2019 23:47:52 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:52 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 74/97] xen-bus: Fix backend state transition on device reset Date: Tue, 1 Oct 2019 18:45:53 -0500 Message-Id: <20191001234616.7825-75-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anthony PERARD , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Anthony PERARD When a frontend wants to reset its state and the backend one, it starts with setting "Closing", then waits for the backend (QEMU) to do the same. But when QEMU is setting "Closing" to its state, it triggers an event (xenstore watch) that re-execute xen_device_backend_changed() and set the backend state to "Closed". QEMU should wait for the frontend to set "Closed" before doing the same. Before setting "Closed" to the backend_state, we are also going to check if there is a frontend. If that the case, when the backend state is set to "Closing" the frontend should react and sets its state to "Closing" then "Closed". The backend should wait for that to happen. Fixes: b6af8926fb858c4f1426e5acb2cfc1f0580ec98a Signed-off-by: Anthony PERARD Reviewed-by: Paul Durrant Message-Id: <20190823101534.465-2-anthony.perard@citrix.com> (cherry picked from commit cb3231460747552d70af9d546dc53d8195bcb796) Signed-off-by: Michael Roth --- hw/xen/xen-bus.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c index 49a725e8c7..0a1033d9a9 100644 --- a/hw/xen/xen-bus.c +++ b/hw/xen/xen-bus.c @@ -515,6 +515,23 @@ static void xen_device_backend_set_online(XenDevice *x= endev, bool online) xen_device_backend_printf(xendev, "online", "%u", online); } =20 +/* + * Tell from the state whether the frontend is likely alive, + * i.e. it will react to a change of state of the backend. + */ +static bool xen_device_state_is_active(enum xenbus_state state) +{ + switch (state) { + case XenbusStateInitWait: + case XenbusStateInitialised: + case XenbusStateConnected: + case XenbusStateClosing: + return true; + default: + return false; + } +} + static void xen_device_backend_changed(void *opaque) { XenDevice *xendev =3D opaque; @@ -538,11 +555,11 @@ static void xen_device_backend_changed(void *opaque) =20 /* * If the toolstack (or unplug request callback) has set the backend - * state to Closing, but there is no active frontend (i.e. the - * state is not Connected) then set the backend state to Closed. + * state to Closing, but there is no active frontend then set the + * backend state to Closed. */ if (xendev->backend_state =3D=3D XenbusStateClosing && - xendev->frontend_state !=3D XenbusStateConnected) { + !xen_device_state_is_active(state)) { xen_device_backend_set_state(xendev, XenbusStateClosed); } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980514; cv=none; d=zoho.com; s=zohoarc; b=FqKznAl0liZ9vrlrQmQtPMZ41MZwAAZQdL8/VmDIKaPY8MfyFKWMMojJkU65j6KiOudCK2gdIB+wYK2FdBx1N/oY6+oy+Xr0pkyenXZbGpEqNQb2GNfwyZPU7rm6D6eMMHvasvl/xGHfWvb4eN/ND3gi/KOytRWu0z8OF2yb50I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980514; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=doHTtE2a7OnFTP5VRoy4hv3u3VNlw6peWDvSNco5ncU=; b=PxVidB6mFh3YUgBUz1/W4ytYVEZ+5vNMdyLIpRWB/cPU+G3ntX81AQKkI6lfuwL5eTvS8Ao8b61ke5oubuywrMXP45dLpjd1RjB8AzFi3bzyCkSYTTTxT+G6xTSAJIx9PX31oeYoGxsgeaSg+kt8ehB24vHQcWbFcyeA9y/iIlQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569980514618828.5485436653402; Tue, 1 Oct 2019 18:41:54 -0700 (PDT) Received: from localhost ([::1]:50306 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTeE-0003J6-7H for importer@patchew.org; Tue, 01 Oct 2019 21:41:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41539) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsw-0002Ja-T6 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRst-00038W-8s for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:48 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:29070 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsr-0002bN-GN; Tue, 01 Oct 2019 19:48:46 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NmLsK132821; Tue, 1 Oct 2019 19:48:24 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4deca-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:24 -0400 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NmN88132956; Tue, 1 Oct 2019 19:48:23 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdj4de70-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:22 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njoff032175; Tue, 1 Oct 2019 23:47:55 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma01wdc.us.ibm.com with ESMTP id 2v9y57fgcc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:55 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlrFg43778512 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:53 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0EEAB2805C; Tue, 1 Oct 2019 23:47:53 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E684D28058; Tue, 1 Oct 2019 23:47:52 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:52 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 75/97] pr-manager: Fix invalid g_free() crash bug Date: Tue, 1 Oct 2019 18:45:54 -0500 Message-Id: <20191001234616.7825-76-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0b-001b2d01.pphosted.com id x91NmLsK132821 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Markus Armbruster pr_manager_worker() passes its @opaque argument to g_free(). Wrong; it points to pr_manager_worker()'s automatic @data. Broken when commit 2f3a7ab39be converted @data from heap- to stack-allocated. Fix by deleting the g_free(). Fixes: 2f3a7ab39bec4ba8022dc4d42ea641165b004e3e Cc: qemu-stable@nongnu.org Signed-off-by: Markus Armbruster Reviewed-by: Philippe Mathieu-Daud=C3=A9 Acked-by: Paolo Bonzini Signed-off-by: Kevin Wolf (cherry picked from commit 6b9d62c2a9e83bbad73fb61406f0ff69b46ff6f3) Signed-off-by: Michael Roth --- scsi/pr-manager.c | 1 - 1 file changed, 1 deletion(-) diff --git a/scsi/pr-manager.c b/scsi/pr-manager.c index d9f4e8c3ad..227bdfaad2 100644 --- a/scsi/pr-manager.c +++ b/scsi/pr-manager.c @@ -38,7 +38,6 @@ static int pr_manager_worker(void *opaque) int fd =3D data->fd; int r; =20 - g_free(data); trace_pr_manager_run(fd, hdr->cmdp[0], hdr->cmdp[1]); =20 /* The reference was taken in pr_manager_execute. */ --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978527; cv=none; d=zoho.com; s=zohoarc; b=LSMhggzCkLBosWvF0ogo+4YKBaq2lKwUCtCbS1oivvz44xF+su05yngrQuvLd+ZkOfWuAi5EtJ4zHfZTv0KVF0ePiuT88RQY30cm6dQSxrVibCJO+K/PTQ+Wb3+o+lO4BgoZF1Q9ze/hBoG9L18kwQNTK3XWQTue1/LUXLKGNrE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978527; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=WmmIvAgcZilDHcIvjaIq1z+SlLCeU8Oc3bQHMyA2MVc=; b=jPMlkODAr1HYu3fAtVNvYR5z1VBBU3gfL7RX2eMW9h/oWageFp1xZdKfnuvP235tneZRDfJGRgeklEewCJ70X5fs4EKSJkFa5qjxY79WHqK21BNAXIv7g2arYVZOsY6g/cUyn3qhlySmhUZlFFk+Kk8QvPnlvhSd6SEKpjX/NZQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978527497943.1357524526912; Tue, 1 Oct 2019 18:08:47 -0700 (PDT) Received: from localhost ([::1]:50032 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT8F-0003dM-Ao for importer@patchew.org; Tue, 01 Oct 2019 21:08:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40653) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsc-00021S-BB for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002aF-QP for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:14720) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsQ-0001zh-Bq; Tue, 01 Oct 2019 19:48:23 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQWQ134232; Tue, 1 Oct 2019 19:47:55 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwberjx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:55 -0400 Received: from m0098404.ppops.net (m0098404.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NltKl138568; Tue, 1 Oct 2019 19:47:55 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccwberjm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:55 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkEU027150; Tue, 1 Oct 2019 23:47:54 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bja-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:54 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlrxm53477790 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:53 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 86D242805E; Tue, 1 Oct 2019 23:47:53 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6B4152805A; Tue, 1 Oct 2019 23:47:53 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:53 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 76/97] iotests: add testing shim for script-style python tests Date: Tue, 1 Oct 2019 18:45:55 -0500 Message-Id: <20191001234616.7825-77-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=948 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: John Snow , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Because the new-style python tests don't use the iotests.main() test launcher, we don't turn on the debugger logging for these scripts when invoked via ./check -d. Refactor the launcher shim into new and old style shims so that they share environmental configuration. Two cleanup notes: debug was not actually used as a global, and there was no reason to create a class in an inner scope just to achieve default variables; we can simply create an instance of the runner with the values we want instead. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-14-jsnow@redhat.com Signed-off-by: John Snow (cherry picked from commit 456a2d5ac7641c7e75c76328a561b528a8607a8e) *prereq for 1a37e31244/88d2aa533a Signed-off-by: Michael Roth --- tests/qemu-iotests/iotests.py | 40 +++++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index a7006662ff..5efc21d801 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -61,7 +61,6 @@ cachemode =3D os.environ.get('CACHEMODE') qemu_default_machine =3D os.environ.get('QEMU_DEFAULT_MACHINE') =20 socket_scm_helper =3D os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_help= er') -debug =3D False =20 luks_default_secret_object =3D 'secret,id=3Dkeysec0,data=3D' + \ os.environ.get('IMGKEYSECRET', '') @@ -817,11 +816,22 @@ def skip_if_unsupported(required_formats=3D[], read_o= nly=3DFalse): return func_wrapper return skip_test_decorator =20 -def main(supported_fmts=3D[], supported_oses=3D['linux'], supported_cache_= modes=3D[], - unsupported_fmts=3D[]): - '''Run tests''' +def execute_unittest(output, verbosity, debug): + runner =3D unittest.TextTestRunner(stream=3Doutput, descriptions=3DTru= e, + verbosity=3Dverbosity) + try: + # unittest.main() will use sys.exit(); so expect a SystemExit + # exception + unittest.main(testRunner=3Drunner) + finally: + if not debug: + sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', + r'Ran \1 tests', output.getvalue())) =20 - global debug +def execute_test(test_function=3DNone, + supported_fmts=3D[], supported_oses=3D['linux'], + supported_cache_modes=3D[], unsupported_fmts=3D[]): + """Run either unittest or script-style tests.""" =20 # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to # indicate that we're not being run via "check". There may be @@ -853,13 +863,15 @@ def main(supported_fmts=3D[], supported_oses=3D['linu= x'], supported_cache_modes=3D[], =20 logging.basicConfig(level=3D(logging.DEBUG if debug else logging.WARN)) =20 - class MyTestRunner(unittest.TextTestRunner): - def __init__(self, stream=3Doutput, descriptions=3DTrue, verbosity= =3Dverbosity): - unittest.TextTestRunner.__init__(self, stream, descriptions, v= erbosity) + if not test_function: + execute_unittest(output, verbosity, debug) + else: + test_function() + +def script_main(test_function, *args, **kwargs): + """Run script-style tests outside of the unittest framework""" + execute_test(test_function, *args, **kwargs) =20 - # unittest.main() will use sys.exit() so expect a SystemExit exception - try: - unittest.main(testRunner=3DMyTestRunner) - finally: - if not debug: - sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran = \1 tests', output.getvalue())) +def main(*args, **kwargs): + """Run tests using the unittest framework""" + execute_test(None, *args, **kwargs) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978182; cv=none; d=zoho.com; s=zohoarc; b=nq+na+nDnJ6oyK88DFtvQZGX9EvPnDiyL/FKnQSqxxclQrXp2yXpuBJg9p8aRFH7oqRQNRBtJ/qrLZOVCnCeZ4zQQJH7v2YFX3KQB9YX73UyiojEf51YZYGO3TgxUctff0JZNOJAx2iRK0BIsXBiFymsNDDS7VHiLp2YPExz7p0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978182; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=vMyqF8Lho5MUcx7ADw7BhnY6yhF+iHigl9wKM3zwYf0=; b=QV2eA6l78rl4FzoC2c9w0TEeuNK/WoL23ebxjtzDn0kpiSCP/043Q7IclxEE3xR2FK5ke5S9ZyZfAgaYrGsESzp0iNX2PpC41U7v76ldsYlLTnFDNKOzyhjY5UJtBMpFK/ezcir6IC9iaGu7iJbtLHS4UdKeMvGrbphz03cVWJ4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15699781822128.099996725090364; Tue, 1 Oct 2019 18:03:02 -0700 (PDT) Received: from localhost ([::1]:49990 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT2e-0006v4-Ay for importer@patchew.org; Tue, 01 Oct 2019 21:02:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40623) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsb-00020V-B6 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002Zg-Lq for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:3658) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsQ-00020Y-Ei; Tue, 01 Oct 2019 19:48:23 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOLf053207; Tue, 1 Oct 2019 19:47:56 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716dj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:56 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlYN8053778; Tue, 1 Oct 2019 19:47:55 -0400 Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716d7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:55 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlTI022434; Tue, 1 Oct 2019 23:47:54 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04dal.us.ibm.com with ESMTP id 2v9y57yd7m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:54 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nls7r53543206 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:54 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B39F28059; Tue, 1 Oct 2019 23:47:54 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E46BA2805A; Tue, 1 Oct 2019 23:47:53 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:53 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 77/97] vpc: Return 0 from vpc_co_create() on success Date: Tue, 1 Oct 2019 18:45:56 -0500 Message-Id: <20191001234616.7825-78-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz blockdev_create_run() directly uses .bdrv_co_create()'s return value as the job's return value. Jobs must return 0 on success, not just any nonnegative value. Therefore, using blockdev-create for VPC images may currently fail as the vpc driver may return a positive integer. Because there is no point in returning a positive integer anywhere in the block layer (all non-negative integers are generally treated as complete success), we probably do not want to add more such cases. Therefore, fix this problem by making the vpc driver always return 0 in case of success. Suggested-by: Kevin Wolf Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf (cherry picked from commit 1a37e3124407b5a145d44478d3ecbdb89c63789f) Signed-off-by: Michael Roth --- block/vpc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/vpc.c b/block/vpc.c index a902a4c54d..dbc3da2bd5 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -881,6 +881,7 @@ static int create_dynamic_disk(BlockBackend *blk, uint8= _t *buf, goto fail; } =20 + ret =3D 0; fail: return ret; } @@ -904,7 +905,7 @@ static int create_fixed_disk(BlockBackend *blk, uint8_t= *buf, return ret; } =20 - return ret; + return 0; } =20 static int calculate_rounded_image_size(BlockdevCreateOptionsVpc *vpc_opts, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569977967; cv=none; d=zoho.com; s=zohoarc; b=afPZ2F01DzVr8cH3S9haPVo6S04kdEh8POUik4SV9fjcRWEQptkYFkTcgVo8i9u1L5SMxuKp2P/wvcCt7I7Yhi1Q22eoUXjjkNW1RvsCpKAGdIohgKrLNowuO8WSiom2RIQiAoUQFxjU8219A8OqzY1DaFbDtOlQxmjBptfJiT4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569977967; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=RZtL8ni3qTi+JP2zYgZLcYCIIyOlf6XG1lQTD+8NZN4=; b=jsmLcS0CFOx4f7CRD34j6wx2xJ0PqYnyHw+H8aaL1ZSVC5mv5rYYD5E6BUC/+OEIOZsajU8uK8n3AH100GMj0GSWlFPwsJHnl05vRlWfEXTDR1w2J3wqEgmXnuD2wiERJOhYP5wRljfB2cL30BbxEf5jL/zD/KD/PavvxAjq2QY= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569977967200815.3132345903739; Tue, 1 Oct 2019 17:59:27 -0700 (PDT) Received: from localhost ([::1]:49946 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFSzF-0003rS-My for importer@patchew.org; Tue, 01 Oct 2019 20:59:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40612) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsb-000209-1s for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002Zm-MS for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:28 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:62468) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsQ-00020X-Ex; Tue, 01 Oct 2019 19:48:23 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlVud053669; Tue, 1 Oct 2019 19:47:56 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716dr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:56 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlu83054361; Tue, 1 Oct 2019 19:47:56 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716dd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:56 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njlma032693; Tue, 1 Oct 2019 23:47:55 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02dal.us.ibm.com with ESMTP id 2v9y57qb8j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:55 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlspG48169240 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:54 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 851C728058; Tue, 1 Oct 2019 23:47:54 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6950028059; Tue, 1 Oct 2019 23:47:54 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:54 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 78/97] iotests: Add supported protocols to execute_test() Date: Tue, 1 Oct 2019 18:45:57 -0500 Message-Id: <20191001234616.7825-79-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=701 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf (cherry picked from commit 88d2aa533a4a1aad44a27c2e6cd5bc5fbcbce7ed) Signed-off-by: Michael Roth --- tests/qemu-iotests/iotests.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 5efc21d801..069cb20be9 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -830,7 +830,8 @@ def execute_unittest(output, verbosity, debug): =20 def execute_test(test_function=3DNone, supported_fmts=3D[], supported_oses=3D['linux'], - supported_cache_modes=3D[], unsupported_fmts=3D[]): + supported_cache_modes=3D[], unsupported_fmts=3D[], + supported_protocols=3D[], unsupported_protocols=3D[]): """Run either unittest or script-style tests.""" =20 # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to @@ -844,6 +845,7 @@ def execute_test(test_function=3DNone, debug =3D '-d' in sys.argv verbosity =3D 1 verify_image_format(supported_fmts, unsupported_fmts) + verify_protocol(supported_protocols, unsupported_protocols) verify_platform(supported_oses) verify_cache_mode(supported_cache_modes) =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569981480; cv=none; d=zoho.com; s=zohoarc; b=dgrKV1psIxXdpodityMPE6UxPmOpW4f0Zk1K6yDPduwual6bpACQckaybS03RKHbyNhgpOVV8YmCoVICzUcu5lFY4ZArtMP7f1SQ5+35ryB2tz4SHzij4s66BxXsqQ3VF7Bm0fbUG0mCVn8B+CS4HRgSvK0GMCVl2IhALoepSyU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569981480; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=YEPZoFLFTjOTjpXVF3M5QdldZBjTItf6upndMgKPZjk=; b=R9W0amqO2kEI7rt24BqM/RJLLdOmhC66w3k/ZCUljPQjjwkbWjAxchexr8+SE4twHpjW41b23IeWOUuh+g/5zSRSLKzIk2K8DCbqSqFUF12QOGwKqBC1T8Dv+PpkBYa9FGybU5hNOoRpY5XSw9T1lq63QFAQvSRumntoUL/f0tw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569981480176856.4533482980415; Tue, 1 Oct 2019 18:58:00 -0700 (PDT) Received: from localhost ([::1]:50452 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTtu-0002jd-TX for importer@patchew.org; Tue, 01 Oct 2019 21:57:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40857) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsf-00026n-VG for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002ZJ-Kk for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:33 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:13302 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsQ-00020d-FN; Tue, 01 Oct 2019 19:48:23 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQOw079000; Tue, 1 Oct 2019 19:47:57 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f013d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:56 -0400 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NluaD079623; Tue, 1 Oct 2019 19:47:56 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcc4f0134-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:56 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkEW027150; Tue, 1 Oct 2019 23:47:55 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bjf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:55 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NltOq16122740 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:55 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0E35F28058; Tue, 1 Oct 2019 23:47:55 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC6AD2805A; Tue, 1 Oct 2019 23:47:54 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:54 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 79/97] iotests: Restrict file Python tests to file Date: Tue, 1 Oct 2019 18:45:58 -0500 Message-Id: <20191001234616.7825-80-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Most of our Python unittest-style tests only support the file protocol. You can run them with any other protocol, but the test will simply ignore your choice and use file anyway. We should let them signal that they require the file protocol so they are skipped when you want to test some other protocol. Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf (cherry picked from commit 103cbc771e5660d1f5bb458be80aa9e363547ae0) Conflicts: tests/qemu-iotests/257 *drop changes for test 257, not present in 4.0 Signed-off-by: Michael Roth --- tests/qemu-iotests/030 | 3 ++- tests/qemu-iotests/040 | 3 ++- tests/qemu-iotests/041 | 3 ++- tests/qemu-iotests/044 | 3 ++- tests/qemu-iotests/045 | 3 ++- tests/qemu-iotests/055 | 3 ++- tests/qemu-iotests/056 | 3 ++- tests/qemu-iotests/057 | 3 ++- tests/qemu-iotests/065 | 3 ++- tests/qemu-iotests/096 | 3 ++- tests/qemu-iotests/118 | 3 ++- tests/qemu-iotests/124 | 3 ++- tests/qemu-iotests/129 | 3 ++- tests/qemu-iotests/132 | 3 ++- tests/qemu-iotests/139 | 3 ++- tests/qemu-iotests/148 | 3 ++- tests/qemu-iotests/151 | 3 ++- tests/qemu-iotests/152 | 3 ++- tests/qemu-iotests/155 | 3 ++- tests/qemu-iotests/163 | 3 ++- tests/qemu-iotests/165 | 3 ++- tests/qemu-iotests/169 | 3 ++- tests/qemu-iotests/196 | 3 ++- tests/qemu-iotests/199 | 3 ++- tests/qemu-iotests/245 | 3 ++- 25 files changed, 50 insertions(+), 25 deletions(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index c6311d1825..cb550812a7 100755 --- a/tests/qemu-iotests/030 +++ b/tests/qemu-iotests/030 @@ -865,4 +865,5 @@ class TestSetSpeed(iotests.QMPTestCase): self.cancel_and_wait(resume=3DTrue) =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2', 'qed']) + iotests.main(supported_fmts=3D['qcow2', 'qed'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index b81133a474..e2ca706117 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -395,4 +395,5 @@ class TestReopenOverlay(ImageCommitTestCase): self.run_commit_test(self.img1, self.img0) =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2', 'qed']) + iotests.main(supported_fmts=3D['qcow2', 'qed'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 index 26bf1701eb..ae6ed952c6 100755 --- a/tests/qemu-iotests/041 +++ b/tests/qemu-iotests/041 @@ -1068,4 +1068,5 @@ class TestOrphanedSource(iotests.QMPTestCase): self.assert_qmp(result, 'error/class', 'GenericError') =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2', 'qed']) + iotests.main(supported_fmts=3D['qcow2', 'qed'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 index 9ec3dba734..05ea1f49c5 100755 --- a/tests/qemu-iotests/044 +++ b/tests/qemu-iotests/044 @@ -118,4 +118,5 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): pass =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/045 b/tests/qemu-iotests/045 index d5484a0ee1..01cc038884 100755 --- a/tests/qemu-iotests/045 +++ b/tests/qemu-iotests/045 @@ -175,4 +175,5 @@ class TestSCMFd(iotests.QMPTestCase): "File descriptor named '%s' not found" % fdname) =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['raw']) + iotests.main(supported_fmts=3D['raw'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055 index 3437c11507..c732a112d6 100755 --- a/tests/qemu-iotests/055 +++ b/tests/qemu-iotests/055 @@ -563,4 +563,5 @@ class TestDriveCompression(iotests.QMPTestCase): target=3D'drive1') =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['raw', 'qcow2']) + iotests.main(supported_fmts=3D['raw', 'qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index e5ac25127b..ce7adf9174 100755 --- a/tests/qemu-iotests/056 +++ b/tests/qemu-iotests/056 @@ -335,4 +335,5 @@ class BackupTest(iotests.QMPTestCase): self.dismissal_failure(True) =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2', 'qed']) + iotests.main(supported_fmts=3D['qcow2', 'qed'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/057 b/tests/qemu-iotests/057 index 9f0a5a3057..9fbba759b6 100755 --- a/tests/qemu-iotests/057 +++ b/tests/qemu-iotests/057 @@ -256,4 +256,5 @@ class TestSnapshotDelete(ImageSnapshotTestCase): self.assert_qmp(result, 'error/class', 'GenericError') =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065 index 8bac383ea7..5b21eb96bd 100755 --- a/tests/qemu-iotests/065 +++ b/tests/qemu-iotests/065 @@ -129,4 +129,5 @@ TestQemuImgInfo =3D None TestQMP =3D None =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/096 b/tests/qemu-iotests/096 index a69439602d..ab9cb47822 100755 --- a/tests/qemu-iotests/096 +++ b/tests/qemu-iotests/096 @@ -67,4 +67,5 @@ class TestLiveSnapshot(iotests.QMPTestCase): self.checkConfig('target') =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/118 b/tests/qemu-iotests/118 index 603e10e8a2..c854378281 100755 --- a/tests/qemu-iotests/118 +++ b/tests/qemu-iotests/118 @@ -703,4 +703,5 @@ if __name__ =3D=3D '__main__': iotests.qemu_default_machine) # Need to support image creation iotests.main(supported_fmts=3D['vpc', 'parallels', 'qcow', 'vdi', 'qco= w2', - 'vmdk', 'raw', 'vhdx', 'qed']) + 'vmdk', 'raw', 'vhdx', 'qed'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3440f54781..ca40ba3be2 100755 --- a/tests/qemu-iotests/124 +++ b/tests/qemu-iotests/124 @@ -779,4 +779,5 @@ class TestIncrementalBackupBlkdebug(TestIncrementalBack= upBase): =20 =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 9e87e1c8d9..cd6b9e9ce7 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -83,4 +83,5 @@ class TestStopWithBlockJob(iotests.QMPTestCase): self.do_test_stop("block-commit", device=3D"drive0") =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D["qcow2"]) + iotests.main(supported_fmts=3D["qcow2"], + supported_protocols=3D["file"]) diff --git a/tests/qemu-iotests/132 b/tests/qemu-iotests/132 index f53ef6e391..0f2a106c81 100755 --- a/tests/qemu-iotests/132 +++ b/tests/qemu-iotests/132 @@ -56,4 +56,5 @@ class TestSingleDrive(iotests.QMPTestCase): 'target image does not match source after mirrorin= g') =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['raw', 'qcow2']) + iotests.main(supported_fmts=3D['raw', 'qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139 index 933b45121a..8dc839ec7e 100755 --- a/tests/qemu-iotests/139 +++ b/tests/qemu-iotests/139 @@ -361,4 +361,5 @@ class TestBlockdevDel(iotests.QMPTestCase): =20 =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D["qcow2"]) + iotests.main(supported_fmts=3D["qcow2"], + supported_protocols=3D["file"]) diff --git a/tests/qemu-iotests/148 b/tests/qemu-iotests/148 index e01b061fe7..8c11c53cba 100755 --- a/tests/qemu-iotests/148 +++ b/tests/qemu-iotests/148 @@ -137,4 +137,5 @@ class TestFifoQuorumEvents(TestQuorumEvents): =20 if __name__ =3D=3D '__main__': iotests.verify_quorum() - iotests.main(supported_fmts=3D["raw"]) + iotests.main(supported_fmts=3D["raw"], + supported_protocols=3D["file"]) diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151 index ad7359fc8d..76ae265cc1 100755 --- a/tests/qemu-iotests/151 +++ b/tests/qemu-iotests/151 @@ -142,4 +142,5 @@ class TestActiveMirror(iotests.QMPTestCase): =20 =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2', 'raw']) + iotests.main(supported_fmts=3D['qcow2', 'raw'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/152 b/tests/qemu-iotests/152 index fec546d033..732bf5f062 100755 --- a/tests/qemu-iotests/152 +++ b/tests/qemu-iotests/152 @@ -59,4 +59,5 @@ class TestUnaligned(iotests.QMPTestCase): =20 =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['raw', 'qcow2']) + iotests.main(supported_fmts=3D['raw', 'qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/155 b/tests/qemu-iotests/155 index 63a5b5e2c0..e19485911c 100755 --- a/tests/qemu-iotests/155 +++ b/tests/qemu-iotests/155 @@ -258,4 +258,5 @@ BaseClass =3D None MirrorBaseClass =3D None =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163 index 158ba5d092..081ccc8ac1 100755 --- a/tests/qemu-iotests/163 +++ b/tests/qemu-iotests/163 @@ -170,4 +170,5 @@ class TestShrink1M(ShrinkBaseClass): ShrinkBaseClass =3D None =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['raw', 'qcow2']) + iotests.main(supported_fmts=3D['raw', 'qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165 index 88f62d3c6d..5650dc7c87 100755 --- a/tests/qemu-iotests/165 +++ b/tests/qemu-iotests/165 @@ -103,4 +103,5 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase): self.vm.shutdown() =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index 527aebd0cb..25924d68fc 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -222,4 +222,5 @@ for cmb in list(itertools.product((True, False), repeat= =3D2)): 'do_test_migration_resume_source', *list(cmb)) =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/196 b/tests/qemu-iotests/196 index 4116ebc92b..92fe9244f8 100755 --- a/tests/qemu-iotests/196 +++ b/tests/qemu-iotests/196 @@ -63,4 +63,5 @@ class TestInvalidateAutoclear(iotests.QMPTestCase): self.assertEqual(f.read(1), b'\x00') =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2']) + iotests.main(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index 651e8df5d9..a2c8ecab5a 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -115,4 +115,5 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): self.assert_qmp(result, 'return/sha256', sha256); =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2'], supported_cache_modes=3D['non= e']) + iotests.main(supported_fmts=3D['qcow2'], supported_cache_modes=3D['non= e'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245 index a04c6235c1..fbfd624878 100644 --- a/tests/qemu-iotests/245 +++ b/tests/qemu-iotests/245 @@ -994,4 +994,5 @@ class TestBlockdevReopen(iotests.QMPTestCase): self.reopen(opts, {'backing': 'hd2'}) =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D["qcow2"]) + iotests.main(supported_fmts=3D["qcow2"], + supported_protocols=3D["file"]) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978459; cv=none; d=zoho.com; s=zohoarc; b=h5i7IkKvoiLf2K+6mIJiZGU3n5mmppE5LonXrN8ZSNElRjPUE8snFx09ump8I48UbLMnn88+6LK5I4pCBtxiV+AHpKRbLra/SgXAc22r5Z9DEm0XhDqlYe/g3b6bh3y6VgM2VxiUEqisDzwBXIjyfsBE6czsdZLgf3D4Ew2SmbI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978459; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=FtVrGpGjNxwlaiys7H0I67b/Q1h01VkzgsOtRRKeGLw=; b=g0DrboHuXW4BjZZQ8DeWTuyQpE8d8MQZiSb1eCO6oDMKmfenvDCu0dsfQHqo5ONA/LMOXC5vGtvVeDDY6qSTYMTiKhRYC1na0D4KgRYs1ce6nYJnEtCoegEpVyALgNAlb6tgCzejq3ijnEOdvUu7ogY7X2hUNXVO2Rv/LBqy+10= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978459895598.7004504303849; Tue, 1 Oct 2019 18:07:39 -0700 (PDT) Received: from localhost ([::1]:50028 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT7A-0002aJ-E7 for importer@patchew.org; Tue, 01 Oct 2019 21:07:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40632) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsb-00020l-JF for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002a1-MO for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:11282) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsS-00021H-Bb; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQrm107943; Tue, 1 Oct 2019 19:47:58 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurpcs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:57 -0400 Received: from m0098399.ppops.net (m0098399.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlvqD108674; Tue, 1 Oct 2019 19:47:57 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurpcf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:57 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjmNd029395; Tue, 1 Oct 2019 23:47:56 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fakb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:56 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NltIV4129460 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:55 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 83EFB2805A; Tue, 1 Oct 2019 23:47:55 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6877228058; Tue, 1 Oct 2019 23:47:55 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:55 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 80/97] iotests: Restrict nbd Python tests to nbd Date: Tue, 1 Oct 2019 18:45:59 -0500 Message-Id: <20191001234616.7825-81-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz We have two Python unittest-style tests that test NBD. As such, they should specify supported_protocols=3D['nbd'] so they are skipped when the user wants to test some other protocol. Furthermore, we should restrict their choice of formats to 'raw'. The idea of a protocol/format combination is to use some format over some protocol; but we always use the raw format over NBD. It does not really matter what the NBD server uses on its end, and it is not a useful test of the respective format driver anyway. Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf (cherry picked from commit 7c932a1d69a6d6ac5c0b615c11d191da3bbe9aa8) Signed-off-by: Michael Roth --- tests/qemu-iotests/147 | 5 ++--- tests/qemu-iotests/205 | 3 ++- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 index 82513279b0..ce1f5b459d 100755 --- a/tests/qemu-iotests/147 +++ b/tests/qemu-iotests/147 @@ -287,6 +287,5 @@ class BuiltinNBD(NBDBlockdevAddBase): =20 =20 if __name__ =3D=3D '__main__': - # Need to support image creation - iotests.main(supported_fmts=3D['vpc', 'parallels', 'qcow', 'vdi', 'qco= w2', - 'vmdk', 'raw', 'vhdx', 'qed']) + iotests.main(supported_fmts=3D['raw'], + supported_protocols=3D['nbd']) diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205 index 31b2f5707a..1cf2215730 100755 --- a/tests/qemu-iotests/205 +++ b/tests/qemu-iotests/205 @@ -153,4 +153,5 @@ class TestNbdServerRemove(iotests.QMPTestCase): =20 =20 if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['generic']) + iotests.main(supported_fmts=3D['raw'], + supported_protocols=3D['nbd']) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979343; cv=none; d=zoho.com; s=zohoarc; b=SmZzYefRjQ+BLM8Rea5use/iV9tU+YaBRjvOTT7fyvE1xmwsZB4lEDVeHd3enrozZxHcENesVZNtd9m80W0FOxUftlZUOUa14NqgQ4IesZBcq023A1+fZmL2PFvEWcNNEtWdYlGMJR1qDUshl03jPBhOE+b4lsQlngcI/5Zcf/E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979343; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=hCDEmHrCq/NrnEWZalFdeHAj3wdh39D/9i1RAT1IWrg=; b=BkK3otgRX731F+kDhpBNOFmiCRmV+Qyi4S1cLznhV+mjCxavZuKW2y0KzO5RV+uROrZOrJ2FonwUFdu2RgcbDCVUOgPMxoIA0oX76NswaNv65dkThluw6FzJWITS6gvYMTAyBMmOHn/5ViqEc7yhFyRVEIzsLGevMpxCiMSlTR4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979343690259.6025967111948; Tue, 1 Oct 2019 18:22:23 -0700 (PDT) Received: from localhost ([::1]:50130 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTLR-000850-Sp for importer@patchew.org; Tue, 01 Oct 2019 21:22:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40773) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRse-00024w-H7 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002al-TJ for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:32 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:17894) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsT-000222-Cc; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPoM088139; Tue, 1 Oct 2019 19:47:58 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6c49-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:58 -0400 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlVVp088357; Tue, 1 Oct 2019 19:47:58 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6c40-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:58 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njm6A023587; Tue, 1 Oct 2019 23:47:57 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577h0x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:57 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlu4N23462304 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:56 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 880D728059; Tue, 1 Oct 2019 23:47:56 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 62AE528058; Tue, 1 Oct 2019 23:47:56 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:56 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 81/97] iotests: Test blockdev-create for vpc Date: Tue, 1 Oct 2019 18:46:00 -0500 Message-Id: <20191001234616.7825-82-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf (cherry picked from commit cb73747e1a47b93d3dfdc3f769c576b053916938) Conflicts: tests/qemu-iotests/group *drop context dep. on tests not in 4.0 Signed-off-by: Michael Roth --- tests/qemu-iotests/266 | 153 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/266.out | 137 +++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 291 insertions(+) create mode 100755 tests/qemu-iotests/266 create mode 100644 tests/qemu-iotests/266.out diff --git a/tests/qemu-iotests/266 b/tests/qemu-iotests/266 new file mode 100755 index 0000000000..5b35cd67e4 --- /dev/null +++ b/tests/qemu-iotests/266 @@ -0,0 +1,153 @@ +#!/usr/bin/env python +# +# Test VPC and file image creation +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import iotests +from iotests import imgfmt + + +def blockdev_create(vm, options): + result =3D vm.qmp_log('blockdev-create', job_id=3D'job0', options=3Dop= tions, + filters=3D[iotests.filter_qmp_testfiles]) + + if 'return' in result: + assert result['return'] =3D=3D {} + vm.run_job('job0') + + +# Successful image creation (defaults) +def implicit_defaults(vm, file_path): + iotests.log("=3D=3D=3D Successful image creation (defaults) =3D=3D=3D") + iotests.log("") + + # 8 heads, 964 cyls/head, 17 secs/cyl + # (Close to 64 MB) + size =3D 8 * 964 * 17 * 512 + + blockdev_create(vm, { 'driver': imgfmt, + 'file': 'protocol-node', + 'size': size }) + + +# Successful image creation (explicit defaults) +def explicit_defaults(vm, file_path): + iotests.log("=3D=3D=3D Successful image creation (explicit defaults) = =3D=3D=3D") + iotests.log("") + + # 16 heads, 964 cyls/head, 17 secs/cyl + # (Close to 128 MB) + size =3D 16 * 964 * 17 * 512 + + blockdev_create(vm, { 'driver': imgfmt, + 'file': 'protocol-node', + 'size': size, + 'subformat': 'dynamic', + 'force-size': False }) + + +# Successful image creation (non-default options) +def non_defaults(vm, file_path): + iotests.log("=3D=3D=3D Successful image creation (non-default options)= =3D=3D=3D") + iotests.log("") + + # Not representable in CHS (fine with force-size=3DTrue) + size =3D 1048576 + + blockdev_create(vm, { 'driver': imgfmt, + 'file': 'protocol-node', + 'size': size, + 'subformat': 'fixed', + 'force-size': True }) + + +# Size not representable in CHS with force-size=3DFalse +def non_chs_size_without_force(vm, file_path): + iotests.log("=3D=3D=3D Size not representable in CHS =3D=3D=3D") + iotests.log("") + + # Not representable in CHS (will not work with force-size=3DFalse) + size =3D 1048576 + + blockdev_create(vm, { 'driver': imgfmt, + 'file': 'protocol-node', + 'size': size, + 'force-size': False }) + + +# Zero size +def zero_size(vm, file_path): + iotests.log("=3D=3D=3D Zero size=3D=3D=3D") + iotests.log("") + + blockdev_create(vm, { 'driver': imgfmt, + 'file': 'protocol-node', + 'size': 0 }) + + +# Maximum CHS size +def maximum_chs_size(vm, file_path): + iotests.log("=3D=3D=3D Maximum CHS size=3D=3D=3D") + iotests.log("") + + blockdev_create(vm, { 'driver': imgfmt, + 'file': 'protocol-node', + 'size': 16 * 65535 * 255 * 512 }) + + +# Actual maximum size +def maximum_size(vm, file_path): + iotests.log("=3D=3D=3D Actual maximum size=3D=3D=3D") + iotests.log("") + + blockdev_create(vm, { 'driver': imgfmt, + 'file': 'protocol-node', + 'size': 0xff000000 * 512, + 'force-size': True }) + + +def main(): + for test_func in [implicit_defaults, explicit_defaults, non_defaults, + non_chs_size_without_force, zero_size, maximum_chs_s= ize, + maximum_size]: + + with iotests.FilePath('t.vpc') as file_path, \ + iotests.VM() as vm: + + vm.launch() + + iotests.log('--- Creating empty file ---') + blockdev_create(vm, { 'driver': 'file', + 'filename': file_path, + 'size': 0 }) + + vm.qmp_log('blockdev-add', driver=3D'file', filename=3Dfile_pa= th, + node_name=3D'protocol-node', + filters=3D[iotests.filter_qmp_testfiles]) + iotests.log('') + + print_info =3D test_func(vm, file_path) + iotests.log('') + + vm.shutdown() + iotests.img_info_log(file_path) + + +iotests.script_main(main, + supported_fmts=3D['vpc'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/266.out b/tests/qemu-iotests/266.out new file mode 100644 index 0000000000..b11953e81f --- /dev/null +++ b/tests/qemu-iotests/266.out @@ -0,0 +1,137 @@ +--- Creating empty file --- +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "file", "filename": "TEST_DIR/PID-t.vpc", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": {"driver": "file", "filename": "T= EST_DIR/PID-t.vpc", "node-name": "protocol-node"}} +{"return": {}} + +=3D=3D=3D Successful image creation (defaults) =3D=3D=3D + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "vpc", "file": "protocol-node", "size": 67125248}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +image: TEST_IMG +file format: IMGFMT +virtual size: 64 MiB (67125248 bytes) +cluster_size: 2097152 + +--- Creating empty file --- +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "file", "filename": "TEST_DIR/PID-t.vpc", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": {"driver": "file", "filename": "T= EST_DIR/PID-t.vpc", "node-name": "protocol-node"}} +{"return": {}} + +=3D=3D=3D Successful image creation (explicit defaults) =3D=3D=3D + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "vpc", "file": "protocol-node", "force-size": false, "size": 134= 250496, "subformat": "dynamic"}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +image: TEST_IMG +file format: IMGFMT +virtual size: 128 MiB (134250496 bytes) +cluster_size: 2097152 + +--- Creating empty file --- +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "file", "filename": "TEST_DIR/PID-t.vpc", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": {"driver": "file", "filename": "T= EST_DIR/PID-t.vpc", "node-name": "protocol-node"}} +{"return": {}} + +=3D=3D=3D Successful image creation (non-default options) =3D=3D=3D + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "vpc", "file": "protocol-node", "force-size": true, "size": 1048= 576, "subformat": "fixed"}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +image: TEST_IMG +file format: IMGFMT +virtual size: 1 MiB (1048576 bytes) + +--- Creating empty file --- +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "file", "filename": "TEST_DIR/PID-t.vpc", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": {"driver": "file", "filename": "T= EST_DIR/PID-t.vpc", "node-name": "protocol-node"}} +{"return": {}} + +=3D=3D=3D Size not representable in CHS =3D=3D=3D + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "vpc", "file": "protocol-node", "force-size": false, "size": 104= 8576}}} +{"return": {}} +Job failed: The requested image size cannot be represented in CHS geometry +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +qemu-img: Could not open 'TEST_IMG': File too small for a VHD header + +--- Creating empty file --- +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "file", "filename": "TEST_DIR/PID-t.vpc", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": {"driver": "file", "filename": "T= EST_DIR/PID-t.vpc", "node-name": "protocol-node"}} +{"return": {}} + +=3D=3D=3D Zero size=3D=3D=3D + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "vpc", "file": "protocol-node", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +image: TEST_IMG +file format: IMGFMT +virtual size: 0 B (0 bytes) +cluster_size: 2097152 + +--- Creating empty file --- +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "file", "filename": "TEST_DIR/PID-t.vpc", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": {"driver": "file", "filename": "T= EST_DIR/PID-t.vpc", "node-name": "protocol-node"}} +{"return": {}} + +=3D=3D=3D Maximum CHS size=3D=3D=3D + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "vpc", "file": "protocol-node", "size": 136899993600}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +image: TEST_IMG +file format: IMGFMT +virtual size: 127 GiB (136899993600 bytes) +cluster_size: 2097152 + +--- Creating empty file --- +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "file", "filename": "TEST_DIR/PID-t.vpc", "size": 0}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": {"driver": "file", "filename": "T= EST_DIR/PID-t.vpc", "node-name": "protocol-node"}} +{"return": {}} + +=3D=3D=3D Actual maximum size=3D=3D=3D + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": = {"driver": "vpc", "file": "protocol-node", "force-size": true, "size": 2190= 433320960}}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +image: TEST_IMG +file format: IMGFMT +virtual size: 1.99 TiB (2190433320960 bytes) +cluster_size: 2097152 + diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 7aa6dc9875..3605796bb2 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -251,3 +251,4 @@ 253 rw auto quick 255 rw auto quick 256 rw auto quick +266 rw quick --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980138; cv=none; d=zoho.com; s=zohoarc; b=EW1sXIp/F2I1SxqQJZhZX86N/fsWsa1irLQp/F9BIEFs2wGm1SH9lryLbrgdeQDuagfpq7fQ2pIdEjqj19z2Exsdqdvts4PwGSrNS91R8oUB7xnZu1yZ0z5CANfMfC1b4YSS8Z9tlWl387QK5oLJoFaUffx50xBOe55UPU9m9lQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980138; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=lsC17UK+CldjA78QgdqKVihE4s+AiaE69IEffuNo68g=; b=Bpr4HHGYY7vHa0Q36uRGqi9QYHD19TyANf8NOUihulRVGEokWC2fieYgMm1b6SK7/txqwK6WrI9pt3rXUqqMuB27/6hnFTY6CbOsvyAPg7HKqBLtqKTZVkeiMaztxu4OuvQXMhYDKZNM/JWgRqK9xZI4Cr2cOprHuFlAOcQi9no= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15699801386621005.4596453286897; Tue, 1 Oct 2019 18:35:38 -0700 (PDT) Received: from localhost ([::1]:50246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTYH-00053Y-9R for importer@patchew.org; Tue, 01 Oct 2019 21:35:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40716) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsd-00023R-BS for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002ab-Px for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:31 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:20604) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsV-00023G-Sr; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlQIw074864; Tue, 1 Oct 2019 19:47:59 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq50-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:59 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlwlR076006; Tue, 1 Oct 2019 19:47:58 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq4f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:58 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njod9031648; Tue, 1 Oct 2019 23:47:57 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fd5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:57 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlvXm43123016 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:57 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 10D1A2805E; Tue, 1 Oct 2019 23:47:57 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E8AB328059; Tue, 1 Oct 2019 23:47:56 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:56 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 82/97] libvhost-user: fix SLAVE_SEND_FD handling Date: Tue, 1 Oct 2019 18:46:01 -0500 Message-Id: <20191001234616.7825-83-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Johannes Berg , "Michael S . Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Johannes Berg It doesn't look like this could possibly work properly since VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD is defined to 10, but the dev->protocol_features has a bitmap. I suppose the peer this was tested with also supported VHOST_USER_PROTOCOL_F_LOG_SHMFD, in which case the test would always be false, but nevertheless the code seems wrong. Use has_feature() to fix this. Fixes: d84599f56c82 ("libvhost-user: support host notifier") Signed-off-by: Johannes Berg Message-Id: <20190903200422.11693-1-johannes@sipsolutions.net> Reviewed-by: Tiwei Bie Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin (cherry picked from commit 8726b70b449896f1211f869ec4f608904f027207) Signed-off-by: Michael Roth --- contrib/libvhost-user/libvhost-user.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index e08d6c7b97..516d0de7bb 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -1085,7 +1085,8 @@ bool vu_set_queue_host_notifier(VuDev *dev, VuVirtq *= vq, int fd, =20 vmsg.fd_num =3D fd_num; =20 - if ((dev->protocol_features & VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD) =3D= =3D 0) { + if (!has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD)) { return false; } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978280; cv=none; d=zoho.com; s=zohoarc; b=cnTsJW0Q/hLsMOXM8VAVJV3bxBobhlNaL8JCQoJGm0QRj2Paw5VOagl7WptGey77f3lwv3SiXk40SS9d3QTAmwVtPf5NhHaKNnkpxY6fPPm4kLDc+pBAR0EsZCiRlwtOgZOLn/ll9ZWFtEU4jXSVG6wxrzMro0AbYg6Ufsm9els= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978280; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ZhlM9BeF0WoMXkG5BYkNVoy4EBiYktLnEbr36go6yvc=; b=I0m8+ze2D4SMCG2OWOLtRCcd8qyariQtMgcDClNfz1H0ceG9bC5EQYLnxjMloQ+8SN9rcWzGoD/qD4qNsVygyZm2H7L6lkXcnj9KwDBH8rY5nuf8A1wcezRKO/aSXMHTAn2JlvwZxPoKR5yiIRyaeDPdkt7hqX2+e2sFO/18knE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978280187126.3450728522082; Tue, 1 Oct 2019 18:04:40 -0700 (PDT) Received: from localhost ([::1]:49992 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFT4H-0007xW-5b for importer@patchew.org; Tue, 01 Oct 2019 21:04:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40712) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsd-00023O-AN for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002aZ-PC for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:30 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:42008 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsT-000228-7D; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPZD179350; Tue, 1 Oct 2019 19:47:59 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnd43-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:58 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlwTY180845; Tue, 1 Oct 2019 19:47:58 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcdgpnd3y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:58 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlC8023568; Tue, 1 Oct 2019 23:47:58 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577h13-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:58 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlvgx46596410 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:57 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8E65328059; Tue, 1 Oct 2019 23:47:57 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 713CF2805E; Tue, 1 Oct 2019 23:47:57 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:57 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 83/97] block/create: Do not abort if a block driver is not available Date: Tue, 1 Oct 2019 18:46:02 -0500 Message-Id: <20191001234616.7825-84-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0b-001b2d01.pphosted.com id x91NlPZD179350 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Philippe Mathieu-Daud=C3=A9 The 'blockdev-create' QMP command was introduced as experimental feature in commit b0292b851b8, using the assert() debug call. It got promoted to 'stable' command in 3fb588a0f2c, but the assert call was not removed. Some block drivers are optional, and bdrv_find_format() might return a NULL value, triggering the assertion. Stable code is not expected to abort, so return an error instead. This is easily reproducible when libnfs is not installed: ./configure [...] module support no Block whitelist (rw) Block whitelist (ro) libiscsi support yes libnfs support no [...] Start QEMU: $ qemu-system-x86_64 -S -qmp unix:/tmp/qemu.qmp,server,nowait Send the 'blockdev-create' with the 'nfs' driver: $ ( cat << 'EOF' {'execute': 'qmp_capabilities'} {'execute': 'blockdev-create', 'arguments': {'job-id': 'x', 'options': {'= size': 0, 'driver': 'nfs', 'location': {'path': '/', 'server': {'host': '::= 1', 'type': 'inet'}}}}, 'id': 'x'} EOF ) | socat STDIO UNIX:/tmp/qemu.qmp {"QMP": {"version": {"qemu": {"micro": 50, "minor": 1, "major": 4}, "pack= age": "v4.1.0-733-g89ea03a7dc"}, "capabilities": ["oob"]}} {"return": {}} QEMU crashes: $ gdb qemu-system-x86_64 core Program received signal SIGSEGV, Segmentation fault. (gdb) bt #0 0x00007ffff510957f in raise () at /lib64/libc.so.6 #1 0x00007ffff50f3895 in abort () at /lib64/libc.so.6 #2 0x00007ffff50f3769 in _nl_load_domain.cold.0 () at /lib64/libc.so.6 #3 0x00007ffff5101a26 in .annobin_assert.c_end () at /lib64/libc.so.6 #4 0x0000555555d7e1f1 in qmp_blockdev_create (job_id=3D0x555556baee40 "x= ", options=3D0x555557666610, errp=3D0x7fffffffc770) at block/create.c:69 #5 0x0000555555c96b52 in qmp_marshal_blockdev_create (args=3D0x7fffdc003= 830, ret=3D0x7fffffffc7f8, errp=3D0x7fffffffc7f0) at qapi/qapi-commands-blo= ck-core.c:1314 #6 0x0000555555deb0a0 in do_qmp_dispatch (cmds=3D0x55555645de70 , request=3D0x7fffdc005c70, allow_oob=3Dfalse, errp=3D0x7fffffffc898)= at qapi/qmp-dispatch.c:131 #7 0x0000555555deb2a1 in qmp_dispatch (cmds=3D0x55555645de70 , request=3D0x7fffdc005c70, allow_oob=3Dfalse) at qapi/qmp-dispatch.c:174 With this patch applied, QEMU returns a QMP error: {'execute': 'blockdev-create', 'arguments': {'job-id': 'x', 'options': {'= size': 0, 'driver': 'nfs', 'location': {'path': '/', 'server': {'host': '::= 1', 'type': 'inet'}}}}, 'id': 'x'} {"id": "x", "error": {"class": "GenericError", "desc": "Block driver 'nfs= ' not found or not supported"}} Cc: qemu-stable@nongnu.org Reported-by: Xu Tian Signed-off-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Eric Blake Reviewed-by: John Snow Signed-off-by: Kevin Wolf (cherry picked from commit d90d5cae2b10efc0e8d0b3cc91ff16201853d3ba) Signed-off-by: Michael Roth --- block/create.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/block/create.c b/block/create.c index 95341219ef..de5e97bb18 100644 --- a/block/create.c +++ b/block/create.c @@ -63,9 +63,13 @@ void qmp_blockdev_create(const char *job_id, BlockdevCre= ateOptions *options, const char *fmt =3D BlockdevDriver_str(options->driver); BlockDriver *drv =3D bdrv_find_format(fmt); =20 + if (!drv) { + error_setg(errp, "Block driver '%s' not found or not supported", f= mt); + return; + } + /* If the driver is in the schema, we know that it exists. But it may = not * be whitelisted. */ - assert(drv); if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) { error_setg(errp, "Driver is not whitelisted"); return; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979230; cv=none; d=zoho.com; s=zohoarc; b=nUQ5ud+P2Qmfc53rvXa9G2vj7nhvILox8ujfBCkpVtxXAcK5b9cISoXnleHJD9r/+n24T/J1HMgUdErUNXrumg+NwhsPJxwM5S84WUuRBt6pmIK/k00dd9t4IylmMCRouAYEqL779o4SY7huJPf1Wn9yUuFiDicGDQ+G7oVi59Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979230; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=gcwz+gwbqsJGCCPur0x56WGq/jH1RbiJuFHovnhoajg=; b=hcXQneXeEy9ZHCaSbn3l2V9GfoaibwFiI/k0HNiEuYFRkjuok17VH5iBh4GFOcnzNhfRupeYaNRvgR+3EKVDiRgQ6lEkDAlSNx0U9nTeLUzmxHHVezRXZG6v/hCgOOS6h4FNw7o/GH6p2Xz6l9V0iLrUkd5VRqb7kirHHfLZ2hI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979230092737.0055007702397; Tue, 1 Oct 2019 18:20:30 -0700 (PDT) Received: from localhost ([::1]:50120 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTJb-0006Y5-S6 for importer@patchew.org; Tue, 01 Oct 2019 21:20:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41280) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsp-0002ED-2Y for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsh-0002uQ-NM for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:62822) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-0002ES-Ih; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlVHM108108; Tue, 1 Oct 2019 19:48:00 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurpde-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:00 -0400 Received: from m0098399.ppops.net (m0098399.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlxN3108734; Tue, 1 Oct 2019 19:47:59 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurpd3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:47:59 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlTQ023573; Tue, 1 Oct 2019 23:47:58 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577h15-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:58 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlw4t53215492 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:58 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 13AFF2805C; Tue, 1 Oct 2019 23:47:58 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EB9FD28059; Tue, 1 Oct 2019 23:47:57 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:57 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 84/97] block/nfs: tear down aio before nfs_close Date: Tue, 1 Oct 2019 18:46:03 -0500 Message-Id: <20191001234616.7825-85-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=889 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Lieven , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Peter Lieven nfs_close is a sync call from libnfs and has its own event handler polling on the nfs FD. Avoid that both QEMU and libnfs are intefering here. CC: qemu-stable@nongnu.org Signed-off-by: Peter Lieven Signed-off-by: Kevin Wolf (cherry picked from commit 601dc6559725f7a614b6f893611e17ff0908e914) Signed-off-by: Michael Roth --- block/nfs.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/block/nfs.c b/block/nfs.c index 531903610b..f04f675c63 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -389,12 +389,14 @@ static void nfs_attach_aio_context(BlockDriverState *= bs, static void nfs_client_close(NFSClient *client) { if (client->context) { + qemu_mutex_lock(&client->mutex); + aio_set_fd_handler(client->aio_context, nfs_get_fd(client->context= ), + false, NULL, NULL, NULL, NULL); + qemu_mutex_unlock(&client->mutex); if (client->fh) { nfs_close(client->context, client->fh); client->fh =3D NULL; } - aio_set_fd_handler(client->aio_context, nfs_get_fd(client->context= ), - false, NULL, NULL, NULL, NULL); nfs_destroy_context(client->context); client->context =3D NULL; } --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980395; cv=none; d=zoho.com; s=zohoarc; b=Lhp5Nw5+TQBRjkhi16mwfpBmyRDwSM7qOax791zNMnHyqxRXucGRfTVxGmCNb/Z70zoR0YKQHzTbf23/jDpq4yg5tbyHoSeHbH/ljtRmBKcYDzcjU2X8xAkGy1uahMXzXTn5Vg4ep7N0vPzAvGzHK/HV2ya6otrVeQiSYonkl+s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980395; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=UGg1YGaR3DAHlqDWdNshyW02BVx6kQ0gLN6GTknYjpo=; b=SNgr0WywFI0leJMulKeO/CXjDArJ9RNyJMQFx5xym/73Ha8fYzVl58WwRfvKJjMNBxE4pclQhcmBy1lY43cxMp6Csu444/zlSL89RQAPLHYjR8ncwAjDo8MLsGttM4Pb89YZi2wqgXNNlIo7B00fAjCX6n+gTRTeDAkSVP6dK74= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569980395190953.3761998722938; Tue, 1 Oct 2019 18:39:55 -0700 (PDT) Received: from localhost ([::1]:50298 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTcK-0001TI-OD for importer@patchew.org; Tue, 01 Oct 2019 21:39:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40729) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsd-00023g-Jw for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002b0-Sm for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:31 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:20292) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsV-00023C-So; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlP3r074782; Tue, 1 Oct 2019 19:48:00 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq5j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:00 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nm0ae076040; Tue, 1 Oct 2019 19:48:00 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq53-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjqmL001450; Tue, 1 Oct 2019 23:47:59 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma03wdc.us.ibm.com with ESMTP id 2v9y56yf8y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:59 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlwnP53674338 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:58 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8EBD928060; Tue, 1 Oct 2019 23:47:58 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 72FCA2805A; Tue, 1 Oct 2019 23:47:58 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:58 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 85/97] blockjob: update nodes head while removing all bdrv Date: Tue, 1 Oct 2019 18:46:04 -0500 Message-Id: <20191001234616.7825-86-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Sergio Lopez , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Sergio Lopez block_job_remove_all_bdrv() iterates through job->nodes, calling bdrv_root_unref_child() for each entry. The call to the latter may reach child_job_[can_]set_aio_ctx(), which will also attempt to traverse job->nodes, potentially finding entries that where freed on previous iterations. To avoid this situation, update job->nodes head on each iteration to ensure that already freed entries are no longer linked to the list. RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=3D1746631 Signed-off-by: Sergio Lopez Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20190911100316.32282-1-mreitz@redhat.com Reviewed-by: Sergio Lopez Signed-off-by: Max Reitz (cherry picked from commit d876bf676f5e7c6aa9ac64555e48cba8734ecb2f) Signed-off-by: Michael Roth --- blockjob.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/blockjob.c b/blockjob.c index 730101d282..d770144fd6 100644 --- a/blockjob.c +++ b/blockjob.c @@ -193,14 +193,23 @@ static const BdrvChildRole child_job =3D { =20 void block_job_remove_all_bdrv(BlockJob *job) { - GSList *l; - for (l =3D job->nodes; l; l =3D l->next) { + /* + * bdrv_root_unref_child() may reach child_job_[can_]set_aio_ctx(), + * which will also traverse job->nodes, so consume the list one by + * one to make sure that such a concurrent access does not attempt + * to process an already freed BdrvChild. + */ + while (job->nodes) { + GSList *l =3D job->nodes; BdrvChild *c =3D l->data; + + job->nodes =3D l->next; + bdrv_op_unblock_all(c->bs, job->blocker); bdrv_root_unref_child(c); + + g_slist_free_1(l); } - g_slist_free(job->nodes); - job->nodes =3D NULL; } =20 int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *= bs, --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569981227; cv=none; d=zoho.com; s=zohoarc; b=IXt/SfX1zt74zxjsCbKt2366IwUhq0bMvaNxiYLnj8fLXod2d8EgEUV/eFeQkZVRMHYEFG84iniAf3b5jkr6WR1cJcb68dV8dy92Pwn+fRLmWPAdr135VJr13inML7BqaUu3kJoZ6v4B8MUZ2+jvR864MGpBgluwH4SGDM2a3Cc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569981227; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=0WILvfkxqjdAu+C/42IDiMNu9Qr0g7SeDOS2hBCOdOM=; b=PsD06kqa9JPxPsVbkQE6VFxKZenv/voG4xY7aFRoucy998J6T2v/NX5WojIGa0w4Xz8Y/LKts4z6lXCdBKFAsJTamcZ5KFjLIZPqQ1Ztz2X/i6sgOUEdtzP4v0qpPY/gRzBX48a5xTMcAz+D8GAmRN8D7GJ1Hud/zXC8dMp9bR0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569981227597122.86345989020117; Tue, 1 Oct 2019 18:53:47 -0700 (PDT) Received: from localhost ([::1]:50412 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTpl-0006ai-1v for importer@patchew.org; Tue, 01 Oct 2019 21:53:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40688) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsc-00022a-UR for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002ao-RB for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:30 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:30942) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsW-00024s-4C; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlRcj086566; Tue, 1 Oct 2019 19:48:01 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat813s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:01 -0400 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nm0VQ087433; Tue, 1 Oct 2019 19:48:00 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vccat813g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:00 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlUF029385; Tue, 1 Oct 2019 23:47:59 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fakm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:47:59 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nlx9V45089114 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:59 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 09D1B28060; Tue, 1 Oct 2019 23:47:59 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E75712805A; Tue, 1 Oct 2019 23:47:58 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:58 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 86/97] curl: Keep pointer to the CURLState in CURLSocket Date: Tue, 1 Oct 2019 18:46:05 -0500 Message-Id: <20191001234616.7825-87-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz A follow-up patch will make curl_multi_do() and curl_multi_read() take a CURLSocket instead of the CURLState. They still need the latter, though, so add a pointer to it to the former. Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Reviewed-by: John Snow Message-id: 20190910124136.10565-2-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz (cherry picked from commit 0487861685294660b23bc146e1ebd5304aa8bbe0) Signed-off-by: Michael Roth --- block/curl.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/curl.c b/block/curl.c index 606709fea4..4eaae9e211 100644 --- a/block/curl.c +++ b/block/curl.c @@ -79,6 +79,7 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_= handle, #define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5 =20 struct BDRVCURLState; +struct CURLState; =20 static bool libcurl_initialized; =20 @@ -96,6 +97,7 @@ typedef struct CURLAIOCB { =20 typedef struct CURLSocket { int fd; + struct CURLState *state; QLIST_ENTRY(CURLSocket) next; } CURLSocket; =20 @@ -179,6 +181,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, i= nt action, if (!socket) { socket =3D g_new0(CURLSocket, 1); socket->fd =3D fd; + socket->state =3D state; QLIST_INSERT_HEAD(&state->sockets, socket, next); } socket =3D NULL; --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980619; cv=none; d=zoho.com; s=zohoarc; b=SOkScxC75BHGXSPP+ze0YE7OSS5l2ckCIuemRQYRTaIMFKobVvfjshKBZqE/dc6bsVaW7/zy6/8+bmbfONTKQDc3GtDkpp8VEDUzUoEX2eIfegogHhcwqjZSrwQwgtiDnsuYQU4M7QnWANRfqxQH4f5TXHpjBijYYhgqZ8/+6d4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980619; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=8K5tgvX2oUGdN6FNXau0ocLbabzJ6qDu3Wngz3VwEBg=; b=PBk6ToNS69SQemmrvEybYw3W53Bl2LKNmPWLUhiQT5d7nLyaYVApojnAVepXk3zJvyzctrRQ/DLtyrLMB6+3jgrUVTBB4AUPoNtumhNx56+ha9hZxUZ6lzfcgaPHORAJbrbD7QwGbVUEFiLaHekgxnTd1C0GOygw+ZXQUVePH/0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569980619742470.4067056263938; Tue, 1 Oct 2019 18:43:39 -0700 (PDT) Received: from localhost ([::1]:50328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTg1-0005Oz-TX for importer@patchew.org; Tue, 01 Oct 2019 21:43:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40770) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRse-00024t-G8 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002bH-SH for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:32 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:34394) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsV-00025n-Sq; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlP0V088131; Tue, 1 Oct 2019 19:48:02 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6c5j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:02 -0400 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlaS9088480; Tue, 1 Oct 2019 19:48:01 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vbsjt6c52-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:01 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlKw027161; Tue, 1 Oct 2019 23:48:00 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bjr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:00 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91NlxtF45089126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:47:59 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 856A528064; Tue, 1 Oct 2019 23:47:59 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 72D0A2805E; Tue, 1 Oct 2019 23:47:59 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:59 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 87/97] curl: Keep *socket until the end of curl_sock_cb() Date: Tue, 1 Oct 2019 18:46:06 -0500 Message-Id: <20191001234616.7825-88-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz This does not really change anything, but it makes the code a bit easier to follow once we use @socket as the opaque pointer for aio_set_fd_handler(). Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20190910124136.10565-3-mreitz@redhat.com Reviewed-by: Maxim Levitsky Reviewed-by: John Snow Signed-off-by: Max Reitz (cherry picked from commit 007f339b1099af46a008dac438ca0943e31dba72) Signed-off-by: Michael Roth --- block/curl.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/block/curl.c b/block/curl.c index 4eaae9e211..a0381ae0b4 100644 --- a/block/curl.c +++ b/block/curl.c @@ -171,10 +171,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, = int action, =20 QLIST_FOREACH(socket, &state->sockets, next) { if (socket->fd =3D=3D fd) { - if (action =3D=3D CURL_POLL_REMOVE) { - QLIST_REMOVE(socket, next); - g_free(socket); - } break; } } @@ -184,7 +180,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, i= nt action, socket->state =3D state; QLIST_INSERT_HEAD(&state->sockets, socket, next); } - socket =3D NULL; =20 trace_curl_sock_cb(action, (int)fd); switch (action) { @@ -206,6 +201,11 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, = int action, break; } =20 + if (action =3D=3D CURL_POLL_REMOVE) { + QLIST_REMOVE(socket, next); + g_free(socket); + } + return 0; } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978794; cv=none; d=zoho.com; s=zohoarc; b=d/IhnmRc76f/ztlkTVZPtUZcF9HP+hZnwo66xrV1cvz7qB+U7tXAOQ6fUtt1e4I2OFgKRyqokbsFjVnVcMGQu3vmJbxGfS62+shzMBkMaWh/LFyxXlTcnhUhQ6rS7H434XLSOoUwWCx2rNPVfsQqWdhP5onMy4c+m48oEuotr84= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978794; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=hJgTfyPP4M6Mu9vOuv/21B2Kkf0UmB+r5Gr9IYu9lII=; b=oAG5AqJ6WOVCkuNBmZLpo0aKWR0CqriIvwJIXuinrCtNKgyfs36OGM+D4hKZGrw3WmSyLoM9yU8TzAqFMfXNmxNPYJmbP+EwNq+ULx+FaYbKOcAJd7jfaNBzROgrCHvd3hfsDkYPBxvwuEJ+JS/5nxPwyAdReHhyymAr1IHJJQE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978794378931.9629927596757; Tue, 1 Oct 2019 18:13:14 -0700 (PDT) Received: from localhost ([::1]:50074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTCZ-0007gS-1e for importer@patchew.org; Tue, 01 Oct 2019 21:13:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40743) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsd-000249-Vl for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002b8-Sd for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:31 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:35726 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsV-000254-Sp; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlUI8146403; Tue, 1 Oct 2019 19:48:01 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26nx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:01 -0400 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlR1g146347; Tue, 1 Oct 2019 19:48:01 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26nr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:01 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlrK001420; Tue, 1 Oct 2019 23:48:00 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03wdc.us.ibm.com with ESMTP id 2v9y56yf94-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:00 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm03Z41746756 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:00 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1977728059; Tue, 1 Oct 2019 23:48:00 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F197528058; Tue, 1 Oct 2019 23:47:59 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:47:59 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 88/97] curl: Check completion in curl_multi_do() Date: Tue, 1 Oct 2019 18:46:07 -0500 Message-Id: <20191001234616.7825-89-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz While it is more likely that transfers complete after some file descriptor has data ready to read, we probably should not rely on it. Better be safe than sorry and call curl_multi_check_completion() in curl_multi_do(), too, just like it is done in curl_multi_read(). With this change, curl_multi_do() and curl_multi_read() are actually the same, so drop curl_multi_read() and use curl_multi_do() as the sole FD handler. Signed-off-by: Max Reitz Message-id: 20190910124136.10565-4-mreitz@redhat.com Reviewed-by: Maxim Levitsky Reviewed-by: John Snow Signed-off-by: Max Reitz (cherry picked from commit 948403bcb1c7e71dcbe8ab8479cf3934a0efcbb5) Signed-off-by: Michael Roth --- block/curl.c | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/block/curl.c b/block/curl.c index a0381ae0b4..bf64c2a0db 100644 --- a/block/curl.c +++ b/block/curl.c @@ -138,7 +138,6 @@ typedef struct BDRVCURLState { =20 static void curl_clean_state(CURLState *s); static void curl_multi_do(void *arg); -static void curl_multi_read(void *arg); =20 #ifdef NEED_CURL_TIMER_CALLBACK /* Called from curl_multi_do_locked, with s->mutex held. */ @@ -185,7 +184,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, i= nt action, switch (action) { case CURL_POLL_IN: aio_set_fd_handler(s->aio_context, fd, false, - curl_multi_read, NULL, NULL, state); + curl_multi_do, NULL, NULL, state); break; case CURL_POLL_OUT: aio_set_fd_handler(s->aio_context, fd, false, @@ -193,7 +192,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, i= nt action, break; case CURL_POLL_INOUT: aio_set_fd_handler(s->aio_context, fd, false, - curl_multi_read, curl_multi_do, NULL, state= ); + curl_multi_do, curl_multi_do, NULL, state); break; case CURL_POLL_REMOVE: aio_set_fd_handler(s->aio_context, fd, false, @@ -415,15 +414,6 @@ static void curl_multi_do(void *arg) { CURLState *s =3D (CURLState *)arg; =20 - qemu_mutex_lock(&s->s->mutex); - curl_multi_do_locked(s); - qemu_mutex_unlock(&s->s->mutex); -} - -static void curl_multi_read(void *arg) -{ - CURLState *s =3D (CURLState *)arg; - qemu_mutex_lock(&s->s->mutex); curl_multi_do_locked(s); curl_multi_check_completion(s->s); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979073; cv=none; d=zoho.com; s=zohoarc; b=Gk2+/jyt2jzSxIS1GumflBq1ic9YG91EbO9HQB53wDVxlFkxzI9eH+dTgyE0eD16FT/k660g9/Xs1xkZnjSzaBfoOORZBhCRLoOxQcoF/DZ87pZfZHJxTk7StI2ViGXpGp02JELiBdVIL8Hwk+aIfT/2IKdw6yhoDOsteUcpePA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979073; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Dg6cyZ6e40+lIWLUTzWNhJ4VY8LclpvmU3rPig3XzSI=; b=PTxt7hB4D2Y3NjClb1YrVNvyBrFWrp+/wZjmmTc3gVKQ4mItuweD6Asy0EPA5o7gnOPEx0w0MlHxCRaN58QMMfL4aHHmqj4P51HyzRUYdLOuF7FTCE4m3dprf5/Lyom4JZ6qpA+dg5/71RbT7SHGmTXXuQFcy004TxHzxW69kGo= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979073176508.15690969106504; Tue, 1 Oct 2019 18:17:53 -0700 (PDT) Received: from localhost ([::1]:50104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTH4-0003kv-VU for importer@patchew.org; Tue, 01 Oct 2019 21:17:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40762) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRse-00024i-Dt for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsW-0002au-RK for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:31 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:38390) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsV-00025Q-UA; Tue, 01 Oct 2019 19:48:24 -0400 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlNmB193547; Tue, 1 Oct 2019 19:48:02 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsgd6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:01 -0400 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nlvg1194375; Tue, 1 Oct 2019 19:48:01 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vc71qsgcv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:01 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njofl032175; Tue, 1 Oct 2019 23:48:02 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma01wdc.us.ibm.com with ESMTP id 2v9y57fgct-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:02 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm0f149676640 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:00 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9548328059; Tue, 1 Oct 2019 23:48:00 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 79A2F2805C; Tue, 1 Oct 2019 23:48:00 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:00 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 89/97] curl: Pass CURLSocket to curl_multi_do() Date: Tue, 1 Oct 2019 18:46:08 -0500 Message-Id: <20191001234616.7825-90-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz curl_multi_do_locked() currently marks all sockets as ready. That is not only inefficient, but in fact unsafe (the loop is). A follow-up patch will change that, but to do so, curl_multi_do_locked() needs to know exactly which socket is ready; and that is accomplished by this patch here. Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20190910124136.10565-5-mreitz@redhat.com Reviewed-by: Maxim Levitsky Reviewed-by: John Snow Signed-off-by: Max Reitz (cherry picked from commit 9dbad87d25587ff640ef878f7b6159fc368ff541) Signed-off-by: Michael Roth --- block/curl.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/block/curl.c b/block/curl.c index bf64c2a0db..5b163d71dc 100644 --- a/block/curl.c +++ b/block/curl.c @@ -184,15 +184,15 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd,= int action, switch (action) { case CURL_POLL_IN: aio_set_fd_handler(s->aio_context, fd, false, - curl_multi_do, NULL, NULL, state); + curl_multi_do, NULL, NULL, socket); break; case CURL_POLL_OUT: aio_set_fd_handler(s->aio_context, fd, false, - NULL, curl_multi_do, NULL, state); + NULL, curl_multi_do, NULL, socket); break; case CURL_POLL_INOUT: aio_set_fd_handler(s->aio_context, fd, false, - curl_multi_do, curl_multi_do, NULL, state); + curl_multi_do, curl_multi_do, NULL, socket); break; case CURL_POLL_REMOVE: aio_set_fd_handler(s->aio_context, fd, false, @@ -391,9 +391,10 @@ static void curl_multi_check_completion(BDRVCURLState = *s) } =20 /* Called with s->mutex held. */ -static void curl_multi_do_locked(CURLState *s) +static void curl_multi_do_locked(CURLSocket *ready_socket) { CURLSocket *socket, *next_socket; + CURLState *s =3D ready_socket->state; int running; int r; =20 @@ -412,12 +413,13 @@ static void curl_multi_do_locked(CURLState *s) =20 static void curl_multi_do(void *arg) { - CURLState *s =3D (CURLState *)arg; + CURLSocket *socket =3D arg; + BDRVCURLState *s =3D socket->state->s; =20 - qemu_mutex_lock(&s->s->mutex); - curl_multi_do_locked(s); - curl_multi_check_completion(s->s); - qemu_mutex_unlock(&s->s->mutex); + qemu_mutex_lock(&s->mutex); + curl_multi_do_locked(socket); + curl_multi_check_completion(s); + qemu_mutex_unlock(&s->mutex); } =20 static void curl_multi_timeout_do(void *arg) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979561; cv=none; d=zoho.com; s=zohoarc; b=Q8GQwwWsjOw26qMo0+T6P6ZrMqcrcxTvJEScMLeNBPKEaYRYwQ52Kx3eV2HX8gaxL7BSvRUJBAa/LwoOwYiGVzgF78Z1LhUbCK4ZeGmYArLHCJAl/ZWeXowEVTS8THkCVBfsohPlg1F91exEuWnZQRNhrG1zRYCHJSN/EXd68Ds= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979561; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=F/YTXQiZpI3R5jFDwl5OjsID9xcPxe9Oh5Ru5yElJ7g=; b=Z3NMkkFek2u67xjRUuQGFI90/wFov3YgkMkuDKSndVSQAdU19GqQnyu7zr+moXHvJudkX0nkN9rpXKWORLrJvEfylP2TsZpE+olGl5lGZZVI3R2R5QsjER4WaNirCTsuCtzJk6Hx7ksU0zwr8T+XVjYPszKjEL3cSGF4+tDV31M= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979561415971.6853736057012; Tue, 1 Oct 2019 18:26:01 -0700 (PDT) Received: from localhost ([::1]:50162 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTOx-0003TG-Ks for importer@patchew.org; Tue, 01 Oct 2019 21:25:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41271) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsp-0002E1-3n for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsh-0002uv-Rg for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:40762) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-00027e-Ri; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlPM2107929; Tue, 1 Oct 2019 19:48:03 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurpeg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:03 -0400 Received: from m0098399.ppops.net (m0098399.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nm2Nk108791; Tue, 1 Oct 2019 19:48:02 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcgaurpe6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:02 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlZ8029380; Tue, 1 Oct 2019 23:48:01 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fakq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:01 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm1RW47513998 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:01 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 11E352805E; Tue, 1 Oct 2019 23:48:01 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EA8F628059; Tue, 1 Oct 2019 23:48:00 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:00 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 90/97] curl: Report only ready sockets Date: Tue, 1 Oct 2019 18:46:09 -0500 Message-Id: <20191001234616.7825-91-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Instead of reporting all sockets to cURL, only report the one that has caused curl_multi_do_locked() to be called. This lets us get rid of the QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are only safe when the current element is removed in each iteration. If it possible for the list to be concurrently modified, we cannot guarantee that only the current element will be removed. Therefore, we must not use QLIST_FOREACH_SAFE() here. Fixes: ff5ca1664af85b24a4180d595ea6873fd3deac57 Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20190910124136.10565-6-mreitz@redhat.com Reviewed-by: Maxim Levitsky Reviewed-by: John Snow Signed-off-by: Max Reitz (cherry picked from commit 9abaf9fc474c3dd53e8e119326abc774c977c331) Signed-off-by: Michael Roth --- block/curl.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/block/curl.c b/block/curl.c index 5b163d71dc..de0cebd361 100644 --- a/block/curl.c +++ b/block/curl.c @@ -391,24 +391,19 @@ static void curl_multi_check_completion(BDRVCURLState= *s) } =20 /* Called with s->mutex held. */ -static void curl_multi_do_locked(CURLSocket *ready_socket) +static void curl_multi_do_locked(CURLSocket *socket) { - CURLSocket *socket, *next_socket; - CURLState *s =3D ready_socket->state; + BDRVCURLState *s =3D socket->state->s; int running; int r; =20 - if (!s->s->multi) { + if (!s->multi) { return; } =20 - /* Need to use _SAFE because curl_multi_socket_action() may trigger - * curl_sock_cb() which might modify this list */ - QLIST_FOREACH_SAFE(socket, &s->sockets, next, next_socket) { - do { - r =3D curl_multi_socket_action(s->s->multi, socket->fd, 0, &ru= nning); - } while (r =3D=3D CURLM_CALL_MULTI_PERFORM); - } + do { + r =3D curl_multi_socket_action(s->multi, socket->fd, 0, &running); + } while (r =3D=3D CURLM_CALL_MULTI_PERFORM); } =20 static void curl_multi_do(void *arg) --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569978901; cv=none; d=zoho.com; s=zohoarc; b=iBwJv1HxQQ5M0SxsNIef54Z0q4wqFp1WXaG8+SZ3E1JfjDaufBFuG7XMVBaB+zQ9mXZZT2zcjaiS1Za7MpmwrHQBnN5vkmeilsaJeSMm7yPDblWhMh8FDndR3C1CrqHqCm039y+fnDnlNv7dltOOZPw1m39fioERkslp51hx/+o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569978901; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=M9oV3BCQgcXRuTxoQFwhWgQJJ9aoKrSCOSBRPI651aM=; b=MOEto6QzUEevdsQj1at/vTpkHjACS5YOYv+l7QI0gMzHvgy9Zqn5rmobnesQEvHJ0e91GagFESxp3ICtVRC0y+DNHy2PE8qJFGzC1DnfUm5Sx3XqIumnbAQWs3lDE9Id7sasMwc5QHQRZ1kJ6aQDjtGJq/F0hcGnfzAr0bA0J4c= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569978901550338.38259328515153; Tue, 1 Oct 2019 18:15:01 -0700 (PDT) Received: from localhost ([::1]:50084 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTEJ-0001WS-NG for importer@patchew.org; Tue, 01 Oct 2019 21:14:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40984) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsi-00029U-Kt for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsc-0002kw-S3 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:36 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:44866) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsc-00028T-DG; Tue, 01 Oct 2019 19:48:30 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOon074757; Tue, 1 Oct 2019 19:48:04 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq6q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:04 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91Nm3Fq076153; Tue, 1 Oct 2019 19:48:03 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq6a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:03 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlPR031629; Tue, 1 Oct 2019 23:48:02 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma02wdc.us.ibm.com with ESMTP id 2v9y587fdd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:02 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm2gZ38011304 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:02 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0F7B528059; Tue, 1 Oct 2019 23:48:02 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E779828058; Tue, 1 Oct 2019 23:48:01 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:01 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 91/97] curl: Handle success in multi_check_completion Date: Tue, 1 Oct 2019 18:46:10 -0500 Message-Id: <20191001234616.7825-92-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Background: As of cURL 7.59.0, it verifies that several functions are not called from within a callback. Among these functions is curl_multi_add_handle(). curl_read_cb() is a callback from cURL and not a coroutine. Waking up acb->co will lead to entering it then and there, which means the current request will settle and the caller (if it runs in the same coroutine) may then issue the next request. In such a case, we will enter curl_setup_preadv() effectively from within curl_read_cb(). Calling curl_multi_add_handle() will then fail and the new request will not be processed. Fix this by not letting curl_read_cb() wake up acb->co. Instead, leave the whole business of settling the AIOCB objects to curl_multi_check_completion() (which is called from our timer callback and our FD handler, so not from any cURL callbacks). Reported-by: Natalie Gavrielov Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=3D1740193 Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20190910124136.10565-7-mreitz@redhat.com Reviewed-by: John Snow Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz (cherry picked from commit bfb23b480a49114315877aacf700b49453e0f9d9) Signed-off-by: Michael Roth --- block/curl.c | 69 ++++++++++++++++++++++------------------------------ 1 file changed, 29 insertions(+), 40 deletions(-) diff --git a/block/curl.c b/block/curl.c index de0cebd361..af40203711 100644 --- a/block/curl.c +++ b/block/curl.c @@ -228,7 +228,6 @@ static size_t curl_read_cb(void *ptr, size_t size, size= _t nmemb, void *opaque) { CURLState *s =3D ((CURLState*)opaque); size_t realsize =3D size * nmemb; - int i; =20 trace_curl_read_cb(realsize); =20 @@ -244,32 +243,6 @@ static size_t curl_read_cb(void *ptr, size_t size, siz= e_t nmemb, void *opaque) memcpy(s->orig_buf + s->buf_off, ptr, realsize); s->buf_off +=3D realsize; =20 - for(i=3D0; iacb[i]; - - if (!acb) - continue; - - if ((s->buf_off >=3D acb->end)) { - size_t request_length =3D acb->bytes; - - qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start, - acb->end - acb->start); - - if (acb->end - acb->start < request_length) { - size_t offset =3D acb->end - acb->start; - qemu_iovec_memset(acb->qiov, offset, 0, - request_length - offset); - } - - acb->ret =3D 0; - s->acb[i] =3D NULL; - qemu_mutex_unlock(&s->s->mutex); - aio_co_wake(acb->co); - qemu_mutex_lock(&s->s->mutex); - } - } - read_end: /* curl will error out if we do not return this value */ return size * nmemb; @@ -350,13 +323,14 @@ static void curl_multi_check_completion(BDRVCURLState= *s) break; =20 if (msg->msg =3D=3D CURLMSG_DONE) { + int i; CURLState *state =3D NULL; + bool error =3D msg->data.result !=3D CURLE_OK; + curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE, (char **)&state); =20 - /* ACBs for successful messages get completed in curl_read_cb = */ - if (msg->data.result !=3D CURLE_OK) { - int i; + if (error) { static int errcount =3D 100; =20 /* Don't lose the original error message from curl, since @@ -368,20 +342,35 @@ static void curl_multi_check_completion(BDRVCURLState= *s) error_report("curl: further errors suppressed"); } } + } =20 - for (i =3D 0; i < CURL_NUM_ACB; i++) { - CURLAIOCB *acb =3D state->acb[i]; + for (i =3D 0; i < CURL_NUM_ACB; i++) { + CURLAIOCB *acb =3D state->acb[i]; =20 - if (acb =3D=3D NULL) { - continue; - } + if (acb =3D=3D NULL) { + continue; + } + + if (!error) { + /* Assert that we have read all data */ + assert(state->buf_off >=3D acb->end); + + qemu_iovec_from_buf(acb->qiov, 0, + state->orig_buf + acb->start, + acb->end - acb->start); =20 - acb->ret =3D -EIO; - state->acb[i] =3D NULL; - qemu_mutex_unlock(&s->mutex); - aio_co_wake(acb->co); - qemu_mutex_lock(&s->mutex); + if (acb->end - acb->start < acb->bytes) { + size_t offset =3D acb->end - acb->start; + qemu_iovec_memset(acb->qiov, offset, 0, + acb->bytes - offset); + } } + + acb->ret =3D error ? -EIO : 0; + state->acb[i] =3D NULL; + qemu_mutex_unlock(&s->mutex); + aio_co_wake(acb->co); + qemu_mutex_lock(&s->mutex); } =20 curl_clean_state(state); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979843; cv=none; d=zoho.com; s=zohoarc; b=EzpBkyiXqo3F6rtEpHgDaGUarp51qP/aclayRc0VL9DgpL0vBCHQT/7B/Z3UuKNnN6qQQxoFgCNV/rpCWb5FRc++3UXxDljWrCWrHGssa9u+z2IHt3CtshyQpACXpWr9n2jkNBNAoPAQrVvv+jklHpMKWy/NIs0rSkmHbKlFxco= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979843; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=TBZ99ea2TcLfXqEJyCKtaAXaE8ZXIIgvw6vrLHub3pU=; b=Y1eYeqfkhVjxpbXbawJ1ho0Svzn0FSHvdoIAqQEUY7XS8udjCoVSR6rgQR2f4DDNuqZWiReV3+oblYuZf56xjz72skyujpx93jzPWg8wZklyE9e9XnplAgSFBOsj8xgWBQ/nI0Ku8wUIEne+picJmxeY/wruMpkhuTWOfUeoB3Q= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979843169931.0502987777702; Tue, 1 Oct 2019 18:30:43 -0700 (PDT) Received: from localhost ([::1]:50194 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTTS-0007QV-HT for importer@patchew.org; Tue, 01 Oct 2019 21:30:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41272) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsp-0002E2-32 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsh-0002uz-PE for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:53242 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-00027w-O6; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlRe1099084; Tue, 1 Oct 2019 19:48:04 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v7b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:03 -0400 Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlOAJ098863; Tue, 1 Oct 2019 19:48:03 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfjp9v73-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:03 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlCA023568; Tue, 1 Oct 2019 23:48:03 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma05wdc.us.ibm.com with ESMTP id 2v9y577h1c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:03 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm2Wf5309018 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:02 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 89AEE28064; Tue, 1 Oct 2019 23:48:02 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6E8C92805E; Tue, 1 Oct 2019 23:48:02 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:02 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 92/97] curl: Check curl_multi_add_handle()'s return code Date: Tue, 1 Oct 2019 18:46:11 -0500 Message-Id: <20191001234616.7825-93-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-stable@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz If we had done that all along, debugging would have been much simpler. (Also, I/O errors are better than hangs.) Signed-off-by: Max Reitz Message-id: 20190910124136.10565-8-mreitz@redhat.com Reviewed-by: Maxim Levitsky Reviewed-by: John Snow Signed-off-by: Max Reitz (cherry picked from commit c34dc07f9f01cf686e512f939aece744723072cd) Signed-off-by: Michael Roth --- block/curl.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/block/curl.c b/block/curl.c index af40203711..7a29b40954 100644 --- a/block/curl.c +++ b/block/curl.c @@ -881,7 +881,13 @@ static void curl_setup_preadv(BlockDriverState *bs, CU= RLAIOCB *acb) trace_curl_setup_preadv(acb->bytes, start, state->range); curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range); =20 - curl_multi_add_handle(s->multi, state->curl); + if (curl_multi_add_handle(s->multi, state->curl) !=3D CURLM_OK) { + state->acb[0] =3D NULL; + acb->ret =3D -EIO; + + curl_clean_state(state); + goto out; + } =20 /* Tell curl it needs to kick things off */ curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980727; cv=none; d=zoho.com; s=zohoarc; b=RX/JptTbW5N4KSbcMzdsCK7+KZZxbH4xbh83LktMyYLW8rI/da7Z3NkHmeY4T1369xSSeXFiqMv1e+WDoBRCr0MzrrANbuJGxQMrpjaZWxtoCAT1rp7bU0tzQlaIz/XEzYeJ/CknzxuAiQufkgq0DfOqtnlPxEBwQ2s6/3Lkl5M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980727; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=xA74NtNzqZTxkv2GXMgZXw4cXRECyV6XnE53pMhlRHA=; b=e69iIYpqATenLoqknPhnVSxOpvfC4DXK70s0mC/xV7O5vopCOYVBuUzjICran7gRO9aBIjr7ZZTByp/r6WxbSSMuuyaRRSZUb+yGk9FJw+LflMJHDq8dSFHqCMDRu87lGsmGCDtvHNaZFKoUm0/oS39QUTf14aYu+pX/j5/L/yA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569980727426878.5856934998571; Tue, 1 Oct 2019 18:45:27 -0700 (PDT) Received: from localhost ([::1]:50336 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFThj-0007H7-8W for importer@patchew.org; Tue, 01 Oct 2019 21:45:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41532) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsw-0002JP-Oq for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRss-00037r-RV for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:48 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:51988 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-0002D1-PW; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlX3d146488; Tue, 1 Oct 2019 19:48:04 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26q6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:04 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkEb027150; Tue, 1 Oct 2019 23:48:03 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bk2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:03 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm3As38535662 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:03 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 104CA28059; Tue, 1 Oct 2019 23:48:03 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E88B12805C; Tue, 1 Oct 2019 23:48:02 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:02 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 93/97] slirp: Fix heap overflow in ip_reass on big packet input Date: Tue, 1 Oct 2019 18:46:12 -0500 Message-Id: <20191001234616.7825-94-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Samuel Thibault , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When the first fragment does not fit in the preallocated buffer, q will already be pointing to the ext buffer, so we mustn't try to update it. Signed-off-by: Samuel Thibault (from libslirp.git commit 126c04acbabd7ad32c2b018fe10dfac2a3bc1210) (from libslirp.git commit e0be80430c390bce181ea04dfcdd6ea3dfa97de1) *squash in e0be80 (clarifying comments) Signed-off-by: Michael Roth --- slirp/src/ip_input.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/slirp/src/ip_input.c b/slirp/src/ip_input.c index a714fecd58..68a99de5b5 100644 --- a/slirp/src/ip_input.c +++ b/slirp/src/ip_input.c @@ -331,6 +331,8 @@ insert: q =3D fp->frag_link.next; m =3D dtom(slirp, q); =20 + int was_ext =3D m->m_flags & M_EXT; + q =3D (struct ipasfrag *) q->ipf_next; while (q !=3D (struct ipasfrag*)&fp->frag_link) { struct mbuf *t =3D dtom(slirp, q); @@ -347,13 +349,12 @@ insert: q =3D fp->frag_link.next; =20 /* - * If the fragments concatenated to an mbuf that's - * bigger than the total size of the fragment, then and - * m_ext buffer was alloced. But fp->ipq_next points to - * the old buffer (in the mbuf), so we must point ip - * into the new buffer. + * If the fragments concatenated to an mbuf that's bigger than the total + * size of the fragment and the mbuf was not already using an m_ext buffe= r, + * then an m_ext buffer was alloced. But fp->ipq_next points to the old + * buffer (in the mbuf), so we must point ip into the new buffer. */ - if (m->m_flags & M_EXT) { + if (!was_ext && m->m_flags & M_EXT) { int delta =3D (char *)q - m->m_dat; q =3D (struct ipasfrag *)(m->m_ext + delta); } --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980119; cv=none; d=zoho.com; s=zohoarc; b=P0groP/xPVM5QTs3H6ulbk3ogdQJpyV4L6O5QG4bynOo3KTc49+ysp+fX7dR+WqXJTegwEN2Gg9wqoQX0sKiNxacCGHnNmTq8AV81Rd9vYbOum/YytHp/7oQmuJzIdtQRoOlKVhQ/vZfNEJVMUGiWZH8hc663J82WhJHgk/4kNU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980119; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=KaatbvRP2hEPl9cXFyS/ZS53iLkqTksGia3/+nKeeNs=; b=fUYBPdZeUdnb7Uw6RVVfdxx9heWqdmR9ZMjNSI+wQuEBuF/EnESo27ihd+Bxu/MdOQVDnrUbA96cukkpjobM8cho+cUJD7IYlAvz4HyGDYyhTTi1kGW/Gauk+hcSipIpQ7An0x6h4fnp03k3wHxportat5TTb62VSQJjk4CK5qQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156998011942676.40587674262588; Tue, 1 Oct 2019 18:35:19 -0700 (PDT) Received: from localhost ([::1]:50244 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTXv-0004Sm-BF for importer@patchew.org; Tue, 01 Oct 2019 21:35:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41553) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsx-0002K0-4N for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsu-00039x-S1 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:50 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:5134) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-0002Gz-8i; Tue, 01 Oct 2019 19:48:34 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlO0j053445; Tue, 1 Oct 2019 19:48:05 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcg1716gx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:05 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk6S029377; Tue, 1 Oct 2019 23:48:04 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fam2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:04 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm3ET38535672 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:03 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8CBAF2805A; Tue, 1 Oct 2019 23:48:03 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 707DE28058; Tue, 1 Oct 2019 23:48:03 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:03 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 94/97] slirp: ip_reass: Fix use after free Date: Tue, 1 Oct 2019 18:46:13 -0500 Message-Id: <20191001234616.7825-95-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Samuel Thibault , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Using ip_deq after m_free might read pointers from an allocation reuse. This would be difficult to exploit, but that is still related with CVE-2019-14378 which generates fragmented IP packets that would trigger this issue and at least produce a DoS. Signed-off-by: Samuel Thibault (from libslirp.git commit c59279437eda91841b9d26079c70b8a540d41204) Signed-off-by: Michael Roth --- slirp/src/ip_input.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/slirp/src/ip_input.c b/slirp/src/ip_input.c index 68a99de5b5..89ae04e0c1 100644 --- a/slirp/src/ip_input.c +++ b/slirp/src/ip_input.c @@ -297,6 +297,7 @@ ip_reass(Slirp *slirp, struct ip *ip, struct ipq *fp) */ while (q !=3D (struct ipasfrag*)&fp->frag_link && ip->ip_off + ip->ip_len > q->ipf_off) { + struct ipasfrag *prev; i =3D (ip->ip_off + ip->ip_len) - q->ipf_off; if (i < q->ipf_len) { q->ipf_len -=3D i; @@ -304,9 +305,10 @@ ip_reass(Slirp *slirp, struct ip *ip, struct ipq *fp) m_adj(dtom(slirp, q), i); break; } + prev =3D q; q =3D q->ipf_next; - m_free(dtom(slirp, q->ipf_prev)); - ip_deq(q->ipf_prev); + ip_deq(prev); + m_free(dtom(slirp, prev)); } =20 insert: --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980121; cv=none; d=zoho.com; s=zohoarc; b=ROuZ6MN7J754FOKgDIRYKrQ3KT6iZ2JR8Icpe/a3eZc/1Te06w47dvI6EHFL/MG3P4sMqXCcgcNEnNJ6GRb+HTDmetj4BjGTdal1hwUUT213a0T4qlmuXs2VhACCOxtl2Ni+fN860HrEQStD+uGDk/P37RKOQ6oT+R/PXXCcSso= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980121; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=aS2Afu/xFO233N2edzeo8T8oFpTNEJx/GWXpGK8Xfes=; b=WA02cp1uknoV45P6S4TRmwOQQ00zmlOH2a6ZPM3AdjkeQXTLY+CMLX2L6+oDTQYycZggjJnUnLk9AudXrhv5mQYe9uDfani+0TJz1KRBhVXLVU9xYr/mhb7nSvUcGLeEDAG7C8serZaTIlGdT6S+Y+x66zuFnWWNNugtSJJRhsI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15699801213222.2708293630853404; Tue, 1 Oct 2019 18:35:21 -0700 (PDT) Received: from localhost ([::1]:50242 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTXv-0004SV-Ai for importer@patchew.org; Tue, 01 Oct 2019 21:35:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41543) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsw-0002Jd-V3 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRss-00037Y-R8 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:48 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:53396 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-0002E4-Jv; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlR3j146358; Tue, 1 Oct 2019 19:48:07 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0b-001b2d01.pphosted.com with ESMTP id 2vcfcp26qq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:06 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjlUN029385; Tue, 1 Oct 2019 23:48:06 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma01dal.us.ibm.com with ESMTP id 2v9y59fam9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:06 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm4cT45089202 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:04 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1881A28064; Tue, 1 Oct 2019 23:48:04 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EFE4D28059; Tue, 1 Oct 2019 23:48:03 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:03 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 95/97] s390: PCI: fix IOMMU region init Date: Tue, 1 Oct 2019 18:46:14 -0500 Message-Id: <20191001234616.7825-96-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Christian Borntraeger , qemu-stable@nongnu.org, Matthew Rosato Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Matthew Rosato The fix in dbe9cf606c shrinks the IOMMU memory region to a size that seems reasonable on the surface, however is actually too small as it is based against a 0-mapped address space. This causes breakage with small guests as they can overrun the IOMMU window. Let's go back to the prior method of initializing iommu for now. Fixes: dbe9cf606c ("s390x/pci: Set the iommu region size mpcifc request") Cc: qemu-stable@nongnu.org Reviewed-by: Pierre Morel Reported-by: Boris Fiuczynski Tested-by: Boris Fiuczynski Reported-by: Stefan Zimmerman Signed-off-by: Matthew Rosato Message-Id: <1569507036-15314-1-git-send-email-mjrosato@linux.ibm.com> Signed-off-by: Christian Borntraeger (cherry picked from commit 7df1dac5f1c85312474df9cb3a8fcae72303da62) Signed-off-by: Michael Roth --- hw/s390x/s390-pci-bus.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 2d0a28d544..a6be6305b8 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -694,10 +694,15 @@ static const MemoryRegionOps s390_msi_ctrl_ops =3D { =20 void s390_pci_iommu_enable(S390PCIIOMMU *iommu) { + /* + * The iommu region is initialized against a 0-mapped address space, + * so the smallest IOMMU region we can define runs from 0 to the end + * of the PCI address space. + */ char *name =3D g_strdup_printf("iommu-s390-%04x", iommu->pbdev->uid); memory_region_init_iommu(&iommu->iommu_mr, sizeof(iommu->iommu_mr), TYPE_S390_IOMMU_MEMORY_REGION, OBJECT(&iommu-= >mr), - name, iommu->pal - iommu->pba + 1); + name, iommu->pal + 1); iommu->enabled =3D true; memory_region_add_subregion(&iommu->mr, 0, MEMORY_REGION(&iommu->iommu= _mr)); g_free(name); --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569980169; cv=none; d=zoho.com; s=zohoarc; b=EE/30NZ8tm1vJStS3ojkLOJzy5t9uHcq/R4WYrk9NAdtSOBO1ia9kiExAbzQadfsQzHhljqecDAxZ/GStCy34dwOwiiqTNYpSWRA8ye55OYN/+4tpkvwM2AD5D5fa+4IoCMb4O+1aFvsFjpMUqGZAblrW3b6JSKsQHO4q0KTiCI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569980169; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=GR+WrTZLTDeei70tqNdHP3aku2UtZeFKXuywqUJiggo=; b=YYAikRGjYRn8+orFbnsljwMBcfggQveyFPXFlxWLzo+f/jwzWMAqsV8erAueEg9/AIgJ7IWFddj093tmf46N4kLqTLr1Glu1rDuXnD12rT9AgWfQWkGrCRBG44NkN6hufsAwJkM4a9gBR1t6yihlD84XJVsHMMEPrcq4pP2oiMw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156998016903037.47479794033529; Tue, 1 Oct 2019 18:36:09 -0700 (PDT) Received: from localhost ([::1]:50248 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTYk-0005cL-OG for importer@patchew.org; Tue, 01 Oct 2019 21:36:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41279) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsp-0002EC-88 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsh-0002ty-LH for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:60382) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-0002DZ-FC; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlP3s074782; Tue, 1 Oct 2019 19:48:06 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq7f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:06 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlRju074895; Tue, 1 Oct 2019 19:48:06 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcdv2vq78-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:06 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91NjkKP027153; Tue, 1 Oct 2019 23:48:05 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma03dal.us.ibm.com with ESMTP id 2v9y587bka-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:05 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm4g649742086 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:04 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 909B328059; Tue, 1 Oct 2019 23:48:04 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7535D2805E; Tue, 1 Oct 2019 23:48:04 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:04 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 96/97] hw/core/loader: Fix possible crash in rom_copy() Date: Tue, 1 Oct 2019 18:46:15 -0500 Message-Id: <20191001234616.7825-97-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Thomas Huth Both, "rom->addr" and "addr" are derived from the binary image that can be loaded with the "-kernel" paramer. The code in rom_copy() then calculates: d =3D dest + (rom->addr - addr); and uses "d" as destination in a memcpy() some lines later. Now with bad kernel images, it is possible that rom->addr is smaller than addr, thus "rom->addr - addr" gets negative and the memcpy() then tries to copy contents from the image to a bad memory location. This could maybe be used to inject code from a kernel image into the QEMU binary, so we better fix it with an additional sanity check here. Cc: qemu-stable@nongnu.org Reported-by: Guangming Liu Buglink: https://bugs.launchpad.net/qemu/+bug/1844635 Message-Id: <20190925130331.27825-1-thuth@redhat.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Thomas Huth (cherry picked from commit e423455c4f23a1a828901c78fe6d03b7dde79319) Signed-off-by: Michael Roth --- hw/core/loader.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/core/loader.c b/hw/core/loader.c index fe5cb24122..4ef2095247 100644 --- a/hw/core/loader.c +++ b/hw/core/loader.c @@ -1240,7 +1240,7 @@ int rom_copy(uint8_t *dest, hwaddr addr, size_t size) if (rom->addr + rom->romsize < addr) { continue; } - if (rom->addr > end) { + if (rom->addr > end || rom->addr < addr) { break; } =20 --=20 2.17.1 From nobody Mon Apr 29 11:50:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1569979524; cv=none; d=zoho.com; s=zohoarc; b=ehgnLVfCQZxjt5im3vBCBEBBZ/v9DIV7ZWAQF6lCzoIjKljm/tweADVnk/Dki+5BfXaXEaq9V/0QfTsXzNkJWAcr74Apk4tOxRwVASp5mATMnTOe8t65oFlEYbnuWkEUKcesSoOHY2OdMyeu7ng41qfruEvKERk0vXDAg+bCDUY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569979524; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=dkT7XtSPxLV3/mNBRFWR6OjrLh7eZ1RisSETRZd87+Q=; b=ETIFh6rOzK9TcuP7iVAEfHqGc4jQROJu3T2XuDuuInh2YOptF9wRl1GVdySpmypSv/QDGni4VTohLa9u168IBf0m1p/vHryuvSHM7c/9n32Ew/RaeR7443nDji4LrDV5s6eKhtIuoSmLyY7QV66YWikset4DG4D1EFoqqCCpQfw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569979524329793.3314777570431; Tue, 1 Oct 2019 18:25:24 -0700 (PDT) Received: from localhost ([::1]:50156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFTOK-0002P5-IA for importer@patchew.org; Tue, 01 Oct 2019 21:25:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41275) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFRsp-0002E6-55 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFRsh-0002uG-Kq for qemu-devel@nongnu.org; Tue, 01 Oct 2019 19:48:40 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:8596 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFRsg-0002Kh-Rn; Tue, 01 Oct 2019 19:48:35 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x91NlOIk037412; Tue, 1 Oct 2019 19:48:07 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b3d7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:07 -0400 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x91NlWov037760; Tue, 1 Oct 2019 19:48:07 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 2vcer2b3d0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 19:48:07 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x91Njk0W006371; Tue, 1 Oct 2019 23:48:06 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04wdc.us.ibm.com with ESMTP id 2v9y57ye9t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Oct 2019 23:48:06 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x91Nm5eh51642878 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Oct 2019 23:48:05 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1B3D928058; Tue, 1 Oct 2019 23:48:05 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F2FF42805A; Tue, 1 Oct 2019 23:48:04 +0000 (GMT) Received: from localhost (unknown [9.53.179.213]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 1 Oct 2019 23:48:04 +0000 (GMT) From: Michael Roth To: qemu-devel@nongnu.org Subject: [PATCH 97/97] scsi: lsi: exit infinite loop while executing script (CVE-2019-12068) Date: Tue, 1 Oct 2019 18:46:16 -0500 Message-Id: <20191001234616.7825-98-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> References: <20191001234616.7825-1-mdroth@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-10-01_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910010203 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , qemu-stable@nongnu.org, Prasad J Pandit Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini When executing script in lsi_execute_script(), the LSI scsi adapter emulator advances 's->dsp' index to read next opcode. This can lead to an infinite loop if the next opcode is empty. Move the existing loop exit after 10k iterations so that it covers no-op opcodes as well. Reported-by: Bugs SysSec Signed-off-by: Paolo Bonzini Signed-off-by: Prasad J Pandit Signed-off-by: Paolo Bonzini (cherry picked from commit de594e47659029316bbf9391efb79da0a1a08e08) Signed-off-by: Michael Roth --- hw/scsi/lsi53c895a.c | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/hw/scsi/lsi53c895a.c b/hw/scsi/lsi53c895a.c index da7239d94f..d3380b6f95 100644 --- a/hw/scsi/lsi53c895a.c +++ b/hw/scsi/lsi53c895a.c @@ -184,6 +184,9 @@ static const char *names[] =3D { /* Flag set if this is a tagged command. */ #define LSI_TAG_VALID (1 << 16) =20 +/* Maximum instructions to process. */ +#define LSI_MAX_INSN 10000 + typedef struct lsi_request { SCSIRequest *req; uint32_t tag; @@ -1131,7 +1134,21 @@ static void lsi_execute_script(LSIState *s) =20 s->istat1 |=3D LSI_ISTAT1_SRUN; again: - insn_processed++; + if (++insn_processed > LSI_MAX_INSN) { + /* Some windows drivers make the device spin waiting for a memory + location to change. If we have been executed a lot of code then + assume this is the case and force an unexpected device disconne= ct. + This is apparently sufficient to beat the drivers into submissi= on. + */ + if (!(s->sien0 & LSI_SIST0_UDC)) { + qemu_log_mask(LOG_GUEST_ERROR, + "lsi_scsi: inf. loop with UDC masked"); + } + lsi_script_scsi_interrupt(s, LSI_SIST0_UDC, 0); + lsi_disconnect(s); + trace_lsi_execute_script_stop(); + return; + } insn =3D read_dword(s, s->dsp); if (!insn) { /* If we receive an empty opcode increment the DSP by 4 bytes @@ -1568,19 +1585,7 @@ again: } } } - if (insn_processed > 10000 && s->waiting =3D=3D LSI_NOWAIT) { - /* Some windows drivers make the device spin waiting for a memory - location to change. If we have been executed a lot of code then - assume this is the case and force an unexpected device disconne= ct. - This is apparently sufficient to beat the drivers into submissi= on. - */ - if (!(s->sien0 & LSI_SIST0_UDC)) { - qemu_log_mask(LOG_GUEST_ERROR, - "lsi_scsi: inf. loop with UDC masked"); - } - lsi_script_scsi_interrupt(s, LSI_SIST0_UDC, 0); - lsi_disconnect(s); - } else if (s->istat1 & LSI_ISTAT1_SRUN && s->waiting =3D=3D LSI_NOWAIT= ) { + if (s->istat1 & LSI_ISTAT1_SRUN && s->waiting =3D=3D LSI_NOWAIT) { if (s->dcntl & LSI_DCNTL_SSM) { lsi_script_dma_interrupt(s, LSI_DSTAT_SSI); } else { @@ -1968,6 +1973,10 @@ static void lsi_reg_writeb(LSIState *s, int offset, = uint8_t val) case 0x2f: /* DSP[24:31] */ s->dsp &=3D 0x00ffffff; s->dsp |=3D val << 24; + /* + * FIXME: if s->waiting !=3D LSI_NOWAIT, this will only execute one + * instruction. Is this correct? + */ if ((s->dmode & LSI_DMODE_MAN) =3D=3D 0 && (s->istat1 & LSI_ISTAT1_SRUN) =3D=3D 0) lsi_execute_script(s); @@ -1986,6 +1995,10 @@ static void lsi_reg_writeb(LSIState *s, int offset, = uint8_t val) break; case 0x3b: /* DCNTL */ s->dcntl =3D val & ~(LSI_DCNTL_PFF | LSI_DCNTL_STD); + /* + * FIXME: if s->waiting !=3D LSI_NOWAIT, this will only execute one + * instruction. Is this correct? + */ if ((val & LSI_DCNTL_STD) && (s->istat1 & LSI_ISTAT1_SRUN) =3D=3D = 0) lsi_execute_script(s); break; --=20 2.17.1