From nobody Fri Mar 29 14:04:23 2024 Delivered-To: importer@patchew.org Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1562054152; cv=none; d=zoho.com; s=zohoarc; b=K8Umy3m9wIdLF9q+XcJZA/ruu33FwawKJEJvHKa6MKfhrizf8w3TmgiYl8IFhaDAZ0gVbC+oqlrbciVcA6dLCYTtWHoDoU0JjT5NQ+ixZurN5dmOWpmVSfcI21+xgwxzG9hOBxXNZDKB15Ujulq2QWFb8J7QoTJ/fReXqkcKUBw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562054152; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To:ARC-Authentication-Results; bh=85eWiZi6tqjJpxxBSSi4B/eZtD93TtH9IH7PI0jIk2g=; b=TjgB7cZo8q956UbN7R6hCypY7yqdiuCUt3SPuocyBRJ8Bh/6+GTCWlMgqZfWopIO7CQSYLpzfx+3AVQkcrfQRqPQRwI87rw7V0IJ10SvEV9ILWSjWhyX1yoFWvqY7gRHLwW/B6yj1ccwJM9U4kvGDV6C2noDEP4HUtJhZUtMqXQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 156205415210362.67361510637977; Tue, 2 Jul 2019 00:55:52 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiDcP-0005KT-NV; Tue, 02 Jul 2019 07:54:25 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiDcP-0005KO-2z for xen-devel@lists.xenproject.org; Tue, 02 Jul 2019 07:54:25 +0000 Received: from mga18.intel.com (unknown [134.134.136.126]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 99c499dc-9c9e-11e9-91fa-23c460302157; Tue, 02 Jul 2019 07:54:21 +0000 (UTC) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Jul 2019 00:42:32 -0700 Received: from gao-cwp.sh.intel.com ([10.239.159.26]) by fmsmga006.fm.intel.com with ESMTP; 02 Jul 2019 00:42:30 -0700 X-Inumbo-ID: 99c499dc-9c9e-11e9-91fa-23c460302157 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,442,1557212400"; d="scan'208";a="362532214" From: Chao Gao To: xen-devel@lists.xenproject.org Date: Tue, 2 Jul 2019 15:46:40 +0800 Message-Id: <1562053600-32685-1-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.9.1 Subject: [Xen-devel] [PATCH] libxl_qmp: wait for completion of device removal X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson , Wei Liu , Chao Gao MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" To remove a device from a domain, a qmp command is sent to qemu. But it is handled by qemu asychronously. Even the qmp command is claimed to be done, the actual handling in qemu side may happen later. This behavior brings two questions: 1. Attaching a device back to a domain right after detaching the device from that domain would fail with error: libxl: error: libxl_qmp.c:341:qmp_handle_error_response: Domain 1:received = an error message from QMP server: Duplicate ID 'pci-pt-60_00.0' for device 2. Accesses to PCI configuration space in Qemu may overlap with later device reset issued by 'xl' or by pciback. In order to avoid mentioned questions, wait for the completion of device removal by querying all pci devices using qmp command and ensuring the targ= et device isn't listed. Only retry 5 times to avoid 'xl' potentially being blo= cked by qemu. Signed-off-by: Chao Gao --- tools/libxl/libxl_qmp.c | 57 +++++++++++++++++++++++++++++++++++++++++++++= +++- 1 file changed, 56 insertions(+), 1 deletion(-) diff --git a/tools/libxl/libxl_qmp.c b/tools/libxl/libxl_qmp.c index 42c8ab8..18f6126 100644 --- a/tools/libxl/libxl_qmp.c +++ b/tools/libxl/libxl_qmp.c @@ -916,6 +916,38 @@ out: return rc; } =20 +static int pci_del_callback(libxl__qmp_handler *qmp, + const libxl__json_object *response, void *opaq= ue) +{ + const char *asked_id =3D opaque; + const libxl__json_object *bus =3D NULL; + GC_INIT(qmp->ctx); + int i, j, rc =3D 0; + + for (i =3D 0; (bus =3D libxl__json_array_get(response, i)); i++) { + const libxl__json_object *devices =3D NULL; + const libxl__json_object *device =3D NULL; + const libxl__json_object *o =3D NULL; + const char *id =3D NULL; + + devices =3D libxl__json_map_get("devices", bus, JSON_ARRAY); + + for (j =3D 0; (device =3D libxl__json_array_get(devices, j)); j++)= { + o =3D libxl__json_map_get("qdev_id", device, JSON_STRING); + id =3D libxl__json_object_get_string(o); + + if (id && strcmp(asked_id, id) =3D=3D 0) { + rc =3D 1; + goto out; + } + } + } + +out: + GC_FREE; + return rc; +} + static int qmp_run_command(libxl__gc *gc, int domid, const char *cmd, libxl__json_object *args, qmp_callback_t callback, void *opaque) @@ -1000,9 +1032,32 @@ int libxl__qmp_pci_add(libxl__gc *gc, int domid, lib= xl_device_pci *pcidev) static int qmp_device_del(libxl__gc *gc, int domid, char *id) { libxl__json_object *args =3D NULL; + libxl__qmp_handler *qmp =3D NULL; + int rc =3D 0; + + qmp =3D libxl__qmp_initialize(gc, domid); + if (!qmp) + return ERROR_FAIL; =20 qmp_parameters_add_string(gc, &args, "id", id); - return qmp_run_command(gc, domid, "device_del", args, NULL, NULL); + rc =3D qmp_synchronous_send(qmp, "device_del", args, + NULL, NULL, qmp->timeout); + if (rc =3D=3D 0) { + unsigned int retry =3D 0; + + do { + if (qmp_synchronous_send(qmp, "query-pci", NULL, + pci_del_callback, id, qmp->timeout) = =3D=3D 0) { + break; + } + LOGD(DEBUG, qmp->domid, + "Waiting for completion of deleting device %s", id); + sleep(1); + } while (retry++ < 5); + } + + libxl__qmp_close(qmp); + return rc; } =20 int libxl__qmp_pci_del(libxl__gc *gc, int domid, libxl_device_pci *pcidev) --=20 1.9.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel