From nobody Fri May 17 07:56:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1664247887465498.3113687938535; Mon, 26 Sep 2022 20:04:47 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.412196.655417 (Exim 4.92) (envelope-from ) id 1od0tL-0002ma-8v; Tue, 27 Sep 2022 03:04:15 +0000 Received: by outflank-mailman (output) from mailman id 412196.655417; Tue, 27 Sep 2022 03:04:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tL-0002mT-6A; Tue, 27 Sep 2022 03:04:15 +0000 Received: by outflank-mailman (input) for mailman id 412196; Tue, 27 Sep 2022 03:04:14 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tK-0002Wn-2U for xen-devel@lists.xenproject.org; Tue, 27 Sep 2022 03:04:14 +0000 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 0fb06a2b-3e11-11ed-9648-05401a9f4f97; Tue, 27 Sep 2022 05:04:12 +0200 (CEST) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 7CDC15C0193; Mon, 26 Sep 2022 23:04:11 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Mon, 26 Sep 2022 23:04:11 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 26 Sep 2022 23:04:10 -0400 (EDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0fb06a2b-3e11-11ed-9648-05401a9f4f97 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1664247851; x=1664334251; bh=jh+Frhwmh5e9JCT1Ke46iLV+JdKVkMGqf4f rllkpN/Q=; b=16v6Q6OYYkHyRXIFYk3fIXCc5kxp/x8ZzzFI9kCroshhXUCyFJG U9kAtnvUVBp3uyiBdEjQ7fJRo0HwH5XsPwynrwQoK1gFtB52fgeeOpkCN6ZUwI1n Ne4N1I2yJjieq3BgLyF10ziJP+Piktqug3ZRa0eyA8GkQl2giu4AoB/tHApDTiY4 KP40jNX07ugI1zo/XX4SqOal3opBFrMW8VAkpo3ytDn72vwtDSnIWWAmQ4SyGFVX l8Jx46yJ5OhgRluDWPbF/ssOwqZbIKQA7qrANksfuiwA4LTj2R8h9b350N3Iz0py MmKDcTldHKKTHI8zrOE7XkwI4p5IQgv26NA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1664247851; x=1664334251; bh=jh+Frhwmh5e9J CT1Ke46iLV+JdKVkMGqf4frllkpN/Q=; b=gZ4D83zX/uTM0NwvNsn/VTSvBwumz U1AFa+B9CcmlkwonHsDLRJBKlGavhiTuJ7waSOzO0cWA9kaKSEQTfHLd7ZdOWfId Egx9aCN1UNhoua89qzl9dpkYkCc6/qMY30c4tmlL3KQtDh1XuFNpGw5v4rkUDJGJ n95n8CUIlXAsFe0DI42y16Qk3HtgySRuklGSg2Lrckso9bDS0fFSf2Kt6x5q8wcr iVWwJWhyP58ce5a/ctlEFb5dhZOXt9E8R/hz19y555NKJ+KOIX6TCQCn9KPyc5oP UsEkmvMBIqOcocX1QwnEM6aOd5mK6ivaxXUpF80O44ufXUDe0OW2JQlag== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeegfedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepffgvmhhi ucforghrihgvucfqsggvnhhouhhruceouggvmhhisehinhhvihhsihgslhgvthhhihhngh hslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepjeffjefggfeugeduvedvjeekgfeh gffhhfffjeetkeelueefffetfffhtdduheetnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepuggvmhhisehinhhvihhsihgslhgvthhhihhnghhs lhgrsgdrtghomh X-ME-Proxy: Feedback-ID: iac594737:Fastmail From: Demi Marie Obenour To: xen-devel@lists.xenproject.org Cc: Demi Marie Obenour , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH 1/5] libxl: Add a utility function for domain resume Date: Mon, 26 Sep 2022 23:03:57 -0400 Message-Id: <6fcc32dc0747a081eb4b182dae1a483c6d1258f4.1664236600.git.demi@invisiblethingslab.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1664247889559100003 Content-Type: text/plain; charset="utf-8" It is necessary to all xs_resume_domain after any successful call to xc_domain_resume, so that XenStore is notified of the resumption. However, it is also very easy to forget to call this. This took me several days to debug. Fix this by adding a utility function to resume a domain and then notify XenStore of the resumption. This function does not resume any device model, so it is still internal to libxl, but it makes future changes to libxl much less error-prone. It also makes libxl itself smaller. Signed-off-by: Demi Marie Obenour --- tools/libs/light/libxl_dom_suspend.c | 41 +++++++++++++++------------------ 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/tools/libs/light/libxl_dom_suspend.c b/tools/libs/light/libxl_= dom_suspend.c index 4fa22bb7391049f2ea4ac32f21660212053bd4bc..fa50e8801f35d173a99ae5dd19e= b941649e14019 100644 --- a/tools/libs/light/libxl_dom_suspend.c +++ b/tools/libs/light/libxl_dom_suspend.c @@ -451,6 +451,22 @@ int libxl__domain_resume_device_model_deprecated(libxl= __gc *gc, uint32_t domid) return 0; } =20 +/* Just resumes the domain. The device model must have been resumed alrea= dy. */ +static int domain_resume_raw(libxl__gc *gc, uint32_t domid, int suspend_ca= ncel) +{ + if (xc_domain_resume(CTX->xch, domid, suspend_cancel)) { + LOGED(ERROR, domid, "xc_domain_resume failed"); + return ERROR_FAIL; + } + + if (!xs_resume_domain(CTX->xsh, domid)) { + LOGED(ERROR, domid, "xs_resume_domain failed"); + return ERROR_FAIL; + } + + return 0; +} + int libxl__domain_resume_deprecated(libxl__gc *gc, uint32_t domid, int sus= pend_cancel) { int rc =3D 0; @@ -469,16 +485,7 @@ int libxl__domain_resume_deprecated(libxl__gc *gc, uin= t32_t domid, int suspend_c } } =20 - if (xc_domain_resume(CTX->xch, domid, suspend_cancel)) { - LOGED(ERROR, domid, "xc_domain_resume failed"); - rc =3D ERROR_FAIL; - goto out; - } - - if (!xs_resume_domain(CTX->xsh, domid)) { - LOGED(ERROR, domid, "xs_resume_domain failed"); - rc =3D ERROR_FAIL; - } + rc =3D domain_resume_raw(gc, domid, suspend_cancel); out: return rc; } @@ -660,19 +667,9 @@ static void domain_resume_done(libxl__egc *egc, /* Convenience aliases */ libxl_domid domid =3D dmrs->domid; =20 - if (rc) goto out; - - if (xc_domain_resume(CTX->xch, domid, dmrs->suspend_cancel)) { - LOGED(ERROR, domid, "xc_domain_resume failed"); - rc =3D ERROR_FAIL; - goto out; - } + if (!rc) + rc =3D domain_resume_raw(gc, domid, dmrs->suspend_cancel); =20 - if (!xs_resume_domain(CTX->xsh, domid)) { - LOGED(ERROR, domid, "xs_resume_domain failed"); - rc =3D ERROR_FAIL; - } -out: dmrs->callback(egc, dmrs, rc); } =20 --=20 Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab From nobody Fri May 17 07:56:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1664247890071330.8387795027045; Mon, 26 Sep 2022 20:04:50 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.412197.655429 (Exim 4.92) (envelope-from ) id 1od0tM-000322-Hs; Tue, 27 Sep 2022 03:04:16 +0000 Received: by outflank-mailman (output) from mailman id 412197.655429; Tue, 27 Sep 2022 03:04:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tM-00031v-Es; Tue, 27 Sep 2022 03:04:16 +0000 Received: by outflank-mailman (input) for mailman id 412197; Tue, 27 Sep 2022 03:04:14 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tK-0002Wn-Qx for xen-devel@lists.xenproject.org; Tue, 27 Sep 2022 03:04:14 +0000 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 108669a6-3e11-11ed-9648-05401a9f4f97; Tue, 27 Sep 2022 05:04:13 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id DEFB95C0198; Mon, 26 Sep 2022 23:04:12 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 26 Sep 2022 23:04:12 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 26 Sep 2022 23:04:12 -0400 (EDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 108669a6-3e11-11ed-9648-05401a9f4f97 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1664247852; x=1664334252; bh=f6ZPHLSgD7XKJHqHB/XRr1A+xgGqvvyVHfV 9tK3yBF8=; b=stP9n8OLysaRZAgI0hVUO5K5vvWrV3KynONeqKpOch8rKi2SGrM L8eJ2uKOWuK+29by+PEMGNMXhYhekWoT2P8brAIA3lEC04h09EnEFdI960dykbxW CeBK130vOgnOiKt32buiQaq6aA06eebe5d6wziEZq1aw0W9Jamd32a8lsKLPG5vQ A+NREmBzLlc60raErznpkvrzxRU3y916X+Z1oaLYH+tNNWsr25LQLUeRamxhojxi paNdyueOV9DmyG0NajCQPEQKczfworoYUTGvuAjTyYEqtIEvr3+Qr9d1CHU1o09b IBJfo/hATUdgCKKPdPJ3S+Ln7IfX/KhaS5g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1664247852; x=1664334252; bh=f6ZPHLSgD7XKJ HqHB/XRr1A+xgGqvvyVHfV9tK3yBF8=; b=L5QjJcMZbB5LyxexrgJcOojGggW8d gR9NOT3QqReStJ3u53F6k24dtHVd/DOQhX/PxaahaQ7dluPEDYRJIBCHSHtDOSJS z3T7B4I+x3+QydGnxdPlWwPL38RYnd6HPnk6WykynYj+XeYhLDNflG4985PsLomI 9z2zoKXEDCia+nnQtzRj9SGoSWTyOQQGTyuA+IBenTXM3yPFbnAPplBlAfyM0/Sf AJv4Sfa9sxZOCYpMBMQuayjs6GUWKWhXrYQq9esdDVaWu5ttPKl+UNyxA1SKJbtG UVLnBte6QmPa0oEL9AoPfpkl8wbDF9vq2BYE81ZWCwTDohoX3UQBtOdyA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeegfedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepffgvmhhi ucforghrihgvucfqsggvnhhouhhruceouggvmhhisehinhhvihhsihgslhgvthhhihhngh hslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepjeffjefggfeugeduvedvjeekgfeh gffhhfffjeetkeelueefffetfffhtdduheetnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepuggvmhhisehinhhvihhsihgslhgvthhhihhnghhs lhgrsgdrtghomh X-ME-Proxy: Feedback-ID: iac594737:Fastmail From: Demi Marie Obenour To: xen-devel@lists.xenproject.org Cc: Demi Marie Obenour , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH 2/5] libxl: Add utility function to check guest status Date: Mon, 26 Sep 2022 23:03:58 -0400 Message-Id: <3bbc7f9038049815bc2af1e4d901347160e09f7b.1664236600.git.demi@invisiblethingslab.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1664247891530100005 Content-Type: text/plain; charset="utf-8" This is used to check that a guest has not been destroyed and to obtain information about it. It will be used in subsequent patches. Signed-off-by: Demi Marie Obenour --- tools/libs/light/libxl_dom_suspend.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/tools/libs/light/libxl_dom_suspend.c b/tools/libs/light/libxl_= dom_suspend.c index fa50e8801f35d173a99ae5dd19eb941649e14019..d2a88ea34efb115a8f2b861daf5= 884d95f39e81d 100644 --- a/tools/libs/light/libxl_dom_suspend.c +++ b/tools/libs/light/libxl_dom_suspend.c @@ -321,27 +321,36 @@ static void suspend_common_wait_guest_watch(libxl__eg= c *egc, suspend_common_wait_guest_check(egc, dsps); } =20 +static int check_guest_status(libxl__gc *gc, const uint32_t domid, + xc_domaininfo_t *info, const char *what) +{ + int ret =3D xc_domain_getinfolist(CTX->xch, domid, 1, info); + + if (ret < 0) { + LOGED(ERROR, domid, "unable to check for status of guest"); + return ERROR_FAIL; + } + + if (!(ret =3D=3D 1 && info->domain =3D=3D domid)) { + LOGED(ERROR, domid, "guest we were %s has been destroyed", what); + return ERROR_FAIL; + } + + return 0; +} + static void suspend_common_wait_guest_check(libxl__egc *egc, libxl__domain_suspend_state *dsps) { STATE_AO_GC(dsps->ao); xc_domaininfo_t info; - int ret; int shutdown_reason; =20 /* Convenience aliases */ const uint32_t domid =3D dsps->domid; =20 - ret =3D xc_domain_getinfolist(CTX->xch, domid, 1, &info); - if (ret < 0) { - LOGED(ERROR, domid, "unable to check for status of guest"); + if (check_guest_status(gc, domid, &info, "suspending")) goto err; - } - - if (!(ret =3D=3D 1 && info.domain =3D=3D domid)) { - LOGED(ERROR, domid, "guest we were suspending has been destroyed"); - goto err; - } =20 if (!(info.flags & XEN_DOMINF_shutdown)) /* keep waiting */ --=20 Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab From nobody Fri May 17 07:56:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1664247891306556.2092033905753; Mon, 26 Sep 2022 20:04:51 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.412198.655440 (Exim 4.92) (envelope-from ) id 1od0tN-0003IJ-PO; Tue, 27 Sep 2022 03:04:17 +0000 Received: by outflank-mailman (output) from mailman id 412198.655440; Tue, 27 Sep 2022 03:04:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tN-0003IA-MP; Tue, 27 Sep 2022 03:04:17 +0000 Received: by outflank-mailman (input) for mailman id 412198; Tue, 27 Sep 2022 03:04:16 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tM-0002Wn-AB for xen-devel@lists.xenproject.org; Tue, 27 Sep 2022 03:04:16 +0000 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 11511f31-3e11-11ed-9648-05401a9f4f97; Tue, 27 Sep 2022 05:04:15 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 3A0A25C0197; Mon, 26 Sep 2022 23:04:14 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 26 Sep 2022 23:04:14 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 26 Sep 2022 23:04:13 -0400 (EDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 11511f31-3e11-11ed-9648-05401a9f4f97 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm2; t=1664247854; x=1664334254; bh=rfEB4qq1us gnZZFHzDZCUIOiVtuMjqLlDvfmQ2Nupnk=; b=b4L0AAU5yoKuVvM9hbud7Jan6f QZzf5HHlM1Y7GOBhUDjEJsKcFBbOywnOHsZHqg7T0x7wDU+r19dK2ozH8Mr3pdpD i46TaXr0b+0RDvvpOMreancL/h8dbybFPb2haz68zwPWZAq6j1Dwe/krRYrdRfc5 Sfky5udi49camu4MEsGHoiV9jaeioyzQlzbUd4D4fMXdILdcKFVZ9xlshiKHqqrB kYb1K3HB4VcTlUbgToP/8F7CE7IqCUVfDwHFOUbwh8hZcadxYN0EWJ7LDXSCXD8j yta/7aqX86/jeOR9mVlxBaRcOQXyhmOnYbQJP0RsFB2+MPSrMcucOgMxwdcg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1664247854; x= 1664334254; bh=rfEB4qq1usgnZZFHzDZCUIOiVtuMjqLlDvfmQ2Nupnk=; b=N RN+tstDGAIb10rEkDvX99cdTa3dE9c9cM2HNuxqXu/puRrCGRwYkZjakYa0Ybo7F 6SxuqvJqUibtSEzDdf4nHi+/ygV/NHaHy+RLC0bxo9W6Ctesr01pMyLbeVuTwKPu 6GCdydSqJLpTBXtuBCFlQJiLJAlZTTwQKzYo8qzQesj/WxnExHVDuCkDkUhNXqwR +hxKRy62+fKgefX2+PxyVKOFW2atLVrR/CMbrEYvZssg+o7oe/jULwdJV5EkXmmy scu9s33QoO9KWFKlvsQlWdcxic6Xzqxbjcd/HtuhiAtGJoV8CGzPTNGno4UHDMpn NSIZ7VDMbEwMgWCt212ZQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeegfedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeffvghm ihcuofgrrhhivgcuqfgsvghnohhurhcuoeguvghmihesihhnvhhishhisghlvghthhhinh hgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpedvuefggfekffduhedtffeuffff tddtfeeihffggfejtddutdfguefftedvudehgfenucffohhmrghinhepghhithhhuhgsrd gtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhep uggvmhhisehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh X-ME-Proxy: Feedback-ID: iac594737:Fastmail From: Demi Marie Obenour To: xen-devel@lists.xenproject.org Cc: Demi Marie Obenour , Wei Liu , Anthony PERARD , Juergen Gross , =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= Subject: [PATCH 3/5] libxl: Properly suspend stubdomains Date: Mon, 26 Sep 2022 23:03:59 -0400 Message-Id: <558648f541d1ae827493fe13ff12ba363bad2392.1664236600.git.demi@invisiblethingslab.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1664247891580100007 Currently, libxl neither pauses nor suspends a stubdomain when suspending the domain it serves. Qubes OS has an out-of-tree patch that just pauses the stubdomain, but that is also insufficient: sys-net (an HVM with an attached PCI device) does not properly resume from suspend on some systems, and the stubdomain considers the TSC clocksource to be unstable after resume. This patch properly suspends the stubdomain. Doing so requires creating a nested libxl__domain_suspend_state structure and freeing it when necessary. Additionally, a new callback function is added that runs when the stubdomain has been suspended. libxl__qmp_suspend_save() is called by this new callback. Saving the state doesn't work on Qubes for two reasons: - save/restore consoles are not enabled (as requiring qemu in dom0) - avoid using QMP Link: https://github.com/QubesOS/qubes-issues/issues/7404 Co-authored-by: Marek Marczykowski-G=C3=B3recki Signed-off-by: Marek Marczykowski-G=C3=B3recki Signed-off-by: Demi Marie Obenour --- tools/libs/light/libxl_dom_suspend.c | 171 ++++++++++++++++++++++++++++---- tools/libs/light/libxl_internal.h | 1 + 2 files changed, 150 insertions(+), 22 deletions(-) diff --git a/tools/libs/light/libxl_dom_suspend.c b/tools/libs/light/libxl_= dom_suspend.c index d2a88ea34efb115a8f2b861daf5884d95f39e81d..d276b3c17e70105c19c82e9da57= 0a24297d039f5 100644 --- a/tools/libs/light/libxl_dom_suspend.c +++ b/tools/libs/light/libxl_dom_suspend.c @@ -19,9 +19,9 @@ =20 /*=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Domai= n suspend =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D*/ =20 -int libxl__domain_suspend_init(libxl__egc *egc, - libxl__domain_suspend_state *dsps, - libxl_domain_type type) +static int libxl__domain_suspend_init_inner(libxl__egc *egc, + libxl__domain_suspend_state *d= sps, + libxl_domain_type type) { STATE_AO_GC(dsps->ao); int rc =3D ERROR_FAIL; @@ -35,6 +35,7 @@ int libxl__domain_suspend_init(libxl__egc *egc, libxl__ev_xswatch_init(&dsps->guest_watch); libxl__ev_time_init(&dsps->guest_timeout); libxl__ev_qmp_init(&dsps->qmp); + dsps->dm_dsps =3D dsps->parent_dsps =3D NULL; =20 if (type =3D=3D LIBXL_DOMAIN_TYPE_INVALID) goto out; dsps->type =3D type; @@ -67,18 +68,95 @@ out: return rc; } =20 +static void domain_suspend_device_model_domain_callback(libxl__egc *egc, + libxl__domain_suspend_state *dsps, + int rc); + +int libxl__domain_suspend_init(libxl__egc *egc, + libxl__domain_suspend_state *dsps, + libxl_domain_type type) +{ + STATE_AO_GC(dsps->ao); + uint32_t const domid =3D dsps->domid; + int rc =3D libxl__domain_suspend_init_inner(egc, dsps, type); + + LOGD(DEBUG, domid, "Initialized suspend state"); + if (type !=3D LIBXL_DOMAIN_TYPE_HVM || + !libxl__stubdomain_is_linux_running(gc, domid)) + return rc; + + LOGD(DEBUG, domid, "Need to suspend stubdomain too"); + /* need to suspend the stubdomain too */ + uint32_t const dm_domid =3D libxl_get_stubdom_id(CTX, domid); + if (rc =3D=3D 0 && dm_domid !=3D 0) { + libxl__domain_suspend_state *dm_dsps; + + GCNEW(dm_dsps); + dm_dsps->domid =3D dm_domid; + dm_dsps->ao =3D dsps->ao; + + dm_dsps->type =3D libxl__domain_type(gc, dm_domid); + if (dm_dsps->type =3D=3D LIBXL_DOMAIN_TYPE_PV || + dm_dsps->type =3D=3D LIBXL_DOMAIN_TYPE_PVH) { + rc =3D libxl__domain_suspend_init_inner(egc, dm_dsps, dm_dsps-= >type); + } else { + LOGD(ERROR, domid, "Stubdomain %" PRIu32 " detected as neither= PV " + "nor PVH (got %d), cannot suspend", dm_domi= d, dm_dsps->type); + rc =3D ERROR_FAIL; + } + if (rc) + libxl__domain_suspend_dispose(gc, dsps); + else { + dm_dsps->callback_common_done =3D domain_suspend_device_model_= domain_callback; + dsps->dm_dsps =3D dm_dsps; + dm_dsps->parent_dsps =3D dsps; + } + } + return rc; +} + void libxl__domain_suspend_dispose(libxl__gc *gc, libxl__domain_suspend_state *dsps) { - libxl__xswait_stop(gc, &dsps->pvcontrol); - libxl__ev_evtchn_cancel(gc, &dsps->guest_evtchn); - libxl__ev_xswatch_deregister(gc, &dsps->guest_watch); - libxl__ev_time_deregister(gc, &dsps->guest_timeout); - libxl__ev_qmp_dispose(gc, &dsps->qmp); + for (;;) { + libxl__xswait_stop(gc, &dsps->pvcontrol); + libxl__ev_evtchn_cancel(gc, &dsps->guest_evtchn); + libxl__ev_xswatch_deregister(gc, &dsps->guest_watch); + libxl__ev_time_deregister(gc, &dsps->guest_timeout); + libxl__ev_qmp_dispose(gc, &dsps->qmp); + if (dsps->dm_dsps =3D=3D NULL) + break; + assert(dsps->parent_dsps =3D=3D NULL); + assert(dsps->dm_dsps->parent_dsps =3D=3D dsps); + dsps =3D dsps->dm_dsps; + assert(dsps->dm_dsps =3D=3D NULL); + } } =20 /*----- callbacks, called by xc_domain_save -----*/ =20 +static void domain_suspend_device_model_domain_callback(libxl__egc *egc, + libxl__domain_suspend_state *dm_dsp= s, + int rc) +{ + STATE_AO_GC(dm_dsps->ao); + libxl__domain_suspend_state *dsps =3D dm_dsps->parent_dsps; + assert(dm_dsps->dm_dsps =3D=3D NULL); + assert(dsps); + assert(dsps->dm_dsps =3D=3D dm_dsps); + if (rc) { + LOGD(ERROR, dsps->domid, + "failed to suspend device model (stubdom id %d), rc=3D%d", dm= _dsps->domid, rc); + } else { + LOGD(DEBUG, dsps->domid, + "Successfully suspended stubdomain (stubdom id %d)", dm_dsps-= >domid); + } + dsps->callback_device_model_done(egc, dsps, rc); /* must be last */ +} + +static void domain_suspend_callback_common(libxl__egc *egc, + libxl__domain_suspend_state *ds= ps); + void libxl__domain_suspend_device_model(libxl__egc *egc, libxl__domain_suspend_state *dsps) { @@ -86,6 +164,7 @@ void libxl__domain_suspend_device_model(libxl__egc *egc, int rc =3D 0; uint32_t const domid =3D dsps->domid; const char *const filename =3D dsps->dm_savefile; + libxl__domain_suspend_state *dm_dsps =3D dsps->dm_dsps; =20 switch (libxl__device_model_version_running(gc, domid)) { case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL: { @@ -95,15 +174,24 @@ void libxl__domain_suspend_device_model(libxl__egc *eg= c, break; } case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN: - /* calls dsps->callback_device_model_done when done */ - libxl__qmp_suspend_save(egc, dsps); /* must be last */ + if (dm_dsps) { + assert(dm_dsps->type =3D=3D LIBXL_DOMAIN_TYPE_PVH || + dm_dsps->type =3D=3D LIBXL_DOMAIN_TYPE_PV); + LOGD(DEBUG, domid, "Suspending stubdomain (domid %" PRIu32 ")", + dm_dsps->domid); + /* calls dm_dsps->callback_common_done when done */ + domain_suspend_callback_common(egc, dm_dsps); /* must be last = */ + } else { + LOGD(DEBUG, domid, "Stubdomain not in use"); + /* calls dsps->callback_device_model_done when done */ + libxl__qmp_suspend_save(egc, dsps); /* must be last */ + } return; default: rc =3D ERROR_INVAL; - goto out; + break; } =20 -out: if (rc) LOGD(ERROR, dsps->domid, "failed to suspend device model, rc=3D%d", rc); @@ -130,8 +218,6 @@ static void domain_suspend_common_done(libxl__egc *egc, libxl__domain_suspend_state *dsps, int rc); =20 -static void domain_suspend_callback_common(libxl__egc *egc, - libxl__domain_suspend_state *ds= ps); static void domain_suspend_callback_common_done(libxl__egc *egc, libxl__domain_suspend_state *dsps, int rc); =20 @@ -308,6 +394,7 @@ static void domain_suspend_common_wait_guest(libxl__egc= *egc, suspend_common_wait_guest_timeout, 60*1000); if (rc) goto err; + return; =20 err: @@ -528,6 +615,7 @@ void libxl__dm_resume(libxl__egc *egc, { STATE_AO_GC(dmrs->ao); int rc =3D 0; + uint32_t dm_domid =3D libxl_get_stubdom_id(CTX, dmrs->domid); =20 /* Convenience aliases */ libxl_domid domid =3D dmrs->domid; @@ -543,7 +631,6 @@ void libxl__dm_resume(libxl__egc *egc, =20 switch (libxl__device_model_version_running(gc, domid)) { case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL: { - uint32_t dm_domid =3D libxl_get_stubdom_id(CTX, domid); const char *path, *state; =20 path =3D DEVICE_MODEL_XS_PATH(gc, dm_domid, domid, "/state"); @@ -563,14 +650,54 @@ void libxl__dm_resume(libxl__egc *egc, if (rc) goto out; break; } - case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN: - qmp->ao =3D dmrs->ao; - qmp->domid =3D domid; - qmp->callback =3D dm_resume_qmp_done; - qmp->payload_fd =3D -1; - rc =3D libxl__ev_qmp_send(egc, qmp, "cont", NULL); + case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN: { + xc_domaininfo_t dm_info; + + if (dm_domid =3D=3D 0 /* || !libxl__stubdomain_is_linux_running() = */) { + LOGD(DEBUG, domid, "Resuming dom0 device model using QMP"); + qmp->ao =3D dmrs->ao; + qmp->domid =3D domid; + qmp->callback =3D dm_resume_qmp_done; + qmp->payload_fd =3D -1; + rc =3D libxl__ev_qmp_send(egc, qmp, "cont", NULL); + if (rc) goto out; + return; + } + + LOGD(DEBUG, domid, "Resuming modern stubdomain: ID %" PRIu32, dm_d= omid); + + rc =3D check_guest_status(gc, dm_domid, &dm_info, "resuming"); if (rc) goto out; - break; + + if ((dm_info.flags & XEN_DOMINF_paused)) { + rc =3D xc_domain_unpause(CTX->xch, dm_domid); + if (rc < 0) { + LOGED(ERROR, domid, + "xc_domain_unpause failed for stubdomain %" PRIu32, + dm_domid); + goto out; + } + LOGD(DEBUG, domid, + "xc_domain_unpause succeeded for stubdomain %" PRIu32, + dm_domid); + } + + if ((dm_info.flags & XEN_DOMINF_shutdown)) { + int shutdown_reason =3D + (dm_info.flags >> XEN_DOMINF_shutdownshift) + & XEN_DOMINF_shutdownmask; + if (shutdown_reason !=3D SHUTDOWN_suspend) { + LOGD(ERROR, domid, "stubdomain %d being resumed shut down" + " with unexpected reason code %d", + dm_domid, shutdown_reason); + rc =3D ERROR_FAIL; + goto out; + } + + rc =3D domain_resume_raw(gc, dm_domid, dmrs->suspend_cancel); + } + goto out; + } default: rc =3D ERROR_INVAL; goto out; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_int= ernal.h index 34063baf81c4eae81790e2f25cc37f5cf58eb196..37e5c98f63472e66100b6301d78= ac0178cbf987e 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -3615,6 +3615,7 @@ struct libxl__domain_suspend_state { struct libxl__domain_suspend_state*, int rc); void (*callback_common_done)(libxl__egc*, struct libxl__domain_suspend_state*, int = ok); + struct libxl__domain_suspend_state *dm_dsps, *parent_dsps; }; int libxl__domain_suspend_init(libxl__egc *egc, libxl__domain_suspend_state *dsps, --=20 Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab From nobody Fri May 17 07:56:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1664247892041848.2629348587474; Mon, 26 Sep 2022 20:04:52 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.412199.655451 (Exim 4.92) (envelope-from ) id 1od0tP-0003Zw-Ag; Tue, 27 Sep 2022 03:04:19 +0000 Received: by outflank-mailman (output) from mailman id 412199.655451; Tue, 27 Sep 2022 03:04:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tP-0003Zn-7k; Tue, 27 Sep 2022 03:04:19 +0000 Received: by outflank-mailman (input) for mailman id 412199; Tue, 27 Sep 2022 03:04:17 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tN-0002Wn-N3 for xen-devel@lists.xenproject.org; Tue, 27 Sep 2022 03:04:17 +0000 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 126bfb2c-3e11-11ed-9648-05401a9f4f97; Tue, 27 Sep 2022 05:04:16 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 16DCB5C0194; Mon, 26 Sep 2022 23:04:16 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 26 Sep 2022 23:04:16 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 26 Sep 2022 23:04:15 -0400 (EDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 126bfb2c-3e11-11ed-9648-05401a9f4f97 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1664247856; x=1664334256; bh=bVEPwgLcbq+BZ1Wl7+Gkkocayb9AhcjX3Ls ejAExFkI=; b=l6pbj9Ojnwg8wA/IiuSwALGS1l2xZ96b9R4dZqI0hqOuZ/5OY0g kTlBJQYsnb+cxSvycEQ17gAu1FgbAdYqUamg6htQEGYbsAh8AR8rTNoKdRzMBRNv +0QUGyjcFZiqxOUlZWWHHbuSCbD93kPoDegn48lISK9pB16bFhiLaB3S3UlCF8Yu 0HPsB+PDaAsfzkG0Jzlpsqv/VqNWdw1mFleGfIV8iXesOh/SiRipJ4qmV29DO0Pa QqR4Fw459OOPbRKWyGAPHm+K91VB9Tz+ez73AhHS+6q3CL8v7y0qVIZoTst91RcF z+sGiQzMo8wJpz9QvWIiimER20QdGq7Z5Ig== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1664247856; x=1664334256; bh=bVEPwgLcbq+BZ 1Wl7+Gkkocayb9AhcjX3LsejAExFkI=; b=ZmdooMZN0BCcdS/rDTwwrNLYEIBCV 0lTv4hGq2wWwci8x0r2+eVV7S/+Um2jMRxBJDwaseF+KWXqz5hod6u2wpnrBoZ/W DDlczt0ztAp7Q8iRdfZhH+T4meVjgOihkZrPC2CmAbPLfnrcLX5FXyjx+a5yYFUq eivnM2evkO1wAdSzAS8RaINXWYlXfs694urDS1MVdDdhV1NHBeIqouALQUqd2GTT tEBrFIxstOS23ilk/TAMNmXYzETbPyQwVfLFPBeidoXtGX4ux+FtCEKWjc2DECRm 7cVq3PwOAlUCnQZECEeuYbAyOBUuZXN9URGIcYELXY9QVkRVLpPdOS9fw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeegfedgieelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepffgvmhhi ucforghrihgvucfqsggvnhhouhhruceouggvmhhisehinhhvihhsihgslhgvthhhihhngh hslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepjeffjefggfeugeduvedvjeekgfeh gffhhfffjeetkeelueefffetfffhtdduheetnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepuggvmhhisehinhhvihhsihgslhgvthhhihhnghhs lhgrsgdrtghomh X-ME-Proxy: Feedback-ID: iac594737:Fastmail From: Demi Marie Obenour To: xen-devel@lists.xenproject.org Cc: Demi Marie Obenour , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH 4/5] libxl: Fix race condition in domain suspension Date: Mon, 26 Sep 2022 23:04:00 -0400 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1664247893534100009 Content-Type: text/plain; charset="utf-8" Check if the domain has suspended after setting the XenStore watch to prevent race conditions. Also check if a guest has suspended when the timeout handler is called, and do not consider this to be a timeout. Signed-off-by: Demi Marie Obenour --- tools/libs/light/libxl_dom_suspend.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/tools/libs/light/libxl_dom_suspend.c b/tools/libs/light/libxl_= dom_suspend.c index d276b3c17e70105c19c82e9da570a24297d039f5..42c0e0a152e04fab34152d71156= 4ffe148f24a4c 100644 --- a/tools/libs/light/libxl_dom_suspend.c +++ b/tools/libs/light/libxl_dom_suspend.c @@ -209,7 +209,8 @@ static void domain_suspend_common_wait_guest_evtchn(lib= xl__egc *egc, libxl__ev_evtchn *evev); static void suspend_common_wait_guest_watch(libxl__egc *egc, libxl__ev_xswatch *xsw, const char *watch_path, const char *event_pa= th); -static void suspend_common_wait_guest_check(libxl__egc *egc, +/* Returns true if a callback was called, false otherwise */ +static bool suspend_common_wait_guest_check(libxl__egc *egc, libxl__domain_suspend_state *dsps); static void suspend_common_wait_guest_timeout(libxl__egc *egc, libxl__ev_time *ev, const struct timeval *requested_abs, int rc); @@ -426,7 +427,7 @@ static int check_guest_status(libxl__gc *gc, const uint= 32_t domid, return 0; } =20 -static void suspend_common_wait_guest_check(libxl__egc *egc, +static bool suspend_common_wait_guest_check(libxl__egc *egc, libxl__domain_suspend_state *dsps) { STATE_AO_GC(dsps->ao); @@ -441,7 +442,7 @@ static void suspend_common_wait_guest_check(libxl__egc = *egc, =20 if (!(info.flags & XEN_DOMINF_shutdown)) /* keep waiting */ - return; + return false; =20 shutdown_reason =3D (info.flags >> XEN_DOMINF_shutdownshift) & XEN_DOMINF_shutdownmask; @@ -452,11 +453,15 @@ static void suspend_common_wait_guest_check(libxl__eg= c *egc, } =20 LOGD(DEBUG, domid, "guest has suspended"); + dsps->guest_responded =3D 1; + libxl__xswait_stop(gc, &dsps->pvcontrol); domain_suspend_common_guest_suspended(egc, dsps); - return; + return true; =20 err: + libxl__xswait_stop(gc, &dsps->pvcontrol); domain_suspend_common_done(egc, dsps, ERROR_FAIL); + return true; } =20 static void suspend_common_wait_guest_timeout(libxl__egc *egc, @@ -464,6 +469,8 @@ static void suspend_common_wait_guest_timeout(libxl__eg= c *egc, { libxl__domain_suspend_state *dsps =3D CONTAINER_OF(ev, *dsps, guest_ti= meout); STATE_AO_GC(dsps->ao); + if (suspend_common_wait_guest_check(egc, dsps)) + return; if (rc =3D=3D ERROR_TIMEDOUT) { LOGD(ERROR, dsps->domid, "guest did not suspend, timed out"); rc =3D ERROR_GUEST_TIMEDOUT; --=20 Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab From nobody Fri May 17 07:56:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1664247887121263.2973873791087; Mon, 26 Sep 2022 20:04:47 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.412200.655461 (Exim 4.92) (envelope-from ) id 1od0tQ-0003rX-NH; Tue, 27 Sep 2022 03:04:20 +0000 Received: by outflank-mailman (output) from mailman id 412200.655461; Tue, 27 Sep 2022 03:04:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tQ-0003r4-Jp; Tue, 27 Sep 2022 03:04:20 +0000 Received: by outflank-mailman (input) for mailman id 412200; Tue, 27 Sep 2022 03:04:19 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1od0tP-0002Wn-62 for xen-devel@lists.xenproject.org; Tue, 27 Sep 2022 03:04:19 +0000 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 133b6b87-3e11-11ed-9648-05401a9f4f97; Tue, 27 Sep 2022 05:04:18 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 6AB235C018F; Mon, 26 Sep 2022 23:04:17 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 26 Sep 2022 23:04:17 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 26 Sep 2022 23:04:16 -0400 (EDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 133b6b87-3e11-11ed-9648-05401a9f4f97 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1664247857; x=1664334257; bh=e866Y8JCmjNsNcx13FdeVQqHWG+JkkDT0Yl HuT4uoZM=; b=LDOyqebkTTqgORPbNfdQxFqwHefTI26vgpX2JbSvVp6oxxZcXH7 FA1Uq4cdVMW7py8Cygm7b7hP9Jscj5KJeV8ZJ+2MzdhLsGcoTkvzcQwv7FFw3e0L CM3ZFAaPjQpRGi985FU8Rv9fkR9DMIIA+UeF7APBCH3Thb3Keu/DSSWV3QbLclM5 xl05QOFkITWf+d62nsBv0RzHvof8HPw08iOef8T0qW+gYMsQpnfusySVPm7uZVIN qkkUF+3JF/tLfNBbW+1ndubolrsS3IyWAenERZjMWWAS8YyKCfJTKplsh7mh3Gyd f69+wxpatDOBU7NkAQJusPBZXREhY6XtnOg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1664247857; x=1664334257; bh=e866Y8JCmjNsN cx13FdeVQqHWG+JkkDT0YlHuT4uoZM=; b=XU7Mdz1v/AlV79K/VMFk+HzFX3eDl NK+/Ps0miTvN2XhIrBk9EitGPHkoOruLNPCNFS4hz7eJBi/04FSgjFJvDM9SeDVg cxKnMsXN9FkVqdDR4JJRLKLt9Gb+EZutrdoiGCDyyOHpaSdOepudB7/knba2D2DC ddoNvHY0eAYe1o3bXBYohFqFnbtOSdBpyaNjpLJHWILvResv4azSDNfSHDdul4U5 SW1Vs2thjpDJARivepKf2jAAsmMeXlc8pWn9KTHPrpb33Onq1tvJGED76a2Tt7U8 T85YOmESSd7CuHv53gNvfEdUiTCXRIWwm0OOv2jSF1lpE5qPtv/tIW0ZA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeegfedgieelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepffgvmhhi ucforghrihgvucfqsggvnhhouhhruceouggvmhhisehinhhvihhsihgslhgvthhhihhngh hslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepjeffjefggfeugeduvedvjeekgfeh gffhhfffjeetkeelueefffetfffhtdduheetnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepuggvmhhisehinhhvihhsihgslhgvthhhihhnghhs lhgrsgdrtghomh X-ME-Proxy: Feedback-ID: iac594737:Fastmail From: Demi Marie Obenour To: xen-devel@lists.xenproject.org Cc: Demi Marie Obenour , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH 5/5] libxl: Add additional domain suspend/resume logs Date: Mon, 26 Sep 2022 23:04:01 -0400 Message-Id: <39c0faac0978315e0c79b9a25bb83dc0f951fed9.1664236600.git.demi@invisiblethingslab.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1664247887808100001 Content-Type: text/plain; charset="utf-8" This was useful when debugging, but is not required. Signed-off-by: Demi Marie Obenour --- tools/libs/light/libxl_dom_suspend.c | 20 ++++++++++++++++++-- tools/libs/light/libxl_domain.c | 1 + 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/tools/libs/light/libxl_dom_suspend.c b/tools/libs/light/libxl_= dom_suspend.c index 42c0e0a152e04fab34152d711564ffe148f24a4c..55a172a46f8703661e696971bee= 07dce93117411 100644 --- a/tools/libs/light/libxl_dom_suspend.c +++ b/tools/libs/light/libxl_dom_suspend.c @@ -321,9 +321,11 @@ static void domain_suspend_common_pvcontrol_suspending= (libxl__egc *egc, STATE_AO_GC(dsps->ao); xs_transaction_t t =3D 0; =20 - if (!rc && !domain_suspend_pvcontrol_acked(state)) + if (!rc && !domain_suspend_pvcontrol_acked(state)) { /* keep waiting */ + LOGD(DEBUG, dsps->domid, "PV control callback without ack"); return; + } =20 libxl__xswait_stop(gc, &dsps->pvcontrol); =20 @@ -405,7 +407,10 @@ static void domain_suspend_common_wait_guest(libxl__eg= c *egc, static void suspend_common_wait_guest_watch(libxl__egc *egc, libxl__ev_xswatch *xsw, const char *watch_path, const char *event_pa= th) { + EGC_GC; libxl__domain_suspend_state *dsps =3D CONTAINER_OF(xsw, *dsps, guest_w= atch); + + LOGD(DEBUG, dsps->domid, "@releaseDomain watch fired, checking guest s= tatus"); suspend_common_wait_guest_check(egc, dsps); } =20 @@ -440,9 +445,11 @@ static bool suspend_common_wait_guest_check(libxl__egc= *egc, if (check_guest_status(gc, domid, &info, "suspending")) goto err; =20 - if (!(info.flags & XEN_DOMINF_shutdown)) + if (!(info.flags & XEN_DOMINF_shutdown)) { + LOGD(DEBUG, domid, "guest we were suspending has not shut down yet= "); /* keep waiting */ return false; + } =20 shutdown_reason =3D (info.flags >> XEN_DOMINF_shutdownshift) & XEN_DOMINF_shutdownmask; @@ -469,11 +476,14 @@ static void suspend_common_wait_guest_timeout(libxl__= egc *egc, { libxl__domain_suspend_state *dsps =3D CONTAINER_OF(ev, *dsps, guest_ti= meout); STATE_AO_GC(dsps->ao); + LOGD(DEBUG, dsps->domid, "Timeout callback triggered"); if (suspend_common_wait_guest_check(egc, dsps)) return; if (rc =3D=3D ERROR_TIMEDOUT) { LOGD(ERROR, dsps->domid, "guest did not suspend, timed out"); rc =3D ERROR_GUEST_TIMEDOUT; + } else { + LOGD(ERROR, dsps->domid, "error in timeout handler (code %d)", rc); } domain_suspend_common_done(egc, dsps, rc); } @@ -628,6 +638,8 @@ void libxl__dm_resume(libxl__egc *egc, libxl_domid domid =3D dmrs->domid; libxl__ev_qmp *qmp =3D &dmrs->qmp; =20 + LOGD(DEBUG, domid, "Resuming device model"); + dm_resume_init(dmrs); =20 rc =3D libxl__ev_time_register_rel(dmrs->ao, @@ -640,6 +652,7 @@ void libxl__dm_resume(libxl__egc *egc, case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL: { const char *path, *state; =20 + LOGD(DEBUG, domid, "Resuming legacy device model: stubdomain ID %"= PRIu32, dm_domid); path =3D DEVICE_MODEL_XS_PATH(gc, dm_domid, domid, "/state"); rc =3D libxl__xs_read_checked(gc, XBT_NULL, path, &state); if (rc) goto out; @@ -706,6 +719,7 @@ void libxl__dm_resume(libxl__egc *egc, goto out; } default: + LOGD(ERROR, domid, "Invalid device model type, cannot resume"); rc =3D ERROR_INVAL; goto out; } @@ -782,6 +796,8 @@ void libxl__domain_resume(libxl__egc *egc, int rc =3D 0; libxl_domain_type type =3D libxl__domain_type(gc, dmrs->domid); =20 + LOGD(DEBUG, dmrs->domid, "Resuming domain"); + if (type =3D=3D LIBXL_DOMAIN_TYPE_INVALID) { rc =3D ERROR_FAIL; goto out; diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domai= n.c index a6b0b509cc04379e9d596a38644e3db5963011ea..e8e0894c8617a36f6fc13af73da= a1ed290a466ce 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -566,6 +566,7 @@ int libxl_domain_suspend_only(libxl_ctx *ctx, uint32_t = domid, dsps->ao =3D ao; dsps->domid =3D domid; dsps->type =3D type; + LOGD(DEBUG, domid, "Received request to suspend domain"); rc =3D libxl__domain_suspend_init(egc, dsps, type); if (rc < 0) goto out_err; dsps->callback_common_done =3D domain_suspend_empty_cb; --=20 Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab