From nobody Sat Apr 27 21:46:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1614054491; cv=none; d=zohomail.com; s=zohoarc; b=kkLfa28cCSZXJm8OSoT1LJ8kYCe7uG+Jwwa8IbDpSBO9FQ7W5GaQNcqUioRxvmdHykH7xMKqULOvhAoRMOFYhtEegu2u9MBZgI665xVVOvEBNNj87uC7I/8G8l05NmgvllPorLxm3zitDw5pQ2V/bM72HDcZ3PVd5W97Ak19HZg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1614054491; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=GKlRj14/Tc+y7ZT0WN6OyzULN4kQUPJAvbeJ/UUhI74=; b=R2UMFWv7ldeBXW1HK3vT+bDtU05tbyGGRdIMs+gVL/WLvSKtI2ZsVlfz0sCvFDi2r0Ed2KI5HKs4K9Es8WMdQxfBPsawYt/IMYzVqDxdqWZ5Z15K9PLxTml+rAopJkPuWPNYS1WzGrrBh8igbEJR3ebR1kI0ueTwO9K0YxF0r1E= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1614054491449190.67355409817537; Mon, 22 Feb 2021 20:28:11 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-9-lnA6nOV3Mu25e9ZYZJ6tTA-1; Mon, 22 Feb 2021 23:28:08 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BD21B1E560; Tue, 23 Feb 2021 04:27:59 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AABCF5D9D0; Tue, 23 Feb 2021 04:27:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3E5A118095CB; Tue, 23 Feb 2021 04:27:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 11N4RnSF027488 for ; Mon, 22 Feb 2021 23:27:50 -0500 Received: by smtp.corp.redhat.com (Postfix) id B6F4E20CA48D; Tue, 23 Feb 2021 04:27:49 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B0C0E2166B2F for ; Tue, 23 Feb 2021 04:27:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 939038919B9 for ; Tue, 23 Feb 2021 04:27:46 +0000 (UTC) Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [62.140.7.102]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-439-Hk5bzynRMZqjzlwCkL98SA-1; Mon, 22 Feb 2021 23:27:44 -0500 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-2-wdznNIq-OTqKFd40W4ik5w-1; Tue, 23 Feb 2021 05:27:41 +0100 Received: from AM8PR04MB7970.eurprd04.prod.outlook.com (2603:10a6:20b:24f::9) by AM0PR0402MB3444.eurprd04.prod.outlook.com (2603:10a6:208:20::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.30; Tue, 23 Feb 2021 04:27:40 +0000 Received: from AM8PR04MB7970.eurprd04.prod.outlook.com ([fe80::8532:12af:9c2:66ca]) by AM8PR04MB7970.eurprd04.prod.outlook.com ([fe80::8532:12af:9c2:66ca%7]) with mapi id 15.20.3868.033; Tue, 23 Feb 2021 04:27:40 +0000 Received: from linux-tbji.devlab.prv.suse.com (75.169.34.2) by AM9P193CA0026.EURP193.PROD.OUTLOOK.COM (2603:10a6:20b:21e::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3868.28 via Frontend Transport; Tue, 23 Feb 2021 04:27:39 +0000 X-MC-Unique: lnA6nOV3Mu25e9ZYZJ6tTA-1 X-MC-Unique: Hk5bzynRMZqjzlwCkL98SA-1 X-MC-Unique: wdznNIq-OTqKFd40W4ik5w-1 From: Jim Fehlig To: libvir-list@redhat.com Subject: [PATCH] libxl: Add lock process indicator to libxlDomainObjPrivate object Date: Mon, 22 Feb 2021 21:27:21 -0700 Message-ID: <20210223042721.5012-1-jfehlig@suse.com> X-Originating-IP: [75.169.34.2] X-ClientProxiedBy: AM9P193CA0026.EURP193.PROD.OUTLOOK.COM (2603:10a6:20b:21e::31) To AM8PR04MB7970.eurprd04.prod.outlook.com (2603:10a6:20b:24f::9) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a383328d-174b-4dde-c0a0-08d8d7b35b16 X-MS-TrafficTypeDiagnostic: AM0PR0402MB3444: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2733 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0 X-Microsoft-Antispam-Message-Info: u8fZrED2j/W+aVHbJNpQd84QK57miBuCHYrGSXBHKCYX/2M0Mej6e1o0uNwb8XKuIv1r+JlpVbd++TYWuDhlNYh43f2gRX583TjZkNAQPsrQvD7nD/FvthDbKhUpCKlsX5uGzy/I3w455tPMIOLohjZ2OWMBbKI7CJxsbF+d+yXYyyDqzCxzApK+68T28adgqX1vPNGhopkVsNZnO7LY+W0P7lgBHTs3wqcrw5U2nuTMgLzlnTlnGFNd21XakvAAlIiiJxbfDYcbWzmP/uBzGRhbZMII9vVhIjM0XwRFcnRbHN9CJcmxoMcDMMunb+x+9k6PHqSowvQbSXUe5o1kgD7UAt8P4u5L87FmVgKUsTj7Wjl3fMDMPcT2lX+FWiZIeWMIF60ipx1iAWhGMznZUvKnSPfwfduc/CUx5O5js0JNzX72neWwMsBDpGHE9eD8IetramOeZtJx0pc+jhAuPNe0x7qgkkUxgJghrfxE7fVXT0q6MC3kIBSs7RG5z1p2exla5gMs0TJyPKHHOqloAJ1p7UcH5VJIVyTuQZgROCYZCxXJZJhHt/OAFvX6TqOIuOkGP1/9gOu8cL7eeKfzIA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR04MB7970.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(366004)(346002)(136003)(396003)(39860400002)(4326008)(66556008)(6666004)(8936002)(66476007)(2906002)(66946007)(1076003)(186003)(16526019)(26005)(107886003)(36756003)(2616005)(956004)(6916009)(8676002)(5660300002)(6486002)(52116002)(7696005)(316002)(478600001)(83380400001)(86362001)(145543001); DIR:OUT; SFP:1101 X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?A2+lV7bivCR7TAyKwihyGmaZxddHLRMRBZOcsPX4QIlWrLOpZWmjCX1CJO1T?= =?us-ascii?Q?mnDcRCHbvVEqA6tCes8/HPFjZ/yr3AvOAW3rvvkNwfHEE+YPzIuno6qmADzY?= =?us-ascii?Q?9yWpbYWLvConuuAQCuyXMQIRRvW4aGWQ2iW15+gBje7x5Wb9sdDJ1vcJKkof?= =?us-ascii?Q?Lvc4hYgh+97fB8LH5B+hZv0DMBinlWqfUXA2lOPW1fMt9lOtK1306nrhGd7m?= =?us-ascii?Q?aj0spnfuhN5TeEg2XD7LbraUoZ6JYmOVbIC//PhbN4Yg8HYFTFGBYyPErF+z?= =?us-ascii?Q?EWK5FrXfMca+6G8si4C01Hxqku7t9ImYRk9oalias85xIcfXWJCH02hgCxYR?= =?us-ascii?Q?ZtDiPO9Q240rmQDydPZFaqOUHb1EISiH6x3QURSXFo2Vnez5PJ1trtKR6qhK?= =?us-ascii?Q?g8jXe0u9lNW281d1ekq+iiDsl8NOB0N/8awkrD40Vg1R36t4EV8/2eZ27ym3?= =?us-ascii?Q?7sJpobPRr7lWu9IgCaA/4IGS1TZytDPQ9R5PY4TyIRNvI3JhWs2JppaOAJjd?= =?us-ascii?Q?XBBfWPglaDmzZ8W6WFBQqhlw6xiRqfJysLLrPEQ+513C8FLiydlfOawztBqa?= =?us-ascii?Q?W8dal8zCcuz4g51AKFAAA5YDi2vfDs00ejhdOP1jeZYZtZ4nfVdlsJ2bCs+G?= =?us-ascii?Q?XteEJbcE1oMNTGicGlixFRnLKM5XmDU0SibR+Rh2BsUVy5NEWrU55ZJDEUUf?= =?us-ascii?Q?31WP3lt7QnnR8SUN7jsstW9MgX3DasxxSzcCjKj8LTcP7Q0mXb/rabrfOf5o?= =?us-ascii?Q?0hQs8gv70DdCTmbbxUWCN+hnvQqnqoag25XgOGtEGTUjhfNmt0WLQV/MQ8WE?= =?us-ascii?Q?kF7GY+cmg0s9cOCzZ0LZ4p3b8AsWE7XycEs6BJUSmYcGVZtlSWeXJ4hm2vau?= =?us-ascii?Q?Pi9BTmOszI2xXjHRlmXAwz41xoT52+MvRRn+3cnkal4Wh/SgJ1Qs4gs5bv/S?= =?us-ascii?Q?teaAu/5CyVdVLGfzvASXUf8UNCLHyvaDMEAoPFrkKzfKhK4a+H7hZxk8pNWo?= =?us-ascii?Q?lfbdrMKhtGQxpnbiY3GcGZfZHcw0XrvQxQ9bnswbyU0J2mW0L8zdhQKBAqQs?= =?us-ascii?Q?mIhtOSaMoR1vItL2yQ8o2MZWkLzkG6YKfzFEnKQz4BfljEVHA0vuVFhFgwVb?= =?us-ascii?Q?ZUAPQyef7cdG/gzfty0gkvkm3pHN/iWXl1lCTbyRgFovvLai12UrdS5XkyEW?= =?us-ascii?Q?IMphlgEaDdG2fWWYXmfjJ+nA+bndif9JUO3FKJgUqhyFOeNouV6jU+8Kqwne?= =?us-ascii?Q?lsx13r2xNDu6O/Mp5/WM7g1+NqAyn9NOfGXj1wvB5lDRtmM9bIBMKU8cVfTo?= =?us-ascii?Q?/GPCvgqPVhyu2GXKFUQOXzrl?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: a383328d-174b-4dde-c0a0-08d8d7b35b16 X-MS-Exchange-CrossTenant-AuthSource: AM8PR04MB7970.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2021 04:27:40.2948 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: A2SZLM8pNgsj263pkijQKASEEoccU/tT7OIbedWb1PCh+e2Gdd9qfOlrwx+b3H2wGj743bZXVLzwqL0XY/TjrA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0402MB3444 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 11N4RnSF027488 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The libvirt libxl driver has no access to FDs associated with VM disks. The disks are opened by libxl.so and any related FDs are not exposed to applications. The prevents using virtlockd's auto-release feature to release locks when the FD is closed. Acquiring and releasing locks is explicitly handled by the libxl driver. The current logic is structured such that locks are acquired in libxlDomainStart and released in libxlDomainCleanup. This works well except for migration, where the locks must be released on the source host before the domain can be started on the destination host, but the domain cannot be cleaned up until the migration confirmation stage. When libxlDomainCleanup if finally called in the confirm stage, locks are again released resulting in confusing errors from virtlockd and libvirtd virtlockd[8095]: resource busy: Lockspace resource 'xxxxxx' is not locked libvirtd[8050]: resource busy: Lockspace resource 'xxxxxx' is not locked libvirtd[8050]: Unable to release lease on testvm The error is also encountered in some error cases, e.g. when libxlDomainStart fails before acquiring locks and libxlDomainCleanup is still used for cleanup. In lieu of a mechanism to check if a lock has been acquired, this patch takes an easy approach to fixing the unnecessary lock releases by adding an indicator to the libxlDomainPrivate object that can be set when the lock is acquired and cleared when the lock is released. libxlDomainCleanup can then skip releasing the lock in cases where it was previously released or never acquired in the first place. Signed-off-by: Jim Fehlig Reviewed-by: Michal Privoznik --- danpb and I discussed this issue on IRC and after pursuing several options I've decided to send this simple approach to the list for comment. I rework= ed the code in several ways but in the end cant escape the fact that there are occasions where domain cleanup needs to be performed but stopping the lock process is not needed. Another simple approach I pursued was creating a cleanup function that skipped virDomainLockProcessPause, but this approach was a bit cleaner IMO. Other suggestions welcomed. All is not lost in the effort as it resulted in several cleanup and improvement patches that I'll post after release. It includes decomposing libxlDomainStart and making it idempotent on error so libxlDomainCleanup only needs to be called after a successful call to libxlDomainStart. src/libxl/libxl_domain.c | 13 +++++++++---- src/libxl/libxl_domain.h | 1 + src/libxl/libxl_migration.c | 8 ++++++-- 3 files changed, 16 insertions(+), 6 deletions(-) diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 604b94a006..014f6aceca 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -861,10 +861,14 @@ libxlDomainCleanup(libxlDriverPrivatePtr driver, virHostdevReAttachDomainDevices(hostdev_mgr, LIBXL_DRIVER_INTERNAL_NAM= E, vm->def, hostdev_flags); =20 - VIR_FREE(priv->lockState); - if (virDomainLockProcessPause(driver->lockManager, vm, &priv->lockStat= e) < 0) - VIR_WARN("Unable to release lease on %s", vm->def->name); - VIR_DEBUG("Preserving lock state '%s'", NULLSTR(priv->lockState)); + if (priv->lockProcessRunning) { + VIR_FREE(priv->lockState); + if (virDomainLockProcessPause(driver->lockManager, vm, &priv->lock= State) < 0) + VIR_WARN("Unable to release lease on %s", vm->def->name); + else + priv->lockProcessRunning =3D false; + VIR_DEBUG("Preserving lock state '%s'", NULLSTR(priv->lockState)); + } =20 libxlLoggerCloseFile(cfg->logger, vm->def->id); vm->def->id =3D -1; @@ -1426,6 +1430,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, priv->lockState) < 0) goto destroy_dom; VIR_FREE(priv->lockState); + priv->lockProcessRunning =3D true; =20 /* Always enable domain death events */ if (libxl_evenable_domain_death(cfg->ctx, vm->def->id, 0, &priv->death= W)) diff --git a/src/libxl/libxl_domain.h b/src/libxl/libxl_domain.h index 00682546e0..5fb76bd303 100644 --- a/src/libxl/libxl_domain.h +++ b/src/libxl/libxl_domain.h @@ -68,6 +68,7 @@ struct _libxlDomainObjPrivate { virThreadPtr migrationDstReceiveThr; unsigned short migrationPort; char *lockState; + bool lockProcessRunning; =20 struct libxlDomainJobObj job; =20 diff --git a/src/libxl/libxl_migration.c b/src/libxl/libxl_migration.c index 631f67930d..a5a9df98ad 100644 --- a/src/libxl/libxl_migration.c +++ b/src/libxl/libxl_migration.c @@ -1242,9 +1242,12 @@ libxlDomainMigrationSrcPerform(libxlDriverPrivatePtr= driver, virObjectLock(vm); =20 if (ret =3D=3D 0) { - if (virDomainLockProcessPause(driver->lockManager, vm, &priv->lock= State) < 0) + if (virDomainLockProcessPause(driver->lockManager, vm, &priv->lock= State) =3D=3D 0) { + priv->lockProcessRunning =3D false; + VIR_DEBUG("Preserving lock state '%s'", NULLSTR(priv->lockStat= e)); + } else { VIR_WARN("Unable to release lease on %s", vm->def->name); - VIR_DEBUG("Preserving lock state '%s'", NULLSTR(priv->lockState)); + } } else { /* * Confirm phase will not be executed if perform fails. End the @@ -1377,6 +1380,7 @@ libxlDomainMigrationSrcConfirm(libxlDriverPrivatePtr = driver, "xen:///system", vm, priv->lockState); + priv->lockProcessRunning =3D true; if (libxl_domain_resume(cfg->ctx, vm->def->id, 1, 0) =3D=3D 0) { ret =3D 0; } else { --=20 2.29.2