From nobody Mon Feb 9 03:52:56 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1618928385; cv=none; d=zohomail.com; s=zohoarc; b=hykxiiY7R6T4FU3yd8NtVlhaqnE9PKFzCZJad8/z1TX7LMbIU+vY7jlzK+HWLHcOq2i6e3buWz04UsDqBIgUtZ5zk3/UQ0g64jOKtIqdb1LntYR0eDkDU4WnsrfEFGpgjoLnUU4mVvsFsCZKiagQ2pGRS/JoQc/jn241i/yjfXI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1618928385; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=IIAGzxw/EMbAV0ZBPFM6qmaR5jltKMXZfrPBgJmzXSE=; b=ME7R+DqMMI10R0XazjdGpF7mCLyGvpV7Uxr65ka0DQacM/SfFD4+gv/ZIXmTkm7TeUhQ1NrQzHMGeAcWPOdYzjI1ocm2DWGdzqArajfby+BzIjMzGI07eo6qLv4w0T8tQgkYV95+bb+nvJTcfoktQswwy9aSeIm88clFnT65cuc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1618928385463141.7772078222314; Tue, 20 Apr 2021 07:19:45 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-187-Nixy98HrMpOwME5aue1ezQ-1; Tue, 20 Apr 2021 10:15:40 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7FA1579EC9; Tue, 20 Apr 2021 14:15:33 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5335019D80; Tue, 20 Apr 2021 14:15:33 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1700844A58; Tue, 20 Apr 2021 14:15:33 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 13KEFQf1005680 for ; Tue, 20 Apr 2021 10:15:26 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7265D5C1C4; Tue, 20 Apr 2021 14:15:26 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.195.241]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9D415C1A1 for ; Tue, 20 Apr 2021 14:15:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618928143; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=IIAGzxw/EMbAV0ZBPFM6qmaR5jltKMXZfrPBgJmzXSE=; b=MxhfNJsqTKywSTInzQMwp4ENQFy4C2di25Ej3kqco7jz2yaBL20ffVTMyVhzm0JIVMFNMr OnMP4olzpUJb+jZ6gBxyuZuZQJ2qX4IQdafK+hX6U0H9ZHk/9nqnCfyltsK4h+5ZhLCpQg N4RywMCo34uqvmZKFIKXLM9x1WZ1KPc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618928384; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=IIAGzxw/EMbAV0ZBPFM6qmaR5jltKMXZfrPBgJmzXSE=; b=BgzacrpBgA3qXaIT2vB1usvmQznJsKWHb1Re3teiWzCGuYLvCNNv54De4apDom8rDkvx3y MKW18C1yo/WPFO/xy7+8RIEo0pYFj+8c4To0NXBlUzKcRm9HY9nQEjQ0j4Rh5m8fUerJbm xulbkjvAlEvzDHsQI9vShuil1Y4V2vs= X-MC-Unique: Nixy98HrMpOwME5aue1ezQ-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH 4/4] lxc: Let the driver detect CGroups earlier Date: Tue, 20 Apr 2021 16:15:18 +0200 Message-Id: <24c665dd86a922310bfd6482fde8b411d0655284.1618928033.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This is the bug I'm facing. I deliberately configured a container so that the source of a to passthrough doesn't exist. The start fails with: lxcContainerPivotRoot:669 : Failed to create /non-existent/path/.oldroot:= Permission denied which is expected. But what is NOT expected is that CGroup hierarchy is left behind. This is because the controller sets up the CGroup hierarchy, user namespace, moves interfaces, etc. and finally checks whether container setup (done in a separate process) succeeded. Only after all this the error is propagated to the LXC driver. The driver aborts the startup and tries to perform the cleanup, but this is missing CGroups because those weren't detected yet. Ideally, whenever a function fails, it tries to unroll back so that is has no artifacts left behind (look at all those frees/FD closes/etc. at end of functions). But with CGroups it is different - the controller process can't clean up after itself, because it is still running inside that CGroup. Therefore, what we have to do is to let the driver detect CGroups as soon as they are created, and proceed with controller execution only after that. Signed-off-by: Michal Privoznik --- src/lxc/lxc_controller.c | 19 +++++++++++++++++-- src/lxc/lxc_process.c | 20 ++++++++++++++++++++ 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index 1c0a370d4b..33f397c11d 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -348,7 +348,7 @@ static int virLXCControllerConsoleSetNonblocking(virLXC= ControllerConsole *consol } =20 =20 -static int virLXCControllerDaemonHandshake(virLXCController *ctrl) +static int virLXCControllerDaemonHandshakeCont(virLXCController *ctrl) { if (lxcContainerSendContinue(ctrl->handshakeFds[1]) < 0) { virReportSystemError(errno, "%s", @@ -358,6 +358,15 @@ static int virLXCControllerDaemonHandshake(virLXCContr= oller *ctrl) return 0; } =20 +static int virLXCControllerDaemonHandshakeWait(virLXCController *ctrl) +{ + if (lxcContainerWaitForContinue(ctrl->handshakeFds[0]) < 0) { + virReportSystemError(errno, "%s", + _("error waiting for continue signal from dae= mon")); + return -1; + } + return 0; +} =20 static int virLXCControllerValidateNICs(virLXCController *ctrl) { @@ -2372,6 +2381,11 @@ virLXCControllerRun(virLXCController *ctrl) if (virLXCControllerSetupCgroupLimits(ctrl) < 0) goto cleanup; =20 + /* Allow daemon to detect CGroups. */ + if (virLXCControllerDaemonHandshakeCont(ctrl) < 0 || + virLXCControllerDaemonHandshakeWait(ctrl) < 0) + goto cleanup; + if (virLXCControllerSetupUserns(ctrl) < 0) goto cleanup; =20 @@ -2401,7 +2415,8 @@ virLXCControllerRun(virLXCController *ctrl) if (virLXCControllerConsoleSetNonblocking(&(ctrl->consoles[i])) < = 0) goto cleanup; =20 - if (virLXCControllerDaemonHandshake(ctrl) < 0) + /* Allow daemon to connect to the monitor. */ + if (virLXCControllerDaemonHandshakeCont(ctrl) < 0) goto cleanup; =20 /* and preemptively close handshakeFds */ diff --git a/src/lxc/lxc_process.c b/src/lxc/lxc_process.c index 493e19f03d..55e74e913b 100644 --- a/src/lxc/lxc_process.c +++ b/src/lxc/lxc_process.c @@ -1473,6 +1473,7 @@ int virLXCProcessStart(virConnectPtr conn, if (g_atomic_int_add(&driver->nactive, 1) =3D=3D 0 && driver->inhibitC= allback) driver->inhibitCallback(true, driver->inhibitOpaque); =20 + /* The first synchronization point is when the controller creates CGro= ups. */ if (lxcContainerWaitForContinue(handshakefds[0]) < 0) { char out[1024]; =20 @@ -1504,6 +1505,25 @@ int virLXCProcessStart(virConnectPtr conn, goto cleanup; } =20 + if (lxcContainerSendContinue(handshakefds[3]) < 0) { + virReportSystemError(errno, "%s", + _("Failed to send continue signal to controll= er")); + goto cleanup; + } + + /* The second synchronization point is when the controller finished + * creating the container. */ + if (lxcContainerWaitForContinue(handshakefds[0]) < 0) { + char out[1024]; + + if (!(virLXCProcessReadLogOutput(vm, logfile, pos, out, 1024) < 0)= ) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("guest failed to start: %s"), out); + } + + goto cleanup; + } + /* And we can get the first monitor connection now too */ if (!(priv->monitor =3D virLXCProcessConnectMonitor(driver, vm))) { /* Intentionally overwrite the real monitor error message, --=20 2.26.3