From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331335; cv=none; d=zohomail.com; s=zohoarc; b=KOP/EepGGmFLYlMmAJFk6ST/p8iMDvJLUQW/qa7YOYMZr63521sYlp/K5ULaMXFdLgPnRpamdMMjHGy6iLdLekGGZ9ZUQm+7x8fwpME6DsYtaw6O4A1US1AEc0Jfqp52lOBozq1fr/f6j3rHwm4kYS8DeIulDiFQYtPp/D9DkbA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331335; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Cl+ZWTLqZhGjV1cgSR8b11jY2+lZT+LoaF+NuMLjGwE=; b=ampZjmCyjxppLnlPFJBQ+oX60GBBrlYOffe1GMvxJOp1kSmS+MYLLjot5xMkvKL4vHKBTr/Xcc7EDZ/LdszodgBFjCxvaNIrePzQyjgAr15GwPO1nplHMactPR5PfuRqLy1qkQmeS893CGbNMLBdVikEC4aEWkFBYsmWEi9LR9s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1661331335894732.1141153544875; Wed, 24 Aug 2022 01:55:35 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-648-LyduLoMQN6KdcmAUTxUaMw-1; Wed, 24 Aug 2022 04:55:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3393680C8F0; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4E8B2166B26; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A6C911946A61; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A82901946A40 for ; Wed, 24 Aug 2022 08:55:25 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 7E6052026D64; Wed, 24 Aug 2022 08:55:25 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29F792026D4C for ; Wed, 24 Aug 2022 08:55:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331334; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Cl+ZWTLqZhGjV1cgSR8b11jY2+lZT+LoaF+NuMLjGwE=; b=XKw3w24FToyxrDea7M1NveWoflbup+Rg7EkEA+eDHqqe51VF0AKuFf01Kac+toJ+1YS9O3 PTioec6f1gv8VYERiT5fBOUDhi+nanzmTJXQNZZwtQF4axS2aGM3M4c4LN37TfvdmNq92+ OUKXRyMsRF4vCWcp3cK5fO2+BTBz04U= X-MC-Unique: LyduLoMQN6KdcmAUTxUaMw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 1/8] virprocess: Core Scheduling support Date: Wed, 24 Aug 2022 10:55:06 +0200 Message-Id: <4d70ffaadda25931753c61f343f6f142ac480c94.1661331156.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331336716100002 Since its 5.14 release the Linux kernel allows userspace to define trusted groups of processes/threads that can run on sibling Hyper Threads (HT) at the same time. This is to mitigate side channel attacks like L1TF or MDS. If there are no tasks to fully utilize all HTs, then a HT will idle instead of running a task from another (un-)trusted group. On low level, this is implemented by cookies (effectively an UL value): processes in the same trusted group share the same cookie and cookie is unique to the group. There are four basic operations: 1) PR_SCHED_CORE_GET -- get cookie of given PID, 2) PR_SCHED_CORE_CREATE -- create a new unique cookie for PID, 3) PR_SCHED_CORE_SHARE_TO -- push cookie of the caller onto another PID, 4) PR_SCHED_CORE_SHARE_FROM -- pull cookie of another PID into the caller. Since a system where the code is built can be different to the one where the code is ran let's provide declaration of some values. It's not unusual for distros to ship older linux-headers than the actual kernel. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrang=C3=A9 --- src/libvirt_private.syms | 4 ++ src/util/virprocess.c | 124 +++++++++++++++++++++++++++++++++++++++ src/util/virprocess.h | 8 +++ 3 files changed, 136 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index ac2802095e..23abf663f6 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3153,6 +3153,10 @@ virProcessKillPainfullyDelay; virProcessNamespaceAvailable; virProcessRunInFork; virProcessRunInMountNamespace; +virProcessSchedCoreAvailable; +virProcessSchedCoreCreate; +virProcessSchedCoreShareFrom; +virProcessSchedCoreShareTo; virProcessSchedPolicyTypeFromString; virProcessSchedPolicyTypeToString; virProcessSetAffinity; diff --git a/src/util/virprocess.c b/src/util/virprocess.c index 013afd91b4..a59e64970d 100644 --- a/src/util/virprocess.c +++ b/src/util/virprocess.c @@ -56,6 +56,10 @@ # include #endif =20 +#ifdef __linux__ +# include +#endif + #include "virprocess.h" #include "virerror.h" #include "viralloc.h" @@ -1874,3 +1878,123 @@ virProcessGetSchedInfo(unsigned long long *cpuWait, return 0; } #endif /* __linux__ */ + +#ifdef __linux__ +# ifndef PR_SCHED_CORE +/* Copied from linux/prctl.h */ +# define PR_SCHED_CORE 62 +# define PR_SCHED_CORE_GET 0 +# define PR_SCHED_CORE_CREATE 1 /* create unique core_sched cookie */ +# define PR_SCHED_CORE_SHARE_TO 2 /* push core_sched cookie to pid */ +# define PR_SCHED_CORE_SHARE_FROM 3 /* pull core_sched cookie to pid */ +# endif + +/* Unfortunately, kernel-headers forgot to export these. */ +# ifndef PR_SCHED_CORE_SCOPE_THREAD +# define PR_SCHED_CORE_SCOPE_THREAD 0 +# define PR_SCHED_CORE_SCOPE_THREAD_GROUP 1 +# define PR_SCHED_CORE_SCOPE_PROCESS_GROUP 2 +# endif + +/** + * virProcessSchedCoreAvailable: + * + * Check whether kernel supports Core Scheduling (CONFIG_SCHED_CORE), i.e.= only + * a defined set of PIDs/TIDs can run on sibling Hyper Threads at the same + * time. + * + * Returns: 1 if Core Scheduling is available, + * 0 if Core Scheduling is NOT available, + * -1 otherwise. + */ +int +virProcessSchedCoreAvailable(void) +{ + unsigned long cookie =3D 0; + int rc; + + /* Let's just see if we can get our own sched cookie, and if yes we can + * safely assume CONFIG_SCHED_CORE kernel is available. */ + rc =3D prctl(PR_SCHED_CORE, PR_SCHED_CORE_GET, 0, + PR_SCHED_CORE_SCOPE_THREAD, &cookie); + + return rc =3D=3D 0 ? 1 : errno =3D=3D EINVAL ? 0 : -1; +} + +/** + * virProcessSchedCoreCreate: + * + * Creates a new trusted group for the caller process. + * + * Returns: 0 on success, + * -1 otherwise, with errno set. + */ +int +virProcessSchedCoreCreate(void) +{ + /* pid =3D 0 (3rd argument) means the calling process. */ + return prctl(PR_SCHED_CORE, PR_SCHED_CORE_CREATE, 0, + PR_SCHED_CORE_SCOPE_THREAD_GROUP, 0); +} + +/** + * virProcessSchedCoreShareFrom: + * @pid: PID to share group with + * + * Places the current caller process into the trusted group of @pid. + * + * Returns: 0 on success, + * -1 otherwise, with errno set. + */ +int +virProcessSchedCoreShareFrom(pid_t pid) +{ + return prctl(PR_SCHED_CORE, PR_SCHED_CORE_SHARE_FROM, pid, + PR_SCHED_CORE_SCOPE_THREAD, 0); +} + +/** + * virProcessSchedCoreShareTo: + * @pid: PID to share group with + * + * Places foreign @pid into the trusted group of the current caller proces= s. + * + * Returns: 0 on success, + * -1 otherwise, with errno set. + */ +int +virProcessSchedCoreShareTo(pid_t pid) +{ + return prctl(PR_SCHED_CORE, PR_SCHED_CORE_SHARE_TO, pid, + PR_SCHED_CORE_SCOPE_THREAD, 0); +} + +#else /* !__linux__ */ + +int +virProcessSchedCoreAvailable(void) +{ + return 0; +} + +int +virProcessSchedCoreCreate(void) +{ + errno =3D ENOSYS; + return -1; +} + +int +virProcessSchedCoreShareFrom(pid_t pid G_GNUC_UNUSED) +{ + errno =3D ENOSYS; + return -1; +} + +int +virProcessSchedCoreShareTo(pid_t pid G_GNUC_UNUSED) +{ + errno =3D ENOSYS; + return -1; +} +#endif /* !__linux__ */ diff --git a/src/util/virprocess.h b/src/util/virprocess.h index 30b6981c73..a3bfd0d911 100644 --- a/src/util/virprocess.h +++ b/src/util/virprocess.h @@ -202,3 +202,11 @@ int virProcessGetStatInfo(unsigned long long *cpuTime, int virProcessGetSchedInfo(unsigned long long *cpuWait, pid_t pid, pid_t tid); + +int virProcessSchedCoreAvailable(void); + +int virProcessSchedCoreCreate(void); + +int virProcessSchedCoreShareFrom(pid_t pid); + +int virProcessSchedCoreShareTo(pid_t pid); --=20 2.35.1 From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331365; cv=none; d=zohomail.com; s=zohoarc; b=PfLH/x07vLCJGoK9GFaJYJrpR/TyPj8zmwkRNHLiWiLJSRtA1z/fxoQ0UfxEdVrnU0oqZSDqk4O/SP8Mw2tYojg64ztAuyGB32eJ+f2gDLDLDaZxiDBySNSp8Mo+pSf6xpGtMD3m14J5p0lnz5WDRGvVDDOz/Rwaw0ikDmaWVMo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331365; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4+twFwEgsHMTnZl+AsS12WtG9oWLPa4SyEu10Zrf+z0=; b=RFWCya8iFhduSVHaLQPLXJrC6KGgV0WVh11Pg/NAkvtVr7F6NCaA+fyxXUr4jCzoa7cOUtmIkY2GS1WNbAMGgtyH9k5QONDcHOBHevx+z8k6zH3ntR31z3GpIVFfoouvO+r69AwOjLDguDZVXRURJK6nwnpYMX7Pp3KEiau4MS4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1661331365247263.58482950951657; Wed, 24 Aug 2022 01:56:05 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-662-6aPwBb4BOVmbDaPKuD7Epw-1; Wed, 24 Aug 2022 04:55:31 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5049884B06F; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A455A14152E0; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C06421946A68; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 210F71946A40 for ; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 1494E2026D64; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4A432026D4C for ; Wed, 24 Aug 2022 08:55:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331363; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=4+twFwEgsHMTnZl+AsS12WtG9oWLPa4SyEu10Zrf+z0=; b=KFPx5eLbc/t0xzm5BiYPqj8ILTxfYVw6BiFQ1QV9VeJJBUS23Fom6Ho3iPc30GcoHHD71O iHLBwfffZig4//OjSe771iKgXtFJbr6SIhxnz3haZVaX3wO4ESIwz559nbIHpvKddkJNRY 9rEiCz5ChwFDABarJ6UgJAyxJAHdh+4= X-MC-Unique: 6aPwBb4BOVmbDaPKuD7Epw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 2/8] virCommand: Introduce APIs for core scheduling Date: Wed, 24 Aug 2022 10:55:07 +0200 Message-Id: <85bfc8529b80169ceb7acd5fbb83fc57635a1c9f.1661331156.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331367005100005 There are two modes of core scheduling that are handy wrt virCommand: 1) create new trusted group when executing a virCommand 2) place freshly executed virCommand into the trusted group of another process. Therefore, implement these two new operations as new APIs: virCommandSetRunAlone() and virCommandSetRunAmong(), respectively. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrang=C3=A9 --- src/libvirt_private.syms | 2 ++ src/util/vircommand.c | 63 ++++++++++++++++++++++++++++++++++++++++ src/util/vircommand.h | 5 ++++ 3 files changed, 70 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 23abf663f6..02d1c753a4 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2103,6 +2103,8 @@ virCommandSetOutputBuffer; virCommandSetOutputFD; virCommandSetPidFile; virCommandSetPreExecHook; +virCommandSetRunAlone; +virCommandSetRunAmong; virCommandSetSELinuxLabel; virCommandSetSendBuffer; virCommandSetUID; diff --git a/src/util/vircommand.c b/src/util/vircommand.c index 4e003266bf..bbfbe19706 100644 --- a/src/util/vircommand.c +++ b/src/util/vircommand.c @@ -148,6 +148,13 @@ struct _virCommand { #endif int mask; =20 + /* schedCore values: + * 0: no core scheduling + * >0: copy scheduling group from PID + * -1: create new scheduling group + */ + pid_t schedCore; + virCommandSendBuffer *sendBuffers; size_t numSendBuffers; }; @@ -434,6 +441,22 @@ virCommandHandshakeChild(virCommand *cmd) static int virExecCommon(virCommand *cmd, gid_t *groups, int ngroups) { + /* Do this before dropping capabilities. */ + if (cmd->schedCore =3D=3D -1 && + virProcessSchedCoreCreate() < 0) { + virReportSystemError(errno, "%s", + _("Unable to set SCHED_CORE")); + return -1; + } + + if (cmd->schedCore > 0 && + virProcessSchedCoreShareFrom(cmd->schedCore) < 0) { + virReportSystemError(errno, + _("Unable to run among %llu"), + (unsigned long long) cmd->schedCore); + return -1; + } + if (cmd->uid !=3D (uid_t)-1 || cmd->gid !=3D (gid_t)-1 || cmd->capabilities || (cmd->flags & VIR_EXEC_CLEAR_CAPS)) { VIR_DEBUG("Setting child uid:gid to %d:%d with caps %llx", @@ -3388,3 +3411,43 @@ virCommandRunNul(virCommand *cmd G_GNUC_UNUSED, return -1; } #endif /* WIN32 */ + +/** + * virCommandSetRunAlone: + * + * Create new trusted group when running the command. In other words, the + * process won't be scheduled to run on a core among with processes from + * another, untrusted group. + */ +void +virCommandSetRunAlone(virCommand *cmd) +{ + if (virCommandHasError(cmd)) + return; + + cmd->schedCore =3D -1; +} + +/** + * virCommandSetRunAmong: + * @pid: pid from a trusted group + * + * When spawning the command place it into the trusted group of @pid so th= at + * these two processes can run on Hyper Threads of a single core at the sa= me + * time. + */ +void +virCommandSetRunAmong(virCommand *cmd, + pid_t pid) +{ + if (virCommandHasError(cmd)) + return; + + if (pid <=3D 0) { + VIR_DEBUG("invalid pid value: %lld", (long long) pid); + cmd->has_error =3D -1; + return; + } + + cmd->schedCore =3D pid; +} diff --git a/src/util/vircommand.h b/src/util/vircommand.h index c7a580e152..98788bcbf7 100644 --- a/src/util/vircommand.h +++ b/src/util/vircommand.h @@ -217,4 +217,9 @@ int virCommandRunNul(virCommand *cmd, virCommandRunNulFunc func, void *data); =20 +void virCommandSetRunAlone(virCommand *cmd); + +void virCommandSetRunAmong(virCommand *cmd, + pid_t pid); + G_DEFINE_AUTOPTR_CLEANUP_FUNC(virCommand, virCommandFree); --=20 2.35.1 From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331363; cv=none; d=zohomail.com; s=zohoarc; b=Ah75XUa+VDOkkRO2AA+iCe/AA78tVTx2hSKvEtZjZGCGc1ZtlkiBg70KBm1JUNycE07MihjyeA9m0ZtnRzLsp+uieihgcRFHOKrqzRjKIM1ZYgcVJ6Ze+le8uR5J/6VdL+sgQGXdK63zIfIjp/AyD9B7X7qQTKXi0CCU6o/KKM4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331363; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=w9721gWfxlGKDZo6zJrGSHC+WB1U6RJiqv4AVMliLG4=; b=YbdTWSq/UuUVt+A00aSa0IuOzhTMLk/Vlp5xupKn8EopOQHGD7DQUhi2grIkLrlIY5xpTH4N65kE0/7JpON0GMspiXX4mecE4B7pcTb3uNJtDK8okjTrIKeIQyRLhuEj7d7fHSEEriLvwI7BS6hz5XgzKj5d5ADNOmqwA1tzwJ4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1661331363673373.3669344025453; Wed, 24 Aug 2022 01:56:03 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-108-XnAHIW72McyTTVni6NDhnw-1; Wed, 24 Aug 2022 04:55:32 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5F5A018A02DC; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 48DFCC15BCA; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1E19B1946A7C; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BF3271946A67 for ; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 9F24A2026D64; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4AE862026D4C for ; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331362; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=w9721gWfxlGKDZo6zJrGSHC+WB1U6RJiqv4AVMliLG4=; b=axoE3Qg7tT/mXAVvzuuIK2af6xtCzUE+TKH0N00qN0g20GbBn7D57tZ5uv8CM6FICEixPO 2E1uqE4WS48bN0xqOGYAmZ57VmbmDtUyDRpDGY3ALPn4KaNjkBA61asrKwwL+gtmFkw2OU Cr1hYtT1iMH8rwQ/SWsarTsH/3O5Fv4= X-MC-Unique: XnAHIW72McyTTVni6NDhnw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 3/8] qemu_conf: Introduce a knob to set SCHED_CORE Date: Wed, 24 Aug 2022 10:55:08 +0200 Message-Id: <3f46888154de86d5b10d274b03ccf53ecde9257c.1661331156.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331365021100002 Content-Type: text/plain; charset="utf-8"; x-default="true" Ideally, we would just pick the best default and users wouldn't have to intervene at all. But in some cases it may be handy to not bother with SCHED_CORE at all or place helper processes into the same group as QEMU. Introduce a knob in qemu.conf to allow users control this behaviour. Signed-off-by: Michal Privoznik --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 16 +++++++++++++ src/qemu/qemu_conf.c | 38 ++++++++++++++++++++++++++++++ src/qemu/qemu_conf.h | 13 ++++++++++ src/qemu/test_libvirtd_qemu.aug.in | 1 + 5 files changed, 69 insertions(+) diff --git a/src/qemu/libvirtd_qemu.aug b/src/qemu/libvirtd_qemu.aug index 0f18775121..ed097ea3d9 100644 --- a/src/qemu/libvirtd_qemu.aug +++ b/src/qemu/libvirtd_qemu.aug @@ -110,6 +110,7 @@ module Libvirtd_qemu =3D | bool_entry "dump_guest_core" | str_entry "stdio_handler" | int_entry "max_threads_per_process" + | str_entry "sched_core" =20 let device_entry =3D bool_entry "mac_filter" | bool_entry "relaxed_acs_check" diff --git a/src/qemu/qemu.conf.in b/src/qemu/qemu.conf.in index 04b7740136..623da72d60 100644 --- a/src/qemu/qemu.conf.in +++ b/src/qemu/qemu.conf.in @@ -952,3 +952,19 @@ # DO NOT use in production. # #deprecation_behavior =3D "none" + +# If this is set then QEMU and its threads will run in a separate scheduli= ng +# group meaning no other process will share Hyper Threads of a single core= with +# QEMU. Each QEMU has its own group. +# +# Possible options are: +# "none" - (default) neither QEMU or any of its helper processes are p= laced +# into separate scheduling group +# "vcpus" - only QEMU vCPU threads are placed into a separate schedulin= g group, +# emulator threads and helper processes remain outside of the= group +# "emulator" - only QEMU and its threads (emulator + vCPUs) are placed into +# separate scheduling group, helper proccesses remain outside= of +# the group +# "full" - both QEMU and its helper processes are placed into separate +# scheduling group +#sched_core =3D "none" diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 3b75cdeb95..2634fe737c 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -66,6 +66,14 @@ VIR_LOG_INIT("qemu.qemu_conf"); #define QEMU_MIGRATION_PORT_MIN 49152 #define QEMU_MIGRATION_PORT_MAX 49215 =20 +VIR_ENUM_IMPL(virQEMUSchedCore, + QEMU_SCHED_CORE_LAST, + "none", + "vcpus", + "emulator", + "full"); + + static virClass *virQEMUDriverConfigClass; static void virQEMUDriverConfigDispose(void *obj); =20 @@ -629,6 +637,7 @@ virQEMUDriverConfigLoadProcessEntry(virQEMUDriverConfig= *cfg, g_auto(GStrv) hugetlbfs =3D NULL; g_autofree char *stdioHandler =3D NULL; g_autofree char *corestr =3D NULL; + g_autofree char *schedCore =3D NULL; size_t i; =20 if (virConfGetValueStringList(conf, "hugetlbfs_mount", true, @@ -706,6 +715,35 @@ virQEMUDriverConfigLoadProcessEntry(virQEMUDriverConfi= g *cfg, } } =20 + if (virConfGetValueString(conf, "sched_core", &schedCore) < 0) + return -1; + if (schedCore) { + int val =3D virQEMUSchedCoreTypeFromString(schedCore); + + if (val < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Unknown sched_core value %s"), + schedCore); + return -1; + } + + if (val !=3D QEMU_SCHED_CORE_NONE) { + int rv =3D virProcessSchedCoreAvailable(); + + if (rv < 0) { + virReportSystemError(errno, "%s", + _("Unable to detect SCHED_CORE")); + return -1; + } else if (rv =3D=3D 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("SCHED_CORE not supported by kernel")); + return -1; + } + } + + cfg->schedCore =3D val; + } + return 0; } =20 diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index c40c452f58..0dd7c55db3 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -44,6 +44,17 @@ =20 #define QEMU_DRIVER_NAME "QEMU" =20 +typedef enum { + QEMU_SCHED_CORE_NONE =3D 0, + QEMU_SCHED_CORE_VCPUS, + QEMU_SCHED_CORE_EMULATOR, + QEMU_SCHED_CORE_FULL, + + QEMU_SCHED_CORE_LAST +} virQEMUSchedCore; + +VIR_ENUM_DECL(virQEMUSchedCore); + typedef struct _virQEMUDriver virQEMUDriver; =20 typedef struct _virQEMUDriverConfig virQEMUDriverConfig; @@ -216,6 +227,8 @@ struct _virQEMUDriverConfig { char **capabilityfilters; =20 char *deprecationBehavior; + + virQEMUSchedCore schedCore; }; =20 G_DEFINE_AUTOPTR_CLEANUP_FUNC(virQEMUDriverConfig, virObjectUnref); diff --git a/src/qemu/test_libvirtd_qemu.aug.in b/src/qemu/test_libvirtd_qe= mu.aug.in index 757d21c33f..1dbd692921 100644 --- a/src/qemu/test_libvirtd_qemu.aug.in +++ b/src/qemu/test_libvirtd_qemu.aug.in @@ -116,3 +116,4 @@ module Test_libvirtd_qemu =3D { "1" =3D "capname" } } { "deprecation_behavior" =3D "none" } +{ "sched_core" =3D "none" } --=20 2.35.1 From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331373; cv=none; d=zohomail.com; s=zohoarc; b=glmqNLAqcdN4q/pe9U1qQ5vVPsJVOBjkr3uy2mAVs6aVVyB0pSd7B4HWlDeSzPOvRNa00qnQWU15sv7iKU9/ZO/EZi/JO8gPiTWI7jp2XWeQIQXGEYV0ltsTMKaVA6lm9NqJVRW0eLyr5KMD03LlRbYj9klC3vnVM2Simu2ReX4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331373; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qFH5hXiWiljuOhrX0W708wJlcp/FidV9tjeitAW53ic=; b=T2Q29GSvRmZMWscYszSa2CEMSrPBxNn6q7XfgSXzzaPALnMJiq8VmVoSo5b4i60DW08ncIt1NANjYoVgLGGNa+9vZqA10Eobqi+lmfMTow8T2dmOdp7JHKMaqm29K2wvYNd1HiP0DylCrJapehXHQ75+KjiiByvx5XdPVfyZCA8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1661331373332949.4346264230771; Wed, 24 Aug 2022 01:56:13 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-605-GsimRitcPK6PnTr8g1UMqw-1; Wed, 24 Aug 2022 04:55:32 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1CC23C0ED7C; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9104D40315A; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 617C01946A42; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 51369194704C for ; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 35CCD2026D07; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id D5BEA2026D4C for ; Wed, 24 Aug 2022 08:55:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331371; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=qFH5hXiWiljuOhrX0W708wJlcp/FidV9tjeitAW53ic=; b=W83RGa1y0b6l7sacT7HeCFBHKi/TEMJQ2vAfrjvejyG56X2b9OAKP/DXiyIMWX9LUa6iTn ER1mIYHYI+EnckEVwMKpOjBuyjHuGP180U6Gyvin+qsJZVGpcHs6c51yoYNH9Pj6shKa/Z Q7s8+SUUckOF4w1/nlUAGtlB7aqCPf0= X-MC-Unique: GsimRitcPK6PnTr8g1UMqw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 4/8] qemu_domain: Introduce qemuDomainSchedCoreStart() Date: Wed, 24 Aug 2022 10:55:09 +0200 Message-Id: <711512bae455fc9cc5737da4912f42d7b32a2099.1661331156.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331375104100001 Content-Type: text/plain; charset="utf-8"; x-default="true" The aim of this helper function is to spawn a child process in which new scheduling group is created. This dummy process will then used to distribute scheduling group from (e.g. when starting helper processes or QEMU itself). The process is not needed for QEMU_SCHED_CORE_NONE case (obviously) nor for QEMU_SCHED_CORE_VCPUS case (because in that case a slightly different child will be forked off). Signed-off-by: Michal Privoznik --- src/qemu/qemu_domain.c | 105 ++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_domain.h | 12 +++++ src/qemu/qemu_process.c | 4 ++ 3 files changed, 121 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index d5fef76211..33376b9453 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1747,6 +1747,8 @@ qemuDomainObjPrivateFree(void *data) g_object_unref(priv->eventThread); } =20 + qemuDomainSchedCoreStop(priv); + g_free(priv); } =20 @@ -1778,6 +1780,9 @@ qemuDomainObjPrivateAlloc(void *opaque) priv->migMaxBandwidth =3D QEMU_DOMAIN_MIG_BANDWIDTH_MAX; priv->driver =3D opaque; =20 + priv->schedCoreChildPID =3D -1; + priv->schedCoreChildFD =3D -1; + return g_steal_pointer(&priv); } =20 @@ -11744,3 +11749,103 @@ qemuDomainObjWait(virDomainObj *vm) =20 return 0; } + + +int +qemuDomainSchedCoreStart(virQEMUDriverConfig *cfg, + virDomainObj *vm) +{ + qemuDomainObjPrivate *priv =3D vm->privateData; + int waitfd[2] =3D { -1, -1 }; + int syncfd[2] =3D { -1, -1 }; + pid_t child =3D -1; + + if (cfg->schedCore =3D=3D QEMU_SCHED_CORE_NONE || + cfg->schedCore =3D=3D QEMU_SCHED_CORE_VCPUS) { + /* We don't need any dummy process for any of these two variants. = */ + return 0; + } + + if (virPipe(waitfd) < 0 || + virPipe(syncfd) < 0) + return -1; + + if ((child =3D virFork()) < 0) + goto error; + + if (child =3D=3D 0) { + /* child */ + int rc; + char c; + + VIR_FORCE_CLOSE(waitfd[1]); + VIR_FORCE_CLOSE(syncfd[0]); + + errno =3D 0; + rc =3D virProcessSchedCoreCreate(); + c =3D errno; + ignore_value(safewrite(syncfd[1], &c, 1)); + VIR_FORCE_CLOSE(syncfd[1]); + + if (rc < 0) { + virReportSystemError(errno, "%s", + _("Unable to set SCHED_CORE")); + _exit(EXIT_FAILURE); + } + + ignore_value(saferead(waitfd[0], &c, 1)); + VIR_FORCE_CLOSE(waitfd[0]); + _exit(EXIT_SUCCESS); + } else { + /* parent */ + char c =3D '\0'; + VIR_FORCE_CLOSE(waitfd[0]); + VIR_FORCE_CLOSE(syncfd[1]); + + if (saferead(syncfd[0], &c, 1) < 0) { + virReportSystemError(errno, "%s", + _("unable to read from pipe")); + goto error; + } + VIR_FORCE_CLOSE(syncfd[0]); + + if (c !=3D 0) { + virReportSystemError(c, "%s", + _("Unable to set SCHED_CORE")); + goto error; + } + + VIR_DEBUG("Spawned dummy process for schedCore (%s) pid=3D%lld fd= =3D%d", + virQEMUSchedCoreTypeToString(cfg->schedCore), + (long long) child, waitfd[1]); + + priv->schedCoreChildPID =3D child; + priv->schedCoreChildFD =3D waitfd[1]; + } + + return 0; + + error: + VIR_FORCE_CLOSE(waitfd[0]); + VIR_FORCE_CLOSE(waitfd[1]); + VIR_FORCE_CLOSE(syncfd[0]); + VIR_FORCE_CLOSE(syncfd[1]); + return -1; +} + + +void +qemuDomainSchedCoreStop(qemuDomainObjPrivate *priv) +{ + if (priv->schedCoreChildFD !=3D -1) { + ignore_value(safewrite(priv->schedCoreChildFD, "q", 1)); + VIR_FORCE_CLOSE(priv->schedCoreChildFD); + } + + if (priv->schedCoreChildPID !=3D -1) { + VIR_DEBUG("Killing dummy procces for schedCore pid=3D%lld", + (long long) priv->schedCoreChildPID); + virProcessAbort(priv->schedCoreChildPID); + priv->schedCoreChildPID =3D -1; + } +} diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 592ee9805b..620830ccf5 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -244,6 +244,11 @@ struct _qemuDomainObjPrivate { =20 unsigned long long originalMemlock; /* Original RLIMIT_MEMLOCK, zero i= f no * restore will be required later = */ + + /* Info on dummy process for schedCore. A short lived process used only + * briefly when starting a guest. Don't save/parse into XML. */ + pid_t schedCoreChildPID; + pid_t schedCoreChildFD; }; =20 #define QEMU_DOMAIN_PRIVATE(vm) \ @@ -1098,3 +1103,10 @@ qemuDomainRemoveLogs(virQEMUDriver *driver, =20 int qemuDomainObjWait(virDomainObj *vm); + +int +qemuDomainSchedCoreStart(virQEMUDriverConfig *cfg, + virDomainObj *vm); + +void +qemuDomainSchedCoreStop(qemuDomainObjPrivate *priv); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 5c8413a6b6..f40f9acc09 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7503,6 +7503,9 @@ qemuProcessLaunch(virConnectPtr conn, if (qemuProcessGenID(vm, flags) < 0) goto cleanup; =20 + if (qemuDomainSchedCoreStart(cfg, vm) < 0) + goto cleanup; + if (qemuExtDevicesStart(driver, vm, incoming !=3D NULL) < 0) goto cleanup; =20 @@ -7775,6 +7778,7 @@ qemuProcessLaunch(virConnectPtr conn, ret =3D 0; =20 cleanup: + qemuDomainSchedCoreStop(priv); qemuDomainSecretDestroy(vm); return ret; } --=20 2.35.1 From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331336; cv=none; d=zohomail.com; s=zohoarc; b=MIKauIhiq1aKZkjmPoQp5caSgOhzsTNr7VIHx0KsiYeEW5H+S9F3aUuoYZ8YRUm7rtj0cq3UPQGg4+PQRKZxYzmtebH4A9Ht32jJGi4eKAYFWhaAwGIlPIFRLuA0sXBMbIw4iHnrrm8ZVt3UYyQ/ENnrBNwq8WKH8CGkCDOSJM0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331336; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8cwt8xFWVHzPNLx9WuUasvTD0tM0REruaZ0aiijzfxU=; b=JJ/z8I5wf8IknR9xVWi/xjBY254vhAADVVQI3h4hGE3mG8k+nb8dzJrDxP0LktR8JZzhCycMpORWgXGRFk07wT5dpGaJB4bR5f1ZBQn7PpD6GY67jBxyDLHUbTsIgJm4XVWNP5Fdw9bqN/StA55P8OFQqZk5GAjCcy4TgCqYN9M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1661331336393219.46331259687543; Wed, 24 Aug 2022 01:55:36 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-163-Nk6-ay9UN5Sr6tB5cRtkGA-1; Wed, 24 Aug 2022 04:55:31 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2F7B238008A8; Wed, 24 Aug 2022 08:55:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1D40E492C3B; Wed, 24 Aug 2022 08:55:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0FD8A1946A43; Wed, 24 Aug 2022 08:55:29 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E05FA1946A5A for ; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id C29CF2026D07; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C7B12026D64 for ; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331335; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=8cwt8xFWVHzPNLx9WuUasvTD0tM0REruaZ0aiijzfxU=; b=ConuGgADTRgO/XUh8yQY/KLf6gWxns40KXCaOP/6h2nzeapyucvvy9k5MIGf0r37fCtNo4 rMDCkcztxf5A8KQpYle+SgJaQbkbaaXc1NkApvR5Yw/qu5m4Fe556UBeCfv4ZnvjPP+2lj kkvEii7OpGgA13UpW4bwfKLZEQWfFcY= X-MC-Unique: Nk6-ay9UN5Sr6tB5cRtkGA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 5/8] qemu_process: Enable SCHED_CORE for QEMU process Date: Wed, 24 Aug 2022 10:55:10 +0200 Message-Id: <2da18136ec2f3a76127f172bb963f1ab82dbeaf2.1661331156.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331336707100001 For QEMU_SCHED_CORE_EMULATOR or QEMU_SCHED_CORE_FULL the QEMU process (and its vCPU threads) should be placed into its own scheduling group. Since we have the dummy process running for exactly this purpose use its PID as an argument to virCommandSetRunAmong(). Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrang=C3=A9 --- src/qemu/qemu_process.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index f40f9acc09..fa25207bfe 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7557,6 +7557,9 @@ qemuProcessLaunch(virConnectPtr conn, virCommandSetMaxProcesses(cmd, cfg->maxProcesses); if (cfg->maxFiles > 0) virCommandSetMaxFiles(cmd, cfg->maxFiles); + if (cfg->schedCore =3D=3D QEMU_SCHED_CORE_EMULATOR || + cfg->schedCore =3D=3D QEMU_SCHED_CORE_FULL) + virCommandSetRunAmong(cmd, priv->schedCoreChildPID); =20 /* In this case, however, zero means that core dumps should be * disabled, and so we always need to set the limit explicitly */ --=20 2.35.1 From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331338; cv=none; d=zohomail.com; s=zohoarc; b=Afw92I2jNZVT6SEJnyVPb7kEjddK39ZQK7OfJXoYHScvPb03KxJaSQxzLyYdBm8auF0oXkmiXtsCoOSAvjLXB9+LGuz3P6Ygcn92CeTL2Vnhx/SW9/4nl7kXIUmKG6QOUR+ypP/VHhlsE4tN9uQKS2/9U2T4W+q/UFvzf+Qq654= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331338; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=B+eNYhfmmQXZnansaZHcSaGLO2FjbGeZfEJr458gAGw=; b=BoRbVzaelh1hiwPlZFoQzzRvV7pjTJh6RqO0L6PVDdOXpdl8gkD2qbB7py1KnvLf92nz0fZbcZhkWRJz46+z1r8Gc3OVwiTe7PPkMuiHOHbDy8ZiRCv8FlUqSGsotRSM1ZmIrInENGum6xSEJTmIujMMyjPZj/6V7BLwU7ix21I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1661331338282558.440733962261; Wed, 24 Aug 2022 01:55:38 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-210-p3x4rw-SPM2_igWUmLKE2g-1; Wed, 24 Aug 2022 04:55:34 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2FB1280231E; Wed, 24 Aug 2022 08:55:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1CB352026D4C; Wed, 24 Aug 2022 08:55:32 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0CCF51946A42; Wed, 24 Aug 2022 08:55:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 758A21946A5C for ; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 5844F2024CC3; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id 046582024CBC for ; Wed, 24 Aug 2022 08:55:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331337; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=B+eNYhfmmQXZnansaZHcSaGLO2FjbGeZfEJr458gAGw=; b=if2ewyiz+3PIX/2Yc+JMIk+K28yNzn8Eu5rjeFQPPKWt2391q4oCkzcpUNZ/M2y31c6eSw GIfez4KB8zZD8DayM+xujVyRVb58r+Zr4qxvS1cBaHqEloE/yLCyTx57Eith881oZxjfDj 6nGJ+MnXeWCcsqU/JVx5IVefr+YlUe0= X-MC-Unique: p3x4rw-SPM2_igWUmLKE2g-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 6/8] qemu: Enable SCHED_CORE for helper processes Date: Wed, 24 Aug 2022 10:55:11 +0200 Message-Id: <048e0c4a7d0439fefe51edf317ebf8bbaeba8680.1661331156.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331338730100006 For QEMU_SCHED_CORE_FULL case, all helper processes should be placed into the same scheduling group as the QEMU process they serve. It may happen though, that a helper process is started before QEMU (cold start of a domain). But we have the dummy process running from which the QEMU process will inherit the scheduling group, so we can use the dummy process PID as an argument to virCommandSetRunAmong(). Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrang=C3=A9 --- src/qemu/qemu_process.c | 9 +++++++++ src/qemu/qemu_security.c | 11 +++++++++++ src/qemu/qemu_virtiofs.c | 11 +++++++++++ 3 files changed, 31 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index fa25207bfe..dbd4408a8a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -2852,6 +2852,15 @@ qemuProcessStartManagedPRDaemon(virDomainObj *vm) * qemu (so that it shares the same view of the system). */ virCommandSetPreExecHook(cmd, qemuProcessStartPRDaemonHook, vm); =20 + if (cfg->schedCore =3D=3D QEMU_SCHED_CORE_FULL) { + pid_t cookie_pid =3D vm->pid; + + if (cookie_pid <=3D 0) + cookie_pid =3D priv->schedCoreChildPID; + + virCommandSetRunAmong(cmd, cookie_pid); + } + if (virCommandRun(cmd, NULL) < 0) goto cleanup; =20 diff --git a/src/qemu/qemu_security.c b/src/qemu/qemu_security.c index 3be1766764..5b7d5f30c2 100644 --- a/src/qemu/qemu_security.c +++ b/src/qemu/qemu_security.c @@ -683,6 +683,9 @@ qemuSecurityCommandRun(virQEMUDriver *driver, int *exitstatus, int *cmdret) { + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + qemuDomainObjPrivate *priv =3D vm->privateData; + if (virSecurityManagerSetChildProcessLabel(driver->securityManager, vm->def, cmd) < 0) return -1; @@ -691,6 +694,14 @@ qemuSecurityCommandRun(virQEMUDriver *driver, virCommandSetUID(cmd, uid); if (gid !=3D (gid_t) -1) virCommandSetGID(cmd, gid); + if (cfg->schedCore =3D=3D QEMU_SCHED_CORE_FULL) { + pid_t pid =3D vm->pid; + + if (pid <=3D 0) + pid =3D priv->schedCoreChildPID; + + virCommandSetRunAmong(cmd, pid); + } =20 if (virSecurityManagerPreFork(driver->securityManager) < 0) return -1; diff --git a/src/qemu/qemu_virtiofs.c b/src/qemu/qemu_virtiofs.c index a04aa08e39..cd947bebfd 100644 --- a/src/qemu/qemu_virtiofs.c +++ b/src/qemu/qemu_virtiofs.c @@ -178,6 +178,7 @@ qemuVirtioFSStart(virQEMUDriver *driver, virDomainFSDef *fs) { g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + qemuDomainObjPrivate *priv =3D vm->privateData; g_autoptr(virCommand) cmd =3D NULL; g_autofree char *socket_path =3D NULL; g_autofree char *pidfile =3D NULL; @@ -251,6 +252,16 @@ qemuVirtioFSStart(virQEMUDriver *driver, virCommandNonblockingFDs(cmd); virCommandDaemonize(cmd); =20 + if (cfg->schedCore =3D=3D QEMU_SCHED_CORE_FULL) { + pid_t cookie_pid =3D vm->pid; + + if (cookie_pid <=3D 0) + cookie_pid =3D priv->schedCoreChildPID; + + virCommandSetRunAmong(cmd, cookie_pid); + } + + if (qemuExtDeviceLogCommand(driver, vm, cmd, "virtiofsd") < 0) goto error; =20 --=20 2.35.1 From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331337; cv=none; d=zohomail.com; s=zohoarc; b=KpTjM4v/k5IBkpplBTwg4sOMgySO2pKN5f3mgutXUQKnIpwar93549kDXxlQRfPx8cbZdJ+r+GcYUw8cCCbtSsFWiGKVkSGHBOCCZx2f1SCbS9hWBKFh4LWLdravqs7pcsXb1zEZleAP4Mevr9BAGf/w3yEzQGH3S/JzUkYxVG0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331337; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bkE7qYGfk61TkobCd6o+d/0lh39YCCYH7gqXK3TiUFQ=; b=m7VKPl7LM8GsjXH1zCMdnfbH6SFBnCztCMpkTQi1DgkWEYgvRQDT2BL4kULgfxVifdPV+oBk9fL6faDy7Elmnhorw4oDXYZhSebvx1k/zkV1gkjgR5lKrI5CYEA4SIYjBEhkSKBHQzQvK6D9rGvPx1m5OiPGy9FG3PNCM9zqJ/c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1661331337029292.6513833426736; Wed, 24 Aug 2022 01:55:37 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-82-fAFOyjFwMwOO6rN2g1cWsw-1; Wed, 24 Aug 2022 04:55:32 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8AF871C14D31; Wed, 24 Aug 2022 08:55:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 79A641415125; Wed, 24 Aug 2022 08:55:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6E5B51946A42; Wed, 24 Aug 2022 08:55:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0D9461946A40 for ; Wed, 24 Aug 2022 08:55:29 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id E62C92024CC2; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id 915BE2024CBC for ; Wed, 24 Aug 2022 08:55:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331336; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=bkE7qYGfk61TkobCd6o+d/0lh39YCCYH7gqXK3TiUFQ=; b=Py+HjtBaYfUUXyk8mamGcbwDYnG3QLMSFiIgVZTqccalmJYgXD5Z6HgPtUTaqLLhZoqqZ4 EOu5fYJkCSfOfl2oV+jsGaxsS7Bw/RwPrX+eklZ9H+PKA6LBD+7knQsZjx79RykzmVTT+v Tf/kIuWvIebDXLlvBPGx+QJM9k0U11Q= X-MC-Unique: fAFOyjFwMwOO6rN2g1cWsw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 7/8] qemu: Enable SCHED_CORE for vCPUs Date: Wed, 24 Aug 2022 10:55:12 +0200 Message-Id: <61a7ad476e55c42bd3a5bd70ddbd12306010e09a.1661331156.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331338717100005 For QEMU_SCHED_CORE_VCPUS case, the vCPU threads should be placed all into one scheduling group, but not the emulator or any of its threads. Therefore, as soon as vCPU TIDs are detected, fork off a child which then creates a separate scheduling group and adds all vCPU threads into it. Please note, this commit only handles the cold boot case. Hotplug is going to be implemented in the next commit. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrang=C3=A9 --- src/qemu/qemu_process.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index dbd4408a8a..83fcb5518a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5875,9 +5875,44 @@ qemuProcessSetupVcpu(virDomainObj *vm, } =20 =20 +static int +qemuProcessSetupAllVcpusSchedCoreHelper(pid_t ppid G_GNUC_UNUSED, + void *opaque) +{ + virDomainObj *vm =3D opaque; + size_t i; + + /* Since we are setting all vCPU threads at once and from a forked off + * child, we don't need the dummy schedCoreChildPID and can create one= on + * our own. */ + if (virProcessSchedCoreCreate() < 0) { + virReportSystemError(errno, "%s", + _("Unable to set SCHED_CORE")); + + return -1; + } + + for (i =3D 0; i < virDomainDefGetVcpusMax(vm->def); i++) { + pid_t vcpupid =3D qemuDomainGetVcpuPid(vm, i); + + if (vcpupid > 0 && + virProcessSchedCoreShareTo(vcpupid) < 0) { + virReportSystemError(errno, + _("unable to share scheduling cookie to %= lld"), + (long long) vcpupid); + return -1; + } + } + + return 0; +} + + static int qemuProcessSetupVcpus(virDomainObj *vm) { + qemuDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(priv->dr= iver); virDomainVcpuDef *vcpu; unsigned int maxvcpus =3D virDomainDefGetVcpusMax(vm->def); size_t i; @@ -5920,6 +5955,10 @@ qemuProcessSetupVcpus(virDomainObj *vm) return -1; } =20 + if (cfg->schedCore =3D=3D QEMU_SCHED_CORE_VCPUS && + virProcessRunInFork(qemuProcessSetupAllVcpusSchedCoreHelper, vm) <= 0) + return -1; + return 0; } =20 --=20 2.35.1 From nobody Fri May 17 03:54:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1661331338; cv=none; d=zohomail.com; s=zohoarc; b=dhLM9Q7uc7IDlBdg0VLWW60xes6BvtbLTpqL1H49b5agMW6PR6BhXdI+nG4sxox7IVa+wHJq6N6ypuZwpETUkaTgOStTe8weAiQ6SgmoqhG0B0vdYs+jw6xVA5wm81vbFVApkUL/rZdCRJvjFncoQNu/OH5e7N2PBpBaJvOFqnQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661331338; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hyWV/Tn/S2JXKi/aTug4LGVl5s/B9tgu3GIENetT/L8=; b=AkCA4WuFT2KkR9WPNz0sIL4jlh4aPWus8cYMnNA6hbdXUqhvE21sTotbInU2nBhBmdGOMyAKw5oGHWN9gsVduij+s9mopbhhUT0lBUEkQCefAXxpkdf2f7wHN627+vOvwd1Rw5x8vScVVi8JZC1l1t5j5c4t1sy7jAt70Nmrfak= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1661331338133322.324777532869; Wed, 24 Aug 2022 01:55:38 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-412-J4JuTL4DOn-aO3HLUmyztQ-1; Wed, 24 Aug 2022 04:55:33 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D9D6B18A6509; Wed, 24 Aug 2022 08:55:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id C7284C15BC3; Wed, 24 Aug 2022 08:55:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id AD9E21946A62; Wed, 24 Aug 2022 08:55:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8734A1946A42 for ; Wed, 24 Aug 2022 08:55:29 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 7C4DD2024CBC; Wed, 24 Aug 2022 08:55:29 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id 285992026D64 for ; Wed, 24 Aug 2022 08:55:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661331337; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=hyWV/Tn/S2JXKi/aTug4LGVl5s/B9tgu3GIENetT/L8=; b=bMEINYMZx/1FEd7SF2G7e3zsI2U0Cu0FL23gDkGAm6b2UWdbKbhorOnyyHRrGV2fQ+jAI3 0A1HUSztcmLB7l2upUv/xZn8lEYxkv77ZRku/5XhwioKMKSLh8CZ7pLjENAVgMtY6m7twf uxcujy4bBv0gzBReTgdYUi1LtIuXjyQ= X-MC-Unique: J4JuTL4DOn-aO3HLUmyztQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 8/8] qemu: Enable for vCPUs on hotplug Date: Wed, 24 Aug 2022 10:55:13 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1661331338731100007 Content-Type: text/plain; charset="utf-8"; x-default="true" As advertised in the previous commit, QEMU_SCHED_CORE_VCPUS case is implemented for hotplug case. The implementation is very similar to the cold boot case, except here we fork off for every vCPU (because the implementation is done in qemuProcessSetupVcpu() which is also the function that's called from hotplug code). But that's okay because our hotplug APIs allow hotplugging one device at the time. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D2074559 Signed-off-by: Michal Privoznik --- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_process.c | 61 +++++++++++++++++++++++++++++++++++++++-- src/qemu/qemu_process.h | 3 +- 3 files changed, 62 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 751fa84013..5f3430d2ba 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -6243,7 +6243,7 @@ qemuDomainHotplugAddVcpu(virQEMUDriver *driver, vcpuinfo->online =3D true; =20 if (vcpupriv->tid > 0 && - qemuProcessSetupVcpu(vm, i) < 0) + qemuProcessSetupVcpu(vm, i, true) < 0) return -1; } =20 diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 83fcb5518a..6e454beb5c 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5818,10 +5818,40 @@ qemuProcessNetworkPrepareDevices(virQEMUDriver *dri= ver, } =20 =20 +struct qemuProcessSetupVcpuSchedCoreHelperData { + pid_t vcpupid; + pid_t dummypid; +}; + +static int +qemuProcessSetupVcpuSchedCoreHelper(pid_t ppid G_GNUC_UNUSED, + void *opaque) +{ + struct qemuProcessSetupVcpuSchedCoreHelperData *data =3D opaque; + + if (virProcessSchedCoreShareFrom(data->dummypid) < 0) { + virReportSystemError(errno, + _("unable to share scheduling cookie from %ll= d"), + (long long) data->dummypid); + return -1; + } + + if (virProcessSchedCoreShareTo(data->vcpupid) < 0) { + virReportSystemError(errno, + _("unable to share scheduling cookie to %lld"= ), + (long long) data->vcpupid); + return -1; + } + + return 0; +} + + /** * qemuProcessSetupVcpu: * @vm: domain object * @vcpuid: id of VCPU to set defaults + * @schedCore: whether to set scheduling group * * This function sets resource properties (cgroups, affinity, scheduler) f= or a * vCPU. This function expects that the vCPU is online and the vCPU pids w= ere @@ -5831,8 +5861,11 @@ qemuProcessNetworkPrepareDevices(virQEMUDriver *driv= er, */ int qemuProcessSetupVcpu(virDomainObj *vm, - unsigned int vcpuid) + unsigned int vcpuid, + bool schedCore) { + qemuDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(priv->dr= iver); pid_t vcpupid =3D qemuDomainGetVcpuPid(vm, vcpuid); virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, vcpuid); virDomainResctrlMonDef *mon =3D NULL; @@ -5845,6 +5878,30 @@ qemuProcessSetupVcpu(virDomainObj *vm, &vcpu->sched) < 0) return -1; =20 + if (schedCore && + cfg->schedCore =3D=3D QEMU_SCHED_CORE_VCPUS) { + struct qemuProcessSetupVcpuSchedCoreHelperData data =3D { .vcpupid= =3D vcpupid, + .dummypid =3D -1 }; + + for (i =3D 0; i < virDomainDefGetVcpusMax(vm->def); i++) { + pid_t temptid =3D qemuDomainGetVcpuPid(vm, i); + + if (temptid > 0) { + data.dummypid =3D temptid; + break; + } + } + + if (data.dummypid =3D=3D -1) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to find a vCPU that is online")); + return -1; + } + + if (virProcessRunInFork(qemuProcessSetupVcpuSchedCoreHelper, &data= ) < 0) + return -1; + } + for (i =3D 0; i < vm->def->nresctrls; i++) { size_t j =3D 0; virDomainResctrlDef *ct =3D vm->def->resctrls[i]; @@ -5951,7 +6008,7 @@ qemuProcessSetupVcpus(virDomainObj *vm) if (!vcpu->online) continue; =20 - if (qemuProcessSetupVcpu(vm, i) < 0) + if (qemuProcessSetupVcpu(vm, i, false) < 0) return -1; } =20 diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index 421efc6016..4dfb2485c0 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -187,7 +187,8 @@ int qemuConnectAgent(virQEMUDriver *driver, virDomainOb= j *vm); =20 =20 int qemuProcessSetupVcpu(virDomainObj *vm, - unsigned int vcpuid); + unsigned int vcpuid, + bool schedCore); int qemuProcessSetupIOThread(virDomainObj *vm, virDomainIOThreadIDDef *iothread); =20 --=20 2.35.1