From nobody Sat Oct 25 23:41:40 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1516079172315766.1826410374068; Mon, 15 Jan 2018 21:06:12 -0800 (PST) Received: from localhost ([::1]:35342 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ebJRn-0006sP-Ar for importer@patchew.org; Tue, 16 Jan 2018 00:06:07 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47907) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ebJ9j-0007zB-7q for qemu-devel@nongnu.org; Mon, 15 Jan 2018 23:47:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ebJ9g-0004mV-5H for qemu-devel@nongnu.org; Mon, 15 Jan 2018 23:47:27 -0500 Received: from ozlabs.org ([103.22.144.67]:57917) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ebJ9f-0004lD-Ip; Mon, 15 Jan 2018 23:47:24 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 3zLHkt1bJRz9t2c; Tue, 16 Jan 2018 15:47:17 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1516078038; bh=FTAK8DGGpVFbnaHFhXG1/EUCwpaieM2NlAZPPaoYVP4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g+Ft0pN07KQcGIokzkObw3LstGN/irNnHjuc56yiEYyyLtsBi8Rm6aF/OWkexoFlA 3aLsUwmDAZt5cyjX+iU4CZUiYqcaHHBoNyaI2DPITyBHARFo1I9rx/do+++gSdrHe9 gLu+An1wcAi/kdNCb/U/R4h7uHZe6xRGQB8N/QoY= From: David Gibson To: groug@kaod.org, joserz@linux.vnet.ibm.com, surajjs@au1.ibm.com, sam.bobroff@au1.ibm.com Date: Tue, 16 Jan 2018 15:47:13 +1100 Message-Id: <20180116044714.12571-2-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180116044714.12571-1-david@gibson.dropbear.id.au> References: <20180116044714.12571-1-david@gibson.dropbear.id.au> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 103.22.144.67 Subject: [Qemu-devel] [PATCHv2 1/2] spapr: Allow some cases where we can't set VSMT mode in the kernel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" At present if we require a vsmt mode that's not equal to the kernel's default, and the kernel doesn't let us change it (e.g. because it's an old kernel without support) then we always fail. But in fact we can cope with the kernel having a different vsmt as long as a) it's >=3D the actual number of vthreads/vcore (so that guest threads that are supposed to be on the same core act like it) b) it's a submultiple of the requested vsmt mode (so that guest threads spaced by the vsmt value will act like they're on different cores) Allowing this case gives us a bit more freedom to adjust the vsmt behaviour without breaking existing cases. Signed-off-by: David Gibson Reviewed-by: Greg Kurz Reviewed-by: Laurent Vivier Tested-by: Greg Kurz --- hw/ppc/spapr.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index e35214bfc3..6d3613d934 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2314,17 +2314,29 @@ static void spapr_set_vsmt_mode(sPAPRMachineState *= spapr, Error **errp) if (kvm_enabled() && (spapr->vsmt !=3D kvm_smt)) { ret =3D kvmppc_set_smt_threads(spapr->vsmt); if (ret) { + /* Looks like KVM isn't able to change VSMT mode */ error_setg(&local_err, "Failed to set KVM's VSMT mode to %d (errno %d)", spapr->vsmt, ret); - if (!vsmt_user) { - error_append_hint(&local_err, "On PPC, a VM with %d thread= s/" - "core on a host with %d threads/core requires= " - " the use of VSMT mode %d.\n", - smp_threads, kvm_smt, spapr->vsmt); + /* We can live with that if the default one is big enough + * for the number of threads, and a submultiple of the one + * we want. In this case we'll waste some vcpu ids, but + * behaviour will be correct */ + if ((kvm_smt >=3D smp_threads) && (spapr->vsmt % kvm_smt) =3D= =3D 0) { + warn_report_err(local_err); + local_err =3D NULL; + goto out; + } else { + if (!vsmt_user) { + error_append_hint(&local_err, + "On PPC, a VM with %d threads/core" + " on a host with %d threads/core" + " requires the use of VSMT mode %d.\= n", + smp_threads, kvm_smt, spapr->vsmt); + } + kvmppc_hint_smt_possible(&local_err); + goto out; } - kvmppc_hint_smt_possible(&local_err); - goto out; } } /* else TCG: nothing to do currently */ --=20 2.14.3 From nobody Sat Oct 25 23:41:40 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1516079047271157.48194013523687; Mon, 15 Jan 2018 21:04:07 -0800 (PST) Received: from localhost ([::1]:35327 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ebJPq-00058g-HQ for importer@patchew.org; Tue, 16 Jan 2018 00:04:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47895) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ebJ9h-0007ys-Ft for qemu-devel@nongnu.org; Mon, 15 Jan 2018 23:47:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ebJ9g-0004mg-6D for qemu-devel@nongnu.org; Mon, 15 Jan 2018 23:47:25 -0500 Received: from ozlabs.org ([2401:3900:2:1::2]:48787) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ebJ9f-0004lG-J4; Mon, 15 Jan 2018 23:47:24 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 3zLHkt3GdTz9t3Z; Tue, 16 Jan 2018 15:47:18 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1516078038; bh=xsEiuGlto+GJ7th57A1bmBXfpBzbq/SuJtF2k3xerPE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fgu4yzfzrrHNWZ/fNLKL9M0pgJ0WAq6vCOi8WTuv9kFci1O2vfuci2JZ9Y0KMPyKn PhPYjhxaCPbt51uFklSU6wp2lk01BXY3M6i9VCtdFUDd6Zr8/4kZYae7fdNWaiAhHd PItMZuLE9rEwKIV2vfnqMIKWAL5SIDebe6/lT3U4= From: David Gibson To: groug@kaod.org, joserz@linux.vnet.ibm.com, surajjs@au1.ibm.com, sam.bobroff@au1.ibm.com Date: Tue, 16 Jan 2018 15:47:14 +1100 Message-Id: <20180116044714.12571-3-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180116044714.12571-1-david@gibson.dropbear.id.au> References: <20180116044714.12571-1-david@gibson.dropbear.id.au> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2401:3900:2:1::2 Subject: [Qemu-devel] [PATCHv2 2/2] spapr: Adjust default VSMT value for better migration compatibility X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" fa98fbfc "PC: KVM: Support machine option to set VSMT mode" introduced the "vsmt" parameter for the pseries machine type, which controls the spacing of the vcpu ids of thread 0 for each virtual core. This was done to bring some consistency and stability to how that was done, while still allowing backwards compatibility for migration and otherwise. The default value we used for vsmt was set to the max of the host's advertised default number of threads and the number of vthreads per vcore in the guest. This was done to continue running without extra parameters on older KVM versions which don't allow the VSMT value to be changed. Unfortunately, even that smaller than before leakage of host configuration into guest visible configuration still breaks things. Specifically a guest with 4 (or less) vthread/vcore will get a different vsmt value when running on a POWER8 (vsmt=3D=3D8) and POWER9 (vsmt=3D=3D4) host. That mean= s the vcpu ids don't line up so you can't migrate between them, though you should be able to. Long term we really want to make vsmt =3D=3D smp_threads for sufficiently new machine types. However, that means that qemu will then require a sufficiently recent KVM (one which supports changing VSMT) - that's still not widely enough deployed to be really comfortable to do. In the meantime we need some default that will work as often as possible. This patch changes that default to 8 in all circumstances. This does change guest visible behaviour (including for existing machine versions) for many cases - just not the most common/important case. Following is case by case justification for why this is still the least worst option. Note that any of the old behaviours can still be duplicated after this patch, it's just that it requires manual intervention by setting the vsmt property on the command line. KVM HV on POWER8 host: This is the overwhelmingly common case in production setups, and is unchanged by design. POWER8 hosts will advertise a default VSMT mode of 8, and > 8 vthreads/vcore isn't permitted KVM HV on POWER7 host: Will break, but POWER7s allowing KVM were never released to the public. KVM HV on POWER9 host: Not yet released to the public, breaking this now will reduce other breakage later. KVM HV on PowerPC 970: Will theoretically break it, but it was barely supported to begin with and already required various user visible hacks to work. Also so old that I just don't care. TCG: This is the nastiest one; it means migration of TCG guests (without manual vsmt setting) will break. Since TCG is rarely used in production I think this is worth it for the other benefits. It does also remove one more barrier to TCG<->KVM migration which could be interesting for debugging applications. KVM PR: As with TCG, this will break migration of existing configurations, without adding extra manual vsmt options. As with TCG, it is rare in production so I think the benefits outweigh breakages. Signed-off-by: David Gibson Reviewed-by: Laurent Vivier Reviewed-by: Jose Ricardo Ziviani Reviewed-by: Greg Kurz --- hw/ppc/spapr.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 6d3613d934..a216ceada8 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2305,9 +2305,14 @@ static void spapr_set_vsmt_mode(sPAPRMachineState *s= papr, Error **errp) } /* In this case, spapr->vsmt has been set by the command line */ } else { - /* Choose a VSMT mode that may be higher than necessary but is - * likely to be compatible with hosts that don't have VSMT. */ - spapr->vsmt =3D MAX(kvm_smt, smp_threads); + /* + * Default VSMT value is tricky, because we need it to be as + * consistent as possible (for migration), but this requires + * changing it for at least some existing cases. We pick 8 as + * the value that we'd get with KVM on POWER8, the + * overwhelmingly common case in production systems. + */ + spapr->vsmt =3D 8; } =20 /* KVM: If necessary, set the SMT mode: */ --=20 2.14.3