From nobody Wed May 8 15:32:23 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1511507283131404.03168671037236; Thu, 23 Nov 2017 23:08:03 -0800 (PST) Received: from localhost ([::1]:47519 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eI85i-00047t-Aa for importer@patchew.org; Fri, 24 Nov 2017 02:08:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33615) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eI83s-0002Wc-By for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eI83n-0007o3-DQ for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:08 -0500 Received: from 19.mo6.mail-out.ovh.net ([188.165.56.177]:38325) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eI83n-0007lg-85 for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:03 -0500 Received: from player696.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo6.mail-out.ovh.net (Postfix) with ESMTP id CEEB6120D9C for ; Fri, 24 Nov 2017 08:06:01 +0100 (CET) Received: from zorba.kaod.org (LFbn-1-2231-173.w90-76.abo.wanadoo.fr [90.76.52.173]) (Authenticated sender: clg@kaod.org) by player696.ha.ovh.net (Postfix) with ESMTPSA id A4E883C0070; Fri, 24 Nov 2017 08:05:56 +0100 (CET) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson , Nikunj A Dadhania Date: Fri, 24 Nov 2017 08:05:48 +0100 Message-Id: <20171124070550.6433-2-clg@kaod.org> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171124070550.6433-1-clg@kaod.org> References: <20171124070550.6433-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14184368502132673363 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrledugddutdehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddm Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 188.165.56.177 Subject: [Qemu-devel] [PATCH v4 1/3] spapr/rtas: disable the decrementer interrupt when a CPU is unplugged X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" When a CPU is stopped with the 'stop-self' RTAS call, its state 'halted' is switched to 1 and, in this case, the MSR is not taken into account anymore in the cpu_has_work() routine. Only the pending hardware interrupts are checked with their LPCR:PECE* enablement bit. If the DECR timer fires after 'stop-self' is called and before the CPU 'stop' state is reached, the nearly-dead CPU will have some work to do and the guest will crash. This case happens very frequently with the not yet upstream P9 XIVE exploitation mode. In XICS mode, the DECR is occasionally fired but after 'stop' state, so no work is to be done and the guest survives. I suspect there is a race between the QEMU mainloop triggering the timers and the TCG CPU thread but I could not quite identify the root cause. To be safe, let's disable in the LPCR all the exceptions which can cause an exit while the CPU is in power-saving mode and reenable them when the CPU is started. Signed-off-by: C=C3=A9dric Le Goater --- Changes in v4: - used the 'lpcr_pm' field of PowerPCCPUClass Changes in v3: - introduced a cpu_ppc_papr_pece_bits() helper to gather the PECE bits depending on the CPU family. =20 - enabled Power-saving mode Exit Cause exceptions only on the boot CPU. =20 Changes in v2: - used a new routine ppc_cpu_pvr_match() to discriminate CPU versions - removed the LPCR:PECE* enablement bit when the CPU is initialized if it is a secondary hw/ppc/spapr_rtas.c | 11 +++++++++++ target/ppc/translate_init.c | 9 ++++++--- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c index cdf0b607a0a0..858adb1bf3a9 100644 --- a/hw/ppc/spapr_rtas.c +++ b/hw/ppc/spapr_rtas.c @@ -162,6 +162,7 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachi= neState *spapr, if (cpu !=3D NULL) { CPUState *cs =3D CPU(cpu); CPUPPCState *env =3D &cpu->env; + PowerPCCPUClass *pcc =3D POWERPC_CPU_GET_CLASS(cpu); =20 if (!cs->halted) { rtas_st(rets, 0, RTAS_OUT_HW_ERROR); @@ -174,6 +175,10 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMach= ineState *spapr, kvm_cpu_synchronize_state(cs); =20 env->msr =3D (1ULL << MSR_SF) | (1ULL << MSR_ME); + + /* Enable Power-saving mode Exit Cause exceptions for the new CPU = */ + env->spr[SPR_LPCR] |=3D pcc->lpcr_pm; + env->nip =3D start; env->gpr[3] =3D r3; cs->halted =3D 0; @@ -197,6 +202,7 @@ static void rtas_stop_self(PowerPCCPU *cpu, sPAPRMachin= eState *spapr, { CPUState *cs =3D CPU(cpu); CPUPPCState *env =3D &cpu->env; + PowerPCCPUClass *pcc =3D POWERPC_CPU_GET_CLASS(cpu); =20 cs->halted =3D 1; qemu_cpu_kick(cs); @@ -210,6 +216,11 @@ static void rtas_stop_self(PowerPCCPU *cpu, sPAPRMachi= neState *spapr, * no need to bother with specific bits, we just clear it. */ env->msr =3D 0; + + /* Disable Power-saving mode Exit Cause exceptions for the CPU. + * This could deliver an interrupt on a dying CPU and crash the + * guest */ + env->spr[SPR_LPCR] &=3D ~pcc->lpcr_pm; } =20 static inline int sysparm_st(target_ulong addr, target_ulong len, diff --git a/target/ppc/translate_init.c b/target/ppc/translate_init.c index 828d7e778c3b..78a4a581bab7 100644 --- a/target/ppc/translate_init.c +++ b/target/ppc/translate_init.c @@ -8911,6 +8911,7 @@ void cpu_ppc_set_papr(PowerPCCPU *cpu, PPCVirtualHype= rvisor *vhyp) CPUPPCState *env =3D &cpu->env; ppc_spr_t *lpcr =3D &env->spr_cb[SPR_LPCR]; ppc_spr_t *amor =3D &env->spr_cb[SPR_AMOR]; + CPUState *cs =3D CPU(cpu); =20 cpu->vhyp =3D vhyp; =20 @@ -8953,10 +8954,12 @@ void cpu_ppc_set_papr(PowerPCCPU *cpu, PPCVirtualHy= pervisor *vhyp) } } =20 - /* Also set the power-saving mode bits which depend on the CPU - * family + /* Only enable Power-saving mode Exit Cause exceptions on the boot + * CPU. The RTAS command start-cpu will enable them on secondaries. */ - lpcr->default_value |=3D pcc->lpcr_pm; + if (cs =3D=3D first_cpu) { + lpcr->default_value |=3D pcc->lpcr_pm; + } =20 /* We should be followed by a CPU reset but update the active value * just in case... --=20 2.13.6 From nobody Wed May 8 15:32:23 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1511507430633830.1418317295972; Thu, 23 Nov 2017 23:10:30 -0800 (PST) Received: from localhost ([::1]:47536 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eI880-0005hw-E8 for importer@patchew.org; Fri, 24 Nov 2017 02:10:24 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33712) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eI83y-0002a5-Ai for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eI83s-0007sM-FA for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:14 -0500 Received: from 3.mo6.mail-out.ovh.net ([178.33.253.26]:33421) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eI83s-0007rP-A0 for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:08 -0500 Received: from player696.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo6.mail-out.ovh.net (Postfix) with ESMTP id F3EBF120D92 for ; Fri, 24 Nov 2017 08:06:06 +0100 (CET) Received: from zorba.kaod.org (LFbn-1-2231-173.w90-76.abo.wanadoo.fr [90.76.52.173]) (Authenticated sender: clg@kaod.org) by player696.ha.ovh.net (Postfix) with ESMTPSA id CD1553C0070; Fri, 24 Nov 2017 08:06:01 +0100 (CET) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson , Nikunj A Dadhania Date: Fri, 24 Nov 2017 08:05:49 +0100 Message-Id: <20171124070550.6433-3-clg@kaod.org> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171124070550.6433-1-clg@kaod.org> References: <20171124070550.6433-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14185775878709349203 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrledugddutdehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddm Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 178.33.253.26 Subject: [Qemu-devel] [PATCH v4 2/3] spapr/rtas: fix reboot of a a SMP TCG guest X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Just like for hot unplug CPUs, when a guest is rebooted, the secondary CPUs can be awaken by the decrementer and start entering SLOF at the same time the boot CPU is. To be safe, let's disable on the secondaries all the exceptions which can cause an exit while the CPU is in power-saving mode. Based on previous work from Nikunj A Dadhania Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- Changes in v4: - used the 'lpcr_pm' field of PowerPCCPUClass Changes in v3: - used the cpu_ppc_papr_pece_bits() helper=20 hw/ppc/spapr_cpu_core.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c index 3a4c17401226..a2290528542d 100644 --- a/hw/ppc/spapr_cpu_core.c +++ b/hw/ppc/spapr_cpu_core.c @@ -26,6 +26,7 @@ static void spapr_cpu_reset(void *opaque) PowerPCCPU *cpu =3D opaque; CPUState *cs =3D CPU(cpu); CPUPPCState *env =3D &cpu->env; + PowerPCCPUClass *pcc =3D POWERPC_CPU_GET_CLASS(cpu); =20 cpu_reset(cs); =20 @@ -35,6 +36,13 @@ static void spapr_cpu_reset(void *opaque) cs->halted =3D 1; =20 env->spr[SPR_HIOR] =3D 0; + + /* Disable Power-saving mode Exit Cause exceptions for the CPU. + * This can cause issues when rebooting the guest if a secondary + * is awaken */ + if (cs !=3D first_cpu) { + env->spr[SPR_LPCR] &=3D ~pcc->lpcr_pm; + } } =20 static void spapr_cpu_destroy(PowerPCCPU *cpu) --=20 2.13.6 From nobody Wed May 8 15:32:23 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1511507291400767.8409792336347; Thu, 23 Nov 2017 23:08:11 -0800 (PST) Received: from localhost ([::1]:47520 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eI85j-000491-7W for importer@patchew.org; Fri, 24 Nov 2017 02:08:03 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33753) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eI841-0002cI-Iz for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eI83x-0007v0-Gv for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:17 -0500 Received: from 13.mo6.mail-out.ovh.net ([188.165.56.124]:33434) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eI83x-0007uQ-BH for qemu-devel@nongnu.org; Fri, 24 Nov 2017 02:06:13 -0500 Received: from player696.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo6.mail-out.ovh.net (Postfix) with ESMTP id 27703120D8B for ; Fri, 24 Nov 2017 08:06:12 +0100 (CET) Received: from zorba.kaod.org (LFbn-1-2231-173.w90-76.abo.wanadoo.fr [90.76.52.173]) (Authenticated sender: clg@kaod.org) by player696.ha.ovh.net (Postfix) with ESMTPSA id 00B6A3C0070; Fri, 24 Nov 2017 08:06:06 +0100 (CET) From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson , Nikunj A Dadhania Date: Fri, 24 Nov 2017 08:05:50 +0100 Message-Id: <20171124070550.6433-4-clg@kaod.org> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171124070550.6433-1-clg@kaod.org> References: <20171124070550.6433-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14187464728613391187 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrledugddutdehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddm Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 188.165.56.124 Subject: [Qemu-devel] [PATCH v4 3/3] spapr/rtas: do not reset the MSR in stop-self command X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" When a CPU is stopped with the 'stop-self' RTAS call, its state 'halted' is switched to 1 and, in this case, the MSR is not taken into account anymore in the cpu_has_work() routine. Only the pending hardware interrupts are checked with their LPCR:PECE* enablement bit. The CPU is now also protected from the decrementer interrupt by the LPCR:PECE* bits which are disabled in the 'stop-self' RTAS call. Reseting the MSR is pointless. Signed-off-by: C=C3=A9dric Le Goater Reviewed-by: David Gibson --- hw/ppc/spapr_rtas.c | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c index 858adb1bf3a9..4bb939d3d111 100644 --- a/hw/ppc/spapr_rtas.c +++ b/hw/ppc/spapr_rtas.c @@ -206,16 +206,6 @@ static void rtas_stop_self(PowerPCCPU *cpu, sPAPRMachi= neState *spapr, =20 cs->halted =3D 1; qemu_cpu_kick(cs); - /* - * While stopping a CPU, the guest calls H_CPPR which - * effectively disables interrupts on XICS level. - * However decrementer interrupts in TCG can still - * wake the CPU up so here we disable interrupts in MSR - * as well. - * As rtas_start_cpu() resets the whole MSR anyway, there is - * no need to bother with specific bits, we just clear it. - */ - env->msr =3D 0; =20 /* Disable Power-saving mode Exit Cause exceptions for the CPU. * This could deliver an interrupt on a dying CPU and crash the --=20 2.13.6