From nobody Sun Nov 24 00:00:26 2024 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2045E142E9F; Sat, 9 Nov 2024 06:33:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731134019; cv=none; b=MIiQi9U7G8nh/lJOabRKRLtapF2sIc0Nf/JO6oi1S3izUe4hOWdFWhoWAs3lgCDctWyPwoJPS10jX+7P+myd0qorWnI2edPKXFdYadiOW4f4EW69qe8PjmUPvRTsGkvgs2476XyQrVPC5mvy5uyMa8qX2FbvxND3Vy5DmS+NgCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731134019; c=relaxed/simple; bh=Wv8e6zK7nnQ58UwpLH39ENunFbinswBhSKn6KcC05hw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dvFnduntCO0u7D/GzxWZlqqEqST6udvG/GJRWHqsOvfxk0KXn9PaZ45g6P3fOWLhYZ/frMEgfhlGpM5aq1HJXeMvx/pKqx5nQanMUWb2WoVuZ+hf0rzbY4i7b2aN35Q0y4j5xyc1m6zahV84g+07EsF/v4RaLQ3p1J0BWUwTbQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=TVtC7ZBX; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="TVtC7ZBX" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A96AEmA014431; Sat, 9 Nov 2024 06:33:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=tZPX1N8QudpayLdFb xxGg89zZpw3fldD1pd09WNFmZs=; b=TVtC7ZBXHA4rriCgia1RdBWtKnm28Q4t5 /WdRsxi3GeolsXQfDQSFCSHiNrY5n1gwfrh7mw9PxEiFSFrCwKWaMWbnGNp37/+x vtLrR8y4F47206vfpvwdemR3l5J5inGsKipELA2LKEYc4P6E/1LNaPM9nS2qAjk7 kzBSkTv31vT2Nsca12Ah5c3z6K/k6eq1pLoy7SiK5ENaUEofeBiwq0+bdn3Lf4I6 Nofine1Dmk9Bkxt8UeyzoaxCc/2BKXP+6meYVFsHFYo2pBf4i+yDPoQjrPDUpjJQ CQpSrZ8I3fbPr1cHgrO/89ZoLQEHNfrrWVD42XETtHIgNcH8GtRPA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 42t27q82mj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:24 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 4A96XOZ0025283; Sat, 9 Nov 2024 06:33:24 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 42t27q82me-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:24 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 4A8J6ImE031474; Sat, 9 Nov 2024 06:33:23 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 42nydmum9j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:22 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 4A96XJv052953592 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 9 Nov 2024 06:33:19 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 37E5020043; Sat, 9 Nov 2024 06:33:19 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0F2B220040; Sat, 9 Nov 2024 06:33:16 +0000 (GMT) Received: from li-c6426e4c-27cf-11b2-a85c-95d65bc0de0e.ibm.com.com (unknown [9.124.214.93]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 9 Nov 2024 06:33:15 +0000 (GMT) From: Gautam Menghani To: mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, vaibhav@linux.ibm.com Cc: Gautam Menghani , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] Revert "KVM: PPC: Book3S HV Nested: Stop forwarding all HFUs to L1" Date: Sat, 9 Nov 2024 12:02:55 +0530 Message-ID: <20241109063301.105289-2-gautam@linux.ibm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241109063301.105289-1-gautam@linux.ibm.com> References: <20241109063301.105289-1-gautam@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: nqh2LgZWOmgYcKf7JpuwP8dgmnPaeeBT X-Proofpoint-ORIG-GUID: dmBmyCaQWZWbbmRF5G6NxR5hHZPo_Eei X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-15_01,2024-10-11_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 adultscore=0 mlxscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=616 impostorscore=0 suspectscore=0 bulkscore=0 phishscore=0 clxscore=1015 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2411090049 Content-Type: text/plain; charset="utf-8" This reverts commit 7c3ded5735141ff4d049747c9f76672a8b737c49. On PowerNV, when a nested guest tries to use a feature prohibited by HFSCR, the nested hypervisor (L1) should get a H_FAC_UNAVAILABLE trap so that L1 can emulate the feature. But with the change introduced by commit 7c3ded573514 ("KVM: PPC: Book3S HV Nested: Stop forwarding all HFUs to L1") the L1 ends up getting a H_EMUL_ASSIST because of which, the L1 ends up injecting a SIGILL when L2 (nested guest) tries to use doorbells. Reviewed-by: Nicholas Piggin Signed-off-by: Gautam Menghani --- arch/powerpc/kvm/book3s_hv.c | 31 ++----------------------------- 1 file changed, 2 insertions(+), 29 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index ad8dc4ccdaab..0a8f143e0a75 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -2060,36 +2060,9 @@ static int kvmppc_handle_nested_exit(struct kvm_vcpu= *vcpu) fallthrough; /* go to facility unavailable handler */ #endif =20 - case BOOK3S_INTERRUPT_H_FAC_UNAVAIL: { - u64 cause =3D vcpu->arch.hfscr >> 56; - - /* - * Only pass HFU interrupts to the L1 if the facility is - * permitted but disabled by the L1's HFSCR, otherwise - * the interrupt does not make sense to the L1 so turn - * it into a HEAI. - */ - if (!(vcpu->arch.hfscr_permitted & (1UL << cause)) || - (vcpu->arch.nested_hfscr & (1UL << cause))) { - ppc_inst_t pinst; - vcpu->arch.trap =3D BOOK3S_INTERRUPT_H_EMUL_ASSIST; - - /* - * If the fetch failed, return to guest and - * try executing it again. - */ - r =3D kvmppc_get_last_inst(vcpu, INST_GENERIC, &pinst); - vcpu->arch.emul_inst =3D ppc_inst_val(pinst); - if (r !=3D EMULATE_DONE) - r =3D RESUME_GUEST; - else - r =3D RESUME_HOST; - } else { - r =3D RESUME_HOST; - } - + case BOOK3S_INTERRUPT_H_FAC_UNAVAIL: + r =3D RESUME_HOST; break; - } =20 case BOOK3S_INTERRUPT_HV_RM_HARD: vcpu->arch.trap =3D 0; --=20 2.45.2 From nobody Sun Nov 24 00:00:26 2024 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 295A02BAF9; Sat, 9 Nov 2024 06:33:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731134030; cv=none; b=jvbZt8+WWVHN7PofoXrSM9iYLAAABFoc1T98ShAA6BirX/7A80YM12XTAuKHyn8buXefyTWQZ5yCLo9LnrmZP0rZjRYmbkuVekEKVYy4peYWE89VXAgvBGuKnMnaHPXPsi7yAOGOHlYhbUIpRBbh0WXJP4YMynprPoSBfaTPYYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731134030; c=relaxed/simple; bh=crBwZEzdt4P6Shs4T1zgayBcwvRsQgOF9Hco+fzKZmI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WsKSj/sJsR/kkANAkF839mJqs9YS/2yDkwaxExqrhPMIdiEAlR7nigKhqFHDpHo56QooJgvqa/oDx8bBogfRaeb1FfEEnZW8gjVeb5wHtN8bS5GD/5Ihw4b9SRwCnnzTXPI7K4ukV1gjPeGlHUz+iziZ84HEl7TupoXRG0fYH+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=W9fU1W3p; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="W9fU1W3p" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A96AF2R013229; Sat, 9 Nov 2024 06:33:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Gz8RsGttr6chp4RjJ Ef/BGCjFYj8UEZxTPfBDTERkcI=; b=W9fU1W3p2RyHS2Rf7ZoJN41nAR7bzHDjY Q9+pz0Y7XYXZ+dix9Rs7AbJueUln0JnGmvloD3pMYpZf/6uu747b34LgnnM4Q2P2 5h6CPDhjyvsn84oXAPNBIC01flYb+XMhny731oe7hS6+V9ZitZVO8VqRHQ1UIKFU IbJol4iM9hJDJmZfxeDjeogZx0E1OiK61aR9Duzs+drAtlcGT77kHP6J5Je8m/jY xwrHjUtaocbPfaZiK8K5tlIiC59w71Eh/gPjwYRFllFra2BMeU3GGW6pgUWxpT5k 3A03RGJfo6Pj1A0tONW0r1GpGQUI3QDXEepprhuxZk7Iu61poK7Hg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 42t27r01w7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:27 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 4A96XRG2023988; Sat, 9 Nov 2024 06:33:27 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 42t27r01w5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:27 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 4A94ajTN023983; Sat, 9 Nov 2024 06:33:26 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 42nxt0hw5j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:26 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 4A96XNkd53215510 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 9 Nov 2024 06:33:23 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 03A3920043; Sat, 9 Nov 2024 06:33:23 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 55C2D20040; Sat, 9 Nov 2024 06:33:20 +0000 (GMT) Received: from li-c6426e4c-27cf-11b2-a85c-95d65bc0de0e.ibm.com.com (unknown [9.124.214.93]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 9 Nov 2024 06:33:20 +0000 (GMT) From: Gautam Menghani To: mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, vaibhav@linux.ibm.com Cc: Gautam Menghani , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] KVM: PPC: Book3S HV: Stop using vc->dpdes for nested KVM guests Date: Sat, 9 Nov 2024 12:02:56 +0530 Message-ID: <20241109063301.105289-3-gautam@linux.ibm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241109063301.105289-1-gautam@linux.ibm.com> References: <20241109063301.105289-1-gautam@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: -xG-80Z3ZCcl4uSADp85sKTAuF6wSY-5 X-Proofpoint-ORIG-GUID: XKqvrCkzaXn5a7YRozyrrnSDMB9OqG7- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-15_01,2024-10-11_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 impostorscore=0 suspectscore=0 bulkscore=0 clxscore=1015 spamscore=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 adultscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2411090049 Content-Type: text/plain; charset="utf-8" commit 6398326b9ba1 ("KVM: PPC: Book3S HV P9: Stop using vc->dpdes") introduced an optimization to use only vcpu->doorbell_request for SMT emulation for Power9 and above guests, but the code for nested guests still relies on the old way of handling doorbells, due to which an L2 guest (see [1]) cannot be booted with XICS with SMT>1. The command to repro this issue is: // To be run in L1 qemu-system-ppc64 \ -drive file=3Drhel.qcow2,format=3Dqcow2 \ -m 20G \ -smp 8,cores=3D1,threads=3D8 \ -cpu host \ -nographic \ -machine pseries,ic-mode=3Dxics -accel kvm Fix the plumbing to utilize vcpu->doorbell_request instead of vcore->dpdes for nested KVM guests on P9 and above. [1] Terminology 1. L0 : PowerNV linux running with HV privileges 2. L1 : Pseries KVM guest running on top of L0 2. L2 : Nested KVM guest running on top of L1 Fixes: 6398326b9ba1 ("KVM: PPC: Book3S HV P9: Stop using vc->dpdes") Signed-off-by: Gautam Menghani --- arch/powerpc/kvm/book3s_hv.c | 9 +++++++++ arch/powerpc/kvm/book3s_hv_nested.c | 14 ++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 0a8f143e0a75..b93a93777237 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -4282,6 +4282,15 @@ static int kvmhv_vcpu_entry_p9_nested(struct kvm_vcp= u *vcpu, u64 time_limit, uns } hvregs.hdec_expiry =3D time_limit; =20 + /* + * hvregs has the doorbell status, so zero it here which + * enables us to receive doorbells when H_ENTER_NESTED is + * in progress for this vCPU + */ + + if (vcpu->arch.doorbell_request) + vcpu->arch.doorbell_request =3D 0; + /* * When setting DEC, we must always deal with irq_work_raise * via NMI vs setting DEC. The problem occurs right as we diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_= hv_nested.c index 05f5220960c6..125440a606ee 100644 --- a/arch/powerpc/kvm/book3s_hv_nested.c +++ b/arch/powerpc/kvm/book3s_hv_nested.c @@ -32,7 +32,7 @@ void kvmhv_save_hv_regs(struct kvm_vcpu *vcpu, struct hv_= guest_state *hr) struct kvmppc_vcore *vc =3D vcpu->arch.vcore; =20 hr->pcr =3D vc->pcr | PCR_MASK; - hr->dpdes =3D vc->dpdes; + hr->dpdes =3D vcpu->arch.doorbell_request; hr->hfscr =3D vcpu->arch.hfscr; hr->tb_offset =3D vc->tb_offset; hr->dawr0 =3D vcpu->arch.dawr0; @@ -105,7 +105,7 @@ static void save_hv_return_state(struct kvm_vcpu *vcpu, { struct kvmppc_vcore *vc =3D vcpu->arch.vcore; =20 - hr->dpdes =3D vc->dpdes; + hr->dpdes =3D vcpu->arch.doorbell_request; hr->purr =3D vcpu->arch.purr; hr->spurr =3D vcpu->arch.spurr; hr->ic =3D vcpu->arch.ic; @@ -143,7 +143,7 @@ static void restore_hv_regs(struct kvm_vcpu *vcpu, cons= t struct hv_guest_state * struct kvmppc_vcore *vc =3D vcpu->arch.vcore; =20 vc->pcr =3D hr->pcr | PCR_MASK; - vc->dpdes =3D hr->dpdes; + vcpu->arch.doorbell_request =3D hr->dpdes; vcpu->arch.hfscr =3D hr->hfscr; vcpu->arch.dawr0 =3D hr->dawr0; vcpu->arch.dawrx0 =3D hr->dawrx0; @@ -170,7 +170,13 @@ void kvmhv_restore_hv_return_state(struct kvm_vcpu *vc= pu, { struct kvmppc_vcore *vc =3D vcpu->arch.vcore; =20 - vc->dpdes =3D hr->dpdes; + /* + * This L2 vCPU might have received a doorbell while H_ENTER_NESTED was b= eing handled. + * Make sure we preserve the doorbell if it was either: + * a) Sent after H_ENTER_NESTED was called on this vCPU (arch.doorbell_= request would be 1) + * b) Doorbell was not handled and L2 exited for some other reason (hr-= >dpdes would be 1) + */ + vcpu->arch.doorbell_request =3D vcpu->arch.doorbell_request | hr->dpdes; vcpu->arch.hfscr =3D hr->hfscr; vcpu->arch.purr =3D hr->purr; vcpu->arch.spurr =3D hr->spurr; --=20 2.45.2 From nobody Sun Nov 24 00:00:26 2024 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58E552BAF9; Sat, 9 Nov 2024 06:33:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731134024; cv=none; b=uE9YnpIeDwaehQ5QiJ4SfGpFURbItm1Y9vqbM84ewu/Hyx0SfV10boNKqpmkZnnxRyduQ+3uug377wWg2/HBs0DBdVRwss9ibY78cesj0kYOKi7gqX+KZa4+DZOofpUbFm1AeXxAh7+gdYLCr/DBUmsh6lNbpcj1fpGpMLXP1Do= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731134024; c=relaxed/simple; bh=IO71OHVoxvynERWdA5dQ+sa75o7luafJEikXzKMaZbM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lC87cQD295pbBZXcR5xj5/T5rEiwyktjm+SCrUm4jsH5/9QvuJRR0fGubxUFIXKIv7/TW3WUfs4dQUi+4radO+R3+iDGIuZOpREA8Q2Q+YdIuxtg7XW6f1qtzcgV636GzEIR86QnxaJuw28TxcFZzQyybtFoo+iCLmkd0jvUj5E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=c7B2RHXW; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="c7B2RHXW" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A96AEmB014431; Sat, 9 Nov 2024 06:33:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=7TK3Xn/Gfmok9H1Vv gxttfEsdUnmZ5GHiiHZRWMPjJY=; b=c7B2RHXWDDgL8ndJGl/IsDffzKJL2M3HP bMo0m8KQvl6eTGUyOFxnh4BQ9r7xnE4MbCNoCeSajL1df4go5TCveMUVRoIOPIEW uqbnRJCC8BxI52n3jWa+L1pHZs4KNIwu4HofNGwVdfAxhxVaOlpJ1T8a9lCsL+sU wQOr8KknYJAfXZkGHJnldyUITQe02Uh1xMiWq8imzDQ/qdC5ACiyeSH64hwIZkAm VxM7hvGJJUt3GZPjIfH5a0SZlTIRgIdy9Qi4mH3HPs7FvU08r9LyJSBOAmMfagwq TAVIb8ce2AbZ2BPMfyAQHPLOFRF4IINu2Jn03Wb7RqSLZgFCfwPVw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 42t27q82ms-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:31 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 4A96XVRC025301; Sat, 9 Nov 2024 06:33:31 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 42t27q82mq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:31 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 4A91MOdi017188; Sat, 9 Nov 2024 06:33:30 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 42nxdsbmtm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 09 Nov 2024 06:33:30 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 4A96XQFk52953352 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 9 Nov 2024 06:33:26 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB9BD20043; Sat, 9 Nov 2024 06:33:26 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 22BE220040; Sat, 9 Nov 2024 06:33:24 +0000 (GMT) Received: from li-c6426e4c-27cf-11b2-a85c-95d65bc0de0e.ibm.com.com (unknown [9.124.214.93]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 9 Nov 2024 06:33:23 +0000 (GMT) From: Gautam Menghani To: mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, vaibhav@linux.ibm.com Cc: Gautam Menghani , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] KVM: PPC: Book3S HV: Avoid returning to nested hypervisor on pending doorbells Date: Sat, 9 Nov 2024 12:02:57 +0530 Message-ID: <20241109063301.105289-4-gautam@linux.ibm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241109063301.105289-1-gautam@linux.ibm.com> References: <20241109063301.105289-1-gautam@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: JfVhh2hWQ7V-vAgrmL58tcAVr8iCFS9t X-Proofpoint-ORIG-GUID: WsQzCeNEzCIichotoXifWUNxOUma5OcL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-15_01,2024-10-11_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 adultscore=0 mlxscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=657 impostorscore=0 suspectscore=0 bulkscore=0 phishscore=0 clxscore=1015 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2411090049 Content-Type: text/plain; charset="utf-8" Commit 6398326b9ba1 ("KVM: PPC: Book3S HV P9: Stop using vc->dpdes") dropped the use of vcore->dpdes for msgsndp / SMT emulation. Prior to that commit, the below code at L1 level (see [1] for terminology) was responsible for setting vc->dpdes for the respective L2 vCPU: if (!nested) { kvmppc_core_prepare_to_enter(vcpu); if (vcpu->arch.doorbell_request) { vc->dpdes =3D 1; smp_wmb(); vcpu->arch.doorbell_request =3D 0; } L1 then sent vc->dpdes to L0 via kvmhv_save_hv_regs(), and while servicing H_ENTER_NESTED at L0, the below condition at L0 level made sure to abort and go back to L1 if vcpu->arch.doorbell_request =3D 1 so that L1 sets vc->dpdes as per above if condition: } else if (vcpu->arch.pending_exceptions || vcpu->arch.doorbell_request || xive_interrupt_pending(vcpu)) { vcpu->arch.ret =3D RESUME_HOST; goto out; } This worked fine since vcpu->arch.doorbell_request was used more like a flag and vc->dpdes was used to pass around the doorbell state. But after Commit 6398326b9ba1 ("KVM: PPC: Book3S HV P9: Stop using vc->dpdes"), vcpu->arch.doorbell_request is the only variable used to pass around doorbell state. With the plumbing for handling doorbells for nested guests updated to use vcpu->arch.doorbell_request over vc->dpdes, the above "else if" stops doorbells from working correctly as L0 aborts execution of L2 and instead goes back to L1. Remove vcpu->arch.doorbell_request from the above "else if" condition as it is no longer needed for L0 to correctly handle the doorbell status while running L2. [1] Terminology 1. L0 : PowerNV linux running with HV privileges 2. L1 : Pseries KVM guest running on top of L0 2. L2 : Nested KVM guest running on top of L1 Fixes: 6398326b9ba1 ("KVM: PPC: Book3S HV P9: Stop using vc->dpdes") Signed-off-by: Gautam Menghani --- arch/powerpc/kvm/book3s_hv.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index b93a93777237..8385d4db1763 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -4894,7 +4894,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 = time_limit, lpcr &=3D ~LPCR_MER; } } else if (vcpu->arch.pending_exceptions || - vcpu->arch.doorbell_request || xive_interrupt_pending(vcpu)) { vcpu->arch.ret =3D RESUME_HOST; goto out; --=20 2.45.2