From nobody Thu Dec 18 05:18:21 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45608156C7B; Mon, 6 May 2024 16:18:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715012311; cv=none; b=kloHbRqb/p4etdkv810JUTZJezeNTyxNirB6zSMmdt9PwcBt3PpyFLcPQCWhyneqQUCKaelVT1JqYxQ9y/klGYsqvnfU44HTLSJw/g1yLE/T8qT78bqIbyAeWJPRwiRg4BFBvXzzWXEsyJn1v60fE49cdqm+x9bJF8hxMDj4MMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715012311; c=relaxed/simple; bh=nYOXwfkD4Aagx4HIXM0yufDLbQFaKlb+z99rD9eDC1I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V0xdnw9cRK84SLUVM4utHti21e/lSwWbLqNL3EgJxgrZ+ggJW690EUJ0Qpgfoz6NmMeXyBXYZli0udteeoMhvW62VXVxChilV84BmQX2+TpkcCfqizOMa5Iv1SOKWCPKPk3oUBWMbGMSKb+9aG0zLYOqOxl3flQUQRSYUASvaOQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=DrIDc9wr; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="DrIDc9wr" Received: from pps.filterd (m0353724.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 446GIH6C023385; Mon, 6 May 2024 16:18:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=YI+s/9jNr6WRySWWHUvNpGEk6ucoohYRHUK1CkL4ZOQ=; b=DrIDc9wrs64KSTS+uTmGrgrgoiJrK3Er+nI/XHpyaMmoksdxb4gTTn2YCkyQj3FD5hrh 7SPIxUpOPC/7IEY1FV5TpCTroobr6L7L0e+UuYReWiJTuUYJvDlrnkZ+48MDUc4e+581 /QVXR7JFBQVlSfbHEdHOR/bx1cZi5Y6Dd1RelfPLUcIdo/PxZJ40CxDSWu20KlPqHFrB xmvn9nCM+CRKIe4rkI0ZMfFLiOm7aKWiMfhx3hQMLbJ1zv4TfCDpCwH+08jX+JLkFa69 8EXSmZAoLDuQ65L9phdTYTzn0bEUJ/ppN8PYbvHDIMsK2lg4DP09lOGAZ/vtYvwYxs2d ig== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3xy2kt800h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 May 2024 16:18:21 +0000 Received: from m0353724.ppops.net (m0353724.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 446GIKBd023408; Mon, 6 May 2024 16:18:20 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3xy2kt800f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 May 2024 16:18:20 +0000 Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 446F0Gcx030859; Mon, 6 May 2024 16:18:20 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 3xwybts4ds-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 May 2024 16:18:20 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 446GIEPQ18547032 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 6 May 2024 16:18:16 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B2B1E20040; Mon, 6 May 2024 16:18:14 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3639220043; Mon, 6 May 2024 16:18:10 +0000 (GMT) Received: from li-c6426e4c-27cf-11b2-a85c-95d65bc0de0e.ibm.com.com (unknown [9.43.105.31]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 6 May 2024 16:18:09 +0000 (GMT) From: Gautam Menghani To: mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen.n.rao@linux.ibm.com Cc: Gautam Menghani , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] arch/powerpc/kvm: Reduce lock contention by moving spinlock from ics to irq_state Date: Mon, 6 May 2024 21:47:31 +0530 Message-ID: <20240506161735.83358-4-gautam@linux.ibm.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240506161735.83358-1-gautam@linux.ibm.com> References: <20240506161735.83358-1-gautam@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 4gipCWtZDXNoC-sVQctrGEDKavqn7ILJ X-Proofpoint-GUID: yRPT_3qffr7ZSW7TOzde6sRtrfcS9e1U X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-06_11,2024-05-06_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0 mlxscore=0 adultscore=0 malwarescore=0 priorityscore=1501 phishscore=0 spamscore=0 lowpriorityscore=0 bulkscore=0 mlxlogscore=559 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2405060114 Content-Type: text/plain; charset="utf-8" Take a spinlock on state of an IRQ instead of an entire ICS. This improves scalability by reducing contention. Signed-off-by: Gautam Menghani --- arch/powerpc/kvm/book3s_hv_rm_xics.c | 8 ++--- arch/powerpc/kvm/book3s_xics.c | 44 ++++++++++++---------------- arch/powerpc/kvm/book3s_xics.h | 2 +- 3 files changed, 23 insertions(+), 31 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_rm_xics.c b/arch/powerpc/kvm/book3s= _hv_rm_xics.c index e42984878503..178bc869b519 100644 --- a/arch/powerpc/kvm/book3s_hv_rm_xics.c +++ b/arch/powerpc/kvm/book3s_hv_rm_xics.c @@ -308,7 +308,7 @@ static void icp_rm_deliver_irq(struct kvmppc_xics *xics= , struct kvmppc_icp *icp, state =3D &ics->irq_state[src]; =20 /* Get a lock on the ICS */ - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); =20 /* Get our server */ if (!icp || state->server !=3D icp->server_num) { @@ -368,7 +368,7 @@ static void icp_rm_deliver_irq(struct kvmppc_xics *xics= , struct kvmppc_icp *icp, * Delivery was successful, did we reject somebody else ? */ if (reject && reject !=3D XICS_IPI) { - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); icp->n_reject++; new_irq =3D reject; check_resend =3D 0; @@ -397,13 +397,13 @@ static void icp_rm_deliver_irq(struct kvmppc_xics *xi= cs, struct kvmppc_icp *icp, smp_mb(); if (!icp->state.need_resend) { state->resend =3D 0; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); check_resend =3D 0; goto again; } } out: - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); } =20 static void icp_rm_down_cppr(struct kvmppc_xics *xics, struct kvmppc_icp *= icp, diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c index 1dc2f77571e7..466c92cf49fb 100644 --- a/arch/powerpc/kvm/book3s_xics.c +++ b/arch/powerpc/kvm/book3s_xics.c @@ -36,21 +36,13 @@ * LOCKING * =3D=3D=3D=3D=3D=3D=3D * - * Each ICS has a spin lock protecting the information about the IRQ - * sources and avoiding simultaneous deliveries of the same interrupt. + * Each IRQ has a spin lock protecting its state sources and avoiding + * simultaneous deliveries of the same interrupt. * * ICP operations are done via a single compare & swap transaction * (most ICP state fits in the union kvmppc_icp_state) */ =20 -/* - * TODO - * =3D=3D=3D=3D - * - * - Make ICS lockless as well, or at least a per-interrupt lock or hashed - * locks array to improve scalability - */ - /* -- ICS routines -- */ =20 static void icp_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *i= cp, @@ -142,7 +134,7 @@ static bool write_xive(struct kvmppc_xics *xics, struct= kvmppc_ics *ics, unsigned long flags; =20 local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); =20 state->server =3D server; state->priority =3D priority; @@ -154,7 +146,7 @@ static bool write_xive(struct kvmppc_xics *xics, struct= kvmppc_ics *ics, deliver =3D true; } =20 - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); =20 return deliver; @@ -207,10 +199,10 @@ int kvmppc_xics_get_xive(struct kvm *kvm, u32 irq, u3= 2 *server, u32 *priority) state =3D &ics->irq_state[src]; =20 local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); *server =3D state->server; *priority =3D state->priority; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); =20 return 0; @@ -406,7 +398,7 @@ static void icp_deliver_irq(struct kvmppc_xics *xics, s= truct kvmppc_icp *icp, =20 /* Get a lock on the ICS */ local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); =20 /* Get our server */ if (!icp || state->server !=3D icp->server_num) { @@ -467,7 +459,7 @@ static void icp_deliver_irq(struct kvmppc_xics *xics, s= truct kvmppc_icp *icp, * Delivery was successful, did we reject somebody else ? */ if (reject && reject !=3D XICS_IPI) { - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); new_irq =3D reject; check_resend =3D false; @@ -497,14 +489,14 @@ static void icp_deliver_irq(struct kvmppc_xics *xics,= struct kvmppc_icp *icp, smp_mb(); if (!icp->state.need_resend) { state->resend =3D 0; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); check_resend =3D false; goto again; } } out: - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); } =20 @@ -992,20 +984,20 @@ static int xics_debug_show(struct seq_file *m, void *= private) seq_printf(m, "=3D=3D=3D=3D=3D=3D=3D=3D=3D\nICS state for ICS 0x%x\n=3D= =3D=3D=3D=3D=3D=3D=3D=3D\n", icsid); =20 - local_irq_save(flags); - arch_spin_lock(&ics->lock); =20 for (i =3D 0; i < KVMPPC_XICS_IRQ_PER_ICS; i++) { struct ics_irq_state *irq =3D &ics->irq_state[i]; + local_irq_save(flags); + arch_spin_lock(&irq->lock); =20 seq_printf(m, "irq 0x%06x: server %#x prio %#x save prio %#x pq_state %= d resend %d masked pending %d\n", irq->number, irq->server, irq->priority, irq->saved_priority, irq->pq_state, irq->resend, irq->masked_pending); =20 + arch_spin_unlock(&irq->lock); + local_irq_restore(flags); } - arch_spin_unlock(&ics->lock); - local_irq_restore(flags); } return 0; } @@ -1189,7 +1181,7 @@ static int xics_get_source(struct kvmppc_xics *xics, = long irq, u64 addr) =20 irqp =3D &ics->irq_state[idx]; local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&irqp->lock); ret =3D -ENOENT; if (irqp->exists) { val =3D irqp->server; @@ -1214,7 +1206,7 @@ static int xics_get_source(struct kvmppc_xics *xics, = long irq, u64 addr) =20 ret =3D 0; } - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&irqp->lock); local_irq_restore(flags); =20 if (!ret && put_user(val, ubufp)) @@ -1254,7 +1246,7 @@ static int xics_set_source(struct kvmppc_xics *xics, = long irq, u64 addr) return -EINVAL; =20 local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&irqp->lock); irqp->server =3D server; irqp->saved_priority =3D prio; if (val & KVM_XICS_MASKED) @@ -1272,7 +1264,7 @@ static int xics_set_source(struct kvmppc_xics *xics, = long irq, u64 addr) if (val & KVM_XICS_QUEUED) irqp->pq_state |=3D PQ_QUEUED; irqp->exists =3D 1; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&irqp->lock); local_irq_restore(flags); =20 if (val & KVM_XICS_PENDING) diff --git a/arch/powerpc/kvm/book3s_xics.h b/arch/powerpc/kvm/book3s_xics.h index feeb0897d555..1ee62b7a8fdf 100644 --- a/arch/powerpc/kvm/book3s_xics.h +++ b/arch/powerpc/kvm/book3s_xics.h @@ -45,6 +45,7 @@ struct ics_irq_state { u8 exists; int intr_cpu; u32 host_irq; + arch_spinlock_t lock; }; =20 /* Atomic ICP state, updated with a single compare & swap */ @@ -95,7 +96,6 @@ struct kvmppc_icp { }; =20 struct kvmppc_ics { - arch_spinlock_t lock; u16 icsid; struct ics_irq_state irq_state[KVMPPC_XICS_IRQ_PER_ICS]; DECLARE_BITMAP(resend_map, KVMPPC_XICS_IRQ_PER_ICS); --=20 2.44.0