From nobody Sat Nov 8 07:38:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1549996021374218.22489546473696; Tue, 12 Feb 2019 10:27:01 -0800 (PST) Received: from localhost ([127.0.0.1]:43962 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gtcll-0002bj-PZ for importer@patchew.org; Tue, 12 Feb 2019 13:26:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38444) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gtcjg-00014p-IL for qemu-devel@nongnu.org; Tue, 12 Feb 2019 13:24:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gtcja-0003VL-Rb for qemu-devel@nongnu.org; Tue, 12 Feb 2019 13:24:46 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:49642 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gtcja-0003BA-7S for qemu-devel@nongnu.org; Tue, 12 Feb 2019 13:24:42 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x1CIJlqP019040 for ; Tue, 12 Feb 2019 13:24:23 -0500 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0b-001b2d01.pphosted.com with ESMTP id 2qm1rhcsyn-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 12 Feb 2019 13:24:23 -0500 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 12 Feb 2019 18:24:20 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 12 Feb 2019 18:24:16 -0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x1CIOFig55115866 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 12 Feb 2019 18:24:15 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0418511C058; Tue, 12 Feb 2019 18:24:15 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CDF6411C04A; Tue, 12 Feb 2019 18:24:14 +0000 (GMT) Received: from smtp.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.1]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 12 Feb 2019 18:24:14 +0000 (GMT) Received: from bahia.lan (sig-9-145-185-96.de.ibm.com [9.145.185.96]) by smtp.lab.toulouse-stg.fr.ibm.com (Postfix) with ESMTP id 8FC02220182; Tue, 12 Feb 2019 19:24:13 +0100 (CET) From: Greg Kurz To: David Gibson Date: Tue, 12 Feb 2019 19:24:13 +0100 In-Reply-To: <154999583316.690774.15072605479770041782.stgit@bahia.lan> References: <154999583316.690774.15072605479770041782.stgit@bahia.lan> User-Agent: StGit/unknown-version MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 x-cbid: 19021218-0016-0000-0000-00000255A3BC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19021218-0017-0000-0000-000032AFC73B Message-Id: <154999585315.690774.7586633403635165044.stgit@bahia.lan> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-02-12_09:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=8 phishscore=0 bulkscore=0 spamscore=0 clxscore=1034 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=787 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902120129 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v4 03/15] spapr_irq: Set LSIs at interrupt controller init X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Cornelia Huck , Gerd Hoffmann , Michael Roth , "Michael S. Tsirkin" , Alexey Kardashevskiy , David Hildenbrand , qemu-devel@nongnu.org, Greg Kurz , qemu-s390x@nongnu.org, Dmitry Fleytman , qemu-ppc@nongnu.org, =?utf-8?q?C=C3=A9dric?= Le Goater , Marcel Apfelbaum , Paolo Bonzini , Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" The pseries machine only uses LSIs to support legacy PCI devices. Every PHB claims 4 LSIs at realize time. When using in-kernel XICS (or upcoming in-kernel XIVE), QEMU synchronizes the state of all irqs, including these LSIs, later on at machine reset. In order to support PHB hotplug, we need a way to tell KVM about the LSIs that doesn't require a machine reset. Since recent machine types allocate all these LSIs in a fixed range for the machine lifetime, identify them when initializing the interrupt controller, long before they get passed to KVM. In order to do that, first disintricate interrupt typing and allocation. Since the vast majority of interrupts are MSIs, make that the default and have only the LSI users to explicitely set the type. It is rather straight forward for XIVE. XICS needs some extra care though: allocation state and type are mixed up in the same bits of the flags field within the interrupt state. Setting the LSI bit there at init time would mean the interrupt is de facto allocated, even if no device asked for it. Introduce a bitmap to track LSIs at the ICS level. In order to keep the patch minimal, the bitmap is only used when writing the source state to KVM and when the interrupt is claimed, so that the code that checks the interrupt type through the flags stays untouched. With older pseries machine using the XICS legacy IRQ allocation scheme, all interrupt numbers come from a common pool and there's no such thing as a fixed range for LSIs. Introduce an helper so that these older machine types can continue to set the type when allocating the LSI. Signed-off-by: Greg Kurz Reviewed-by: C=C3=A9dric Le Goater --- hw/intc/spapr_xive.c | 7 +------ hw/intc/xics.c | 10 ++++++++-- hw/intc/xics_kvm.c | 2 +- hw/ppc/pnv_psi.c | 3 ++- hw/ppc/spapr_events.c | 4 ++-- hw/ppc/spapr_irq.c | 42 ++++++++++++++++++++++++++++++++-------= --- hw/ppc/spapr_pci.c | 6 ++++-- hw/ppc/spapr_vio.c | 2 +- include/hw/ppc/spapr_irq.h | 5 +++-- include/hw/ppc/spapr_xive.h | 2 +- include/hw/ppc/xics.h | 4 +++- 11 files changed, 58 insertions(+), 29 deletions(-) diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 290a290e43a5..815263ca72ab 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -480,18 +480,13 @@ static void spapr_xive_register_types(void) =20 type_init(spapr_xive_register_types) =20 -bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn, bool lsi) +bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn) { - XiveSource *xsrc =3D &xive->source; - if (lisn >=3D xive->nr_irqs) { return false; } =20 xive->eat[lisn].w |=3D cpu_to_be64(EAS_VALID); - if (lsi) { - xive_source_irq_set_lsi(xsrc, lisn); - } return true; } =20 diff --git a/hw/intc/xics.c b/hw/intc/xics.c index 7cac138067e2..26e8940d7329 100644 --- a/hw/intc/xics.c +++ b/hw/intc/xics.c @@ -636,6 +636,7 @@ static void ics_base_realize(DeviceState *dev, Error **= errp) return; } ics->irqs =3D g_malloc0(ics->nr_irqs * sizeof(ICSIRQState)); + ics->lsi_map =3D bitmap_new(ics->nr_irqs); } =20 static int ics_base_dispatch_pre_save(void *opaque) @@ -733,12 +734,17 @@ ICPState *xics_icp_get(XICSFabric *xi, int server) return xic->icp_get(xi, server); } =20 -void ics_set_irq_type(ICSState *ics, int srcno, bool lsi) +void ics_set_lsi(ICSState *ics, int srcno) +{ + set_bit(srcno, ics->lsi_map); +} + +void ics_claim_irq(ICSState *ics, int srcno) { assert(!(ics->irqs[srcno].flags & XICS_FLAGS_IRQ_MASK)); =20 ics->irqs[srcno].flags |=3D - lsi ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IRQ_MSI; + test_bit(srcno, ics->lsi_map) ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IR= Q_MSI; } =20 static void xics_register_types(void) diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c index dff13300504c..e63979abc7fc 100644 --- a/hw/intc/xics_kvm.c +++ b/hw/intc/xics_kvm.c @@ -271,7 +271,7 @@ static int ics_set_kvm_state(ICSState *ics, int version= _id) state |=3D KVM_XICS_MASKED; } =20 - if (ics->irqs[i].flags & XICS_FLAGS_IRQ_LSI) { + if (test_bit(i, ics->lsi_map)) { state |=3D KVM_XICS_LEVEL_SENSITIVE; if (irq->status & XICS_STATUS_ASSERTED) { state |=3D KVM_XICS_PENDING; diff --git a/hw/ppc/pnv_psi.c b/hw/ppc/pnv_psi.c index 8ced09506321..e6089e1035c0 100644 --- a/hw/ppc/pnv_psi.c +++ b/hw/ppc/pnv_psi.c @@ -487,7 +487,8 @@ static void pnv_psi_realize(DeviceState *dev, Error **e= rrp) } =20 for (i =3D 0; i < ics->nr_irqs; i++) { - ics_set_irq_type(ics, i, true); + ics_set_lsi(ics, i); + ics_claim_irq(ics, i); } =20 psi->qirqs =3D qemu_allocate_irqs(ics_simple_set_irq, ics, ics->nr_irq= s); diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c index b9c7ecb9e987..559026d0981c 100644 --- a/hw/ppc/spapr_events.c +++ b/hw/ppc/spapr_events.c @@ -713,7 +713,7 @@ void spapr_events_init(sPAPRMachineState *spapr) epow_irq =3D spapr_irq_findone(spapr, &error_fatal); } =20 - spapr_irq_claim(spapr, epow_irq, false, &error_fatal); + spapr_irq_claim(spapr, epow_irq, &error_fatal); =20 QTAILQ_INIT(&spapr->pending_events); =20 @@ -737,7 +737,7 @@ void spapr_events_init(sPAPRMachineState *spapr) hp_irq =3D spapr_irq_findone(spapr, &error_fatal); } =20 - spapr_irq_claim(spapr, hp_irq, false, &error_fatal); + spapr_irq_claim(spapr, hp_irq, &error_fatal); =20 spapr_event_sources_register(spapr->event_sources, EVENT_CLASS_HOT= _PLUG, hp_irq); diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 8217e0215411..3fc34d7c8a43 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -16,10 +16,13 @@ #include "hw/ppc/spapr_xive.h" #include "hw/ppc/xics.h" #include "hw/ppc/xics_spapr.h" +#include "hw/pci-host/spapr.h" #include "sysemu/kvm.h" =20 #include "trace.h" =20 +#define SPAPR_IRQ_PCI_LSI_NR (SPAPR_MAX_PHBS * PCI_NUM_PINS) + void spapr_irq_msi_init(sPAPRMachineState *spapr, uint32_t nr_msis) { spapr->irq_map_nr =3D nr_msis; @@ -102,6 +105,7 @@ static void spapr_irq_init_xics(sPAPRMachineState *spap= r, Error **errp) MachineState *machine =3D MACHINE(spapr); int nr_irqs =3D spapr->irq->nr_irqs; Error *local_err =3D NULL; + int i; =20 if (kvm_enabled()) { if (machine_kernel_irqchip_allowed(machine) && @@ -128,6 +132,14 @@ static void spapr_irq_init_xics(sPAPRMachineState *spa= pr, Error **errp) &local_err); } =20 + /* Identify the PCI LSIs */ + if (!SPAPR_MACHINE_GET_CLASS(spapr)->legacy_irq_allocation) { + for (i =3D 0; i < SPAPR_IRQ_PCI_LSI_NR; ++i) { + ics_set_lsi(spapr->ics, + i + SPAPR_IRQ_PCI_LSI - spapr->irq->xics_offset); + } + } + error: error_propagate(errp, local_err); } @@ -135,7 +147,7 @@ error: #define ICS_IRQ_FREE(ics, srcno) \ (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_MASK))) =20 -static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool ls= i, +static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, Error **errp) { ICSState *ics =3D spapr->ics; @@ -152,7 +164,7 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spap= r, int irq, bool lsi, return -1; } =20 - ics_set_irq_type(ics, irq - ics->offset, lsi); + ics_claim_irq(ics, irq - ics->offset); return 0; } =20 @@ -296,16 +308,21 @@ static void spapr_irq_init_xive(sPAPRMachineState *sp= apr, Error **errp) =20 /* Enable the CPU IPIs */ for (i =3D 0; i < nr_servers; ++i) { - spapr_xive_irq_claim(spapr->xive, SPAPR_IRQ_IPI + i, false); + spapr_xive_irq_claim(spapr->xive, SPAPR_IRQ_IPI + i); + } + + /* Identify the PCI LSIs */ + for (i =3D 0; i < SPAPR_IRQ_PCI_LSI_NR; ++i) { + xive_source_irq_set_lsi(&spapr->xive->source, SPAPR_IRQ_PCI_LSI + = i); } =20 spapr_xive_hcall_init(spapr); } =20 -static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, bool ls= i, +static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, Error **errp) { - if (!spapr_xive_irq_claim(spapr->xive, irq, lsi)) { + if (!spapr_xive_irq_claim(spapr->xive, irq)) { error_setg(errp, "IRQ %d is invalid", irq); return -1; } @@ -465,19 +482,19 @@ static void spapr_irq_init_dual(sPAPRMachineState *sp= apr, Error **errp) } } =20 -static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, bool ls= i, +static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, Error **errp) { Error *local_err =3D NULL; int ret; =20 - ret =3D spapr_irq_xics.claim(spapr, irq, lsi, &local_err); + ret =3D spapr_irq_xics.claim(spapr, irq, &local_err); if (local_err) { error_propagate(errp, local_err); return ret; } =20 - ret =3D spapr_irq_xive.claim(spapr, irq, lsi, &local_err); + ret =3D spapr_irq_xive.claim(spapr, irq, &local_err); if (local_err) { error_propagate(errp, local_err); return ret; @@ -630,9 +647,9 @@ void spapr_irq_init(sPAPRMachineState *spapr, Error **e= rrp) spapr->irq->nr_irqs); } =20 -int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **e= rrp) +int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp) { - return spapr->irq->claim(spapr, irq, lsi, errp); + return spapr->irq->claim(spapr, irq, errp); } =20 void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num) @@ -712,6 +729,11 @@ int spapr_irq_find(sPAPRMachineState *spapr, int num, = bool align, Error **errp) return first + ics->offset; } =20 +void spapr_irq_set_lsi_legacy(sPAPRMachineState *spapr, int irq) +{ + ics_set_lsi(spapr->ics, irq - spapr->irq->xics_offset); +} + #define SPAPR_IRQ_XICS_LEGACY_NR_IRQS 0x400 =20 sPAPRIrq spapr_irq_xics_legacy =3D { diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index c3fb0ac884b0..d68595531d5a 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -391,7 +391,7 @@ static void rtas_ibm_change_msi(PowerPCCPU *cpu, sPAPRM= achineState *spapr, } =20 for (i =3D 0; i < req_num; i++) { - spapr_irq_claim(spapr, irq + i, false, &err); + spapr_irq_claim(spapr, irq + i, &err); if (err) { if (i) { spapr_irq_free(spapr, irq, i); @@ -1742,9 +1742,11 @@ static void spapr_phb_realize(DeviceState *dev, Erro= r **errp) "can't allocate LSIs: "); return; } + + spapr_irq_set_lsi_legacy(spapr, irq); } =20 - spapr_irq_claim(spapr, irq, true, &local_err); + spapr_irq_claim(spapr, irq, &local_err); if (local_err) { error_propagate_prepend(errp, local_err, "can't allocate LSIs:= "); return; diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c index 2b7e7ecac57f..b1beefc24be5 100644 --- a/hw/ppc/spapr_vio.c +++ b/hw/ppc/spapr_vio.c @@ -512,7 +512,7 @@ static void spapr_vio_busdev_realize(DeviceState *qdev,= Error **errp) } } =20 - spapr_irq_claim(spapr, dev->irq, false, &local_err); + spapr_irq_claim(spapr, dev->irq, &local_err); if (local_err) { error_propagate(errp, local_err); return; diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h index 5e30858dc22a..0e6c65d55430 100644 --- a/include/hw/ppc/spapr_irq.h +++ b/include/hw/ppc/spapr_irq.h @@ -37,7 +37,7 @@ typedef struct sPAPRIrq { uint32_t xics_offset; =20 void (*init)(sPAPRMachineState *spapr, Error **errp); - int (*claim)(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp= ); + int (*claim)(sPAPRMachineState *spapr, int irq, Error **errp); void (*free)(sPAPRMachineState *spapr, int irq, int num); qemu_irq (*qirq)(sPAPRMachineState *spapr, int irq); void (*print_info)(sPAPRMachineState *spapr, Monitor *mon); @@ -56,7 +56,7 @@ extern sPAPRIrq spapr_irq_xive; extern sPAPRIrq spapr_irq_dual; =20 void spapr_irq_init(sPAPRMachineState *spapr, Error **errp); -int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **e= rrp); +int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp); void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num); qemu_irq spapr_qirq(sPAPRMachineState *spapr, int irq); int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id); @@ -67,5 +67,6 @@ void spapr_irq_reset(sPAPRMachineState *spapr, Error **er= rp); */ int spapr_irq_find(sPAPRMachineState *spapr, int num, bool align, Error **= errp); #define spapr_irq_findone(spapr, errp) spapr_irq_find(spapr, 1, false, err= p) +void spapr_irq_set_lsi_legacy(sPAPRMachineState *spapr, int irq); =20 #endif diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 9bec9192e4a0..885ca169cb29 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -37,7 +37,7 @@ typedef struct sPAPRXive { MemoryRegion tm_mmio; } sPAPRXive; =20 -bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn, bool lsi); +bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn); bool spapr_xive_irq_free(sPAPRXive *xive, uint32_t lisn); void spapr_xive_pic_print_info(sPAPRXive *xive, Monitor *mon); =20 diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h index fad786e8b22d..18b083fe2aec 100644 --- a/include/hw/ppc/xics.h +++ b/include/hw/ppc/xics.h @@ -133,6 +133,7 @@ struct ICSState { uint32_t offset; ICSIRQState *irqs; XICSFabric *xics; + unsigned long *lsi_map; }; =20 #define ICS_PROP_XICS "xics" @@ -193,7 +194,8 @@ void ics_simple_write_xive(ICSState *ics, int nr, int s= erver, void ics_simple_set_irq(void *opaque, int srcno, int val); void ics_kvm_set_irq(void *opaque, int srcno, int val); =20 -void ics_set_irq_type(ICSState *ics, int srcno, bool lsi); +void ics_set_lsi(ICSState *ics, int srcno); +void ics_claim_irq(ICSState *ics, int srcno); void icp_pic_print_info(ICPState *icp, Monitor *mon); void ics_pic_print_info(ICSState *ics, Monitor *mon); =20