From nobody Sat Feb 7 11:30:51 2026 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BFF839FC3 for ; Wed, 21 Feb 2024 05:43:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494204; cv=none; b=qs+Q49YEpFaAHHuUWVXJi12MrFxeSj+6xj9M5VMW5Ajw8Q0mjdCZBDB3upFpcB5t2HqALTRB9z3kbWDFJ0S4c5tvDwSchgu2GZY/4UZuDHRMq+Y1aGtZfaall0lfmjLgmd+KnAedkP3JlskUQU926mVU6fzqdI0VjuGjKisWGO4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494204; c=relaxed/simple; bh=ZmOXn9U2kRyvale593aVJAMMtGmI9I8PRxYMMaYh3Hg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ifMRX4ODAoZtxcUU2iWz7LRJrQVYPwtOaYWbTFizoU0/f//1SE0FYJRYwJWIrWe3LyX7hk6kTytRZxqDt1NJi9k49U6xCOZWYDxE698Nt+UE/C28rX1tkJVHXvSYm1+piM9/2Na+A2WvPp0kgsz4r2qahIGjIcvXrLGfKeJKDy8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=fYgh58Q8; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="fYgh58Q8" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494200; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=et9uSczKRmO7T/9bBY6kYRtqxJPq32jgEhjumGlg3NA=; b=fYgh58Q8CQGX1i5n2girseOonhB0gU5edl8fQi1hJyftVFoeOzvXH8l1LV7JLbwVC9SaTm uhdw/yl5tVkIRDRK3aavw40TR/wba9NxQ7Az0YFPNvpMk9WWupqsLLCOCRFcdozxDrNSHW ajsCY9K/5csZvg4mikj3wx/4XhXfPK8= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 01/10] KVM: arm64: vgic: Store LPIs in an xarray Date: Wed, 21 Feb 2024 05:42:44 +0000 Message-ID: <20240221054253.3848076-2-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Using a linked-list for LPIs is less than ideal as it of course requires iterative searches to find a particular entry. An xarray is a better data structure for this use case, as it provides faster searches and can still handle a potentially sparse range of INTID allocations. Start by storing LPIs in an xarray, punting usage of the xarray to a subsequent change. The observant among you will notice that we added yet another lock to the chain of locking order rules; document the ordering of the xa_lock. Don't worry, we'll get rid of the lpi_list_lock one day... Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-init.c | 3 +++ arch/arm64/kvm/vgic/vgic-its.c | 16 ++++++++++++++++ arch/arm64/kvm/vgic/vgic.c | 4 +++- include/kvm/arm_vgic.h | 2 ++ 4 files changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-ini= t.c index e949e1d0fd9f..411719053107 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -56,6 +56,7 @@ void kvm_vgic_early_init(struct kvm *kvm) INIT_LIST_HEAD(&dist->lpi_list_head); INIT_LIST_HEAD(&dist->lpi_translation_cache); raw_spin_lock_init(&dist->lpi_list_lock); + xa_init_flags(&dist->lpi_xa, XA_FLAGS_LOCK_IRQ); } =20 /* CREATION */ @@ -366,6 +367,8 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm) =20 if (vgic_supports_direct_msis(kvm)) vgic_v4_teardown(kvm); + + xa_destroy(&dist->lpi_xa); } =20 static void __kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index e2764d0ffa9f..fb2d3c356984 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -52,6 +52,12 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u3= 2 intid, if (!irq) return ERR_PTR(-ENOMEM); =20 + ret =3D xa_reserve_irq(&dist->lpi_xa, intid, GFP_KERNEL_ACCOUNT); + if (ret) { + kfree(irq); + return ERR_PTR(ret); + } + INIT_LIST_HEAD(&irq->lpi_list); INIT_LIST_HEAD(&irq->ap_list); raw_spin_lock_init(&irq->irq_lock); @@ -86,12 +92,22 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u= 32 intid, goto out_unlock; } =20 + ret =3D xa_err(xa_store(&dist->lpi_xa, intid, irq, 0)); + if (ret) { + xa_release(&dist->lpi_xa, intid); + kfree(irq); + goto out_unlock; + } + list_add_tail(&irq->lpi_list, &dist->lpi_list_head); dist->lpi_list_count++; =20 out_unlock: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); =20 + if (ret) + return ERR_PTR(ret); + /* * We "cache" the configuration table entries in our struct vgic_irq's. * However we only have those structs for mapped IRQs, so we read in diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index db2a95762b1b..92e4d2fea365 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -30,7 +30,8 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = =3D { * its->its_lock (mutex) * vgic_cpu->ap_list_lock must be taken with IRQs disabled * kvm->lpi_list_lock must be taken with IRQs disabled - * vgic_irq->irq_lock must be taken with IRQs disabled + * vgic_dist->lpi_xa.xa_lock must be taken with IRQs disabled + * vgic_irq->irq_lock must be taken with IRQs disabled * * As the ap_list_lock might be taken from the timer interrupt handler, * we have to disable IRQs before taking this lock and everything lower @@ -131,6 +132,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic= _irq *irq) return; =20 list_del(&irq->lpi_list); + xa_erase(&dist->lpi_xa, irq->intid); dist->lpi_list_count--; =20 kfree(irq); diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 8cc38e836f54..795b35656b54 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -275,6 +276,7 @@ struct vgic_dist { =20 /* Protects the lpi_list and the count value below. */ raw_spinlock_t lpi_list_lock; + struct xarray lpi_xa; struct list_head lpi_list_head; int lpi_list_count; =20 --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 309B13A8E5 for ; Wed, 21 Feb 2024 05:43:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494205; cv=none; b=IeOEdz1BPKJNez76JfVC+kfLZSRE0tsnhM6fdepxHP2EG5k7T9XlfHMKr13iorHrLTzNvK8gWTPf0jqJBX/YCqjuPFEulUgnZNoVz2K96vVuzBltiAWb1WH5GxUu0CyBxh6yDQSAwVu53eD0/q9CTdxsNpoZ8cbhJ4GHfPedfjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494205; c=relaxed/simple; bh=PdEyg5rDXT1t31oqeD150AOaybowNPAh6qErSbtdqXs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u2eIgXgj0pew8GYGXYsi+0DSSjpSK83x/ZFm9tH3GAnt0TSbqtTd0C3YVKRBXoRWVk3cMpV6TcgFIba+mj+AbI9z5CiWouuvvQAUi/COdU+8bis85RA1SIGYF2bHM9ahvdBcerkI1+dPS2ZYZ874bqHBp2i/quz7nvKDso8Vj0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=mbz+iDp+; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="mbz+iDp+" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mj321azO2g3V6z75I7i/wkEQoxgT4KRGdjlI33lsjxQ=; b=mbz+iDp+sw32XYqQJv3DxB3aud7oUieMyg8J1rhMwXBs5ROt8iTNNC8UxSHnazQzP+3+B0 twdyKRXR7PasSyIRSD+4uYe2Gx9jg4AdcRBXUONHgEXY4DGuz5EnLw/Za+daWMM7sKgN9V KaOcci7KlpsTEBdUrUkbceIT/gbzOPE= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 02/10] KVM: arm64: vgic: Use xarray to find LPI in vgic_get_lpi() Date: Wed, 21 Feb 2024 05:42:45 +0000 Message-ID: <20240221054253.3848076-3-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Iterating over the LPI linked-list is less than ideal when the desired index is already known. Use the INTID to index the LPI xarray instead. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 92e4d2fea365..7d17dbc8f5dc 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -55,8 +55,9 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = =3D { */ =20 /* - * Iterate over the VM's list of mapped LPIs to find the one with a - * matching interrupt ID and return a reference to the IRQ structure. + * Index the VM's xarray of mapped LPIs and return a reference to the IRQ + * structure. The caller is expected to call vgic_put_irq() later once it's + * finished with the IRQ. */ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid) { @@ -66,20 +67,10 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u= 32 intid) =20 raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); =20 - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { - if (irq->intid !=3D intid) - continue; - - /* - * This increases the refcount, the caller is expected to - * call vgic_put_irq() later once it's finished with the IRQ. - */ + irq =3D xa_load(&dist->lpi_xa, intid); + if (irq) vgic_get_irq_kref(irq); - goto out_unlock; - } - irq =3D NULL; =20 -out_unlock: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); =20 return irq; --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-182.mta0.migadu.com (out-182.mta0.migadu.com [91.218.175.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2882E3B193 for ; Wed, 21 Feb 2024 05:43:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494208; cv=none; b=EOXS9Ctp/1boTXuUXevKyKHaCiJGS/XEVqz7gRAyAxTCopRvYeijz/ZIqXJc56S2GRFxFq1E1RTjRiXaLhAT5rGPOD828BQkowkzWHDYtaWrH+FsKHVxVozM43ftjx3Ey95zizCKXzQI1Hncau6Lb831HTdd497awuvMKRL5NfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494208; c=relaxed/simple; bh=RxF9w33LgnXLTuXV5O9RLdzK9dXUqE5MSbsOmdxxwVA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LCtyuJ9AtPEq0qAr2LfMpcymzPEBhIb8DTh11NsRtvGBErFjvtc0LLrOwikdEdJEcr5pYOZIH8AIMDWh/87g2K6hL3Lsl+fwo02TWh7PlL6UI0ozWHMTF8ZeJY9SvJTICvfDey8912UdZLwLlukhiFhkK2SYw5U2sWV2iTNBVPE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=GL2D5v0S; arc=none smtp.client-ip=91.218.175.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="GL2D5v0S" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494204; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p7/gBwWxFbIZwFC+F2czZFaZoLMKZvcePyaMRonjua4=; b=GL2D5v0SC8vLCvny+w0lysq1H7+eVYaJPjgBXHDAX31SpwfBHeusS509F4eACvmVOuX2oW YZQOwMa3/zFkcg+V/vPyOoDP+9UuhCmyIp2XJVxwtXOjQ3Cs4Ip3o/VYERU1asvwz4SbZF cnqOWEIcewTFCBKCM25xdIFUmrmg9AE= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 03/10] KVM: arm64: vgic-v3: Iterate the xarray to find pending LPIs Date: Wed, 21 Feb 2024 05:42:46 +0000 Message-ID: <20240221054253.3848076-4-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Start walking the LPI xarray to find pending LPIs in preparation for the removal of the LPI linked-list. Note that the 'basic' iterator is chosen here as each iteration needs to drop the xarray read lock (RCU) as reads/writes to guest memory can potentially block. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-v3.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c index 9465d3706ab9..4ea3340786b9 100644 --- a/arch/arm64/kvm/vgic/vgic-v3.c +++ b/arch/arm64/kvm/vgic/vgic-v3.c @@ -380,6 +380,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) struct vgic_irq *irq; gpa_t last_ptr =3D ~(gpa_t)0; bool vlpi_avail =3D false; + unsigned long index; int ret =3D 0; u8 val; =20 @@ -396,7 +397,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) vlpi_avail =3D true; } =20 - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { + xa_for_each(&dist->lpi_xa, index, irq) { int byte_offset, bit_nr; struct kvm_vcpu *vcpu; gpa_t pendbase, ptr; --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [91.218.175.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17CCA39FC9 for ; Wed, 21 Feb 2024 05:43:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494209; cv=none; b=CE3pPC6e7oWRuydgyYrWxt8u0OreFbrDqazgkhcoF18dVTzRwy4UbR/96/Ukqtgr+DzkqeBZ6zVASoqUaLdI635jQFVV9/spFWD5P/bIV1l0gxav7sR1Rxkai/7twZyOFPx2G1m/sOKnCTtDFVjH+2ofXP6YgVKF+E7Vn/YEL/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494209; c=relaxed/simple; bh=egvxRC64pImQWpYJ5H/66l//0W0vYIRaEzxh49R2ggA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JDXUzHkJbLySEteSPqxuw1cm8XrLhGPDirgTvZrUpUaeWGoZgtaoAAngyUFttoKvE/lcuo9AEK7x8q7l6QIwpc2bfOTun7NJsuJx+KXzkBJvmnv/huXgxt061/Pf6t4Oq5d8ncwhcu2niZ5Js+uZOpgbs50CNFmRhAlQ5khQJpo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=MGYDRQmp; arc=none smtp.client-ip=91.218.175.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="MGYDRQmp" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494206; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hwGBWvx+wwsJa7xNRdwDH7MJ0Q05E6x3UjwcqNXt4/0=; b=MGYDRQmpErICmMJRKOaSwPhNZwcGWG9i3eBvE7S9eAoTsD13nFyL/UaN2lmj6158Yql47k AE2cr3rXzXv4faz9wb7vFc2SwzJUyIy/dLvzFP+PJ69TdIyorSkUwcQq8Iej9FIRHfuPhI PvHsPUfiiZp1eoJHZjh3RHF+HbDCms4= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 04/10] KVM: arm64: vgic-its: Walk the LPI xarray in vgic_copy_lpi_list() Date: Wed, 21 Feb 2024 05:42:47 +0000 Message-ID: <20240221054253.3848076-5-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Start iterating the LPI xarray in anticipation of removing the LPI linked-list. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-its.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index fb2d3c356984..b9874dc04608 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -327,6 +327,8 @@ static int update_lpi_config(struct kvm *kvm, struct vg= ic_irq *irq, return 0; } =20 +#define GIC_LPI_MAX_INTID ((1 << INTERRUPT_ID_BITS_ITS) - 1) + /* * Create a snapshot of the current LPIs targeting @vcpu, so that we can * enumerate those LPIs without holding any lock. @@ -335,6 +337,7 @@ static int update_lpi_config(struct kvm *kvm, struct vg= ic_irq *irq, int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid= _ptr) { struct vgic_dist *dist =3D &kvm->arch.vgic; + XA_STATE(xas, &dist->lpi_xa, GIC_LPI_OFFSET); struct vgic_irq *irq; unsigned long flags; u32 *intids; @@ -353,7 +356,9 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu= *vcpu, u32 **intid_ptr) return -ENOMEM; =20 raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { + rcu_read_lock(); + + xas_for_each(&xas, irq, GIC_LPI_MAX_INTID) { if (i =3D=3D irq_count) break; /* We don't need to "get" the IRQ, as we hold the list lock. */ @@ -361,6 +366,8 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu= *vcpu, u32 **intid_ptr) continue; intids[i++] =3D irq->intid; } + + rcu_read_unlock(); raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); =20 *intid_ptr =3D intids; --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-185.mta0.migadu.com (out-185.mta0.migadu.com [91.218.175.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 483B93BB47 for ; Wed, 21 Feb 2024 05:43:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.185 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494212; cv=none; b=UJq65WB7uYXm0IzJTvggq2sytX4SrVfB9Vx15mCFbaU/oYPg75U/Fs95lM1Pls+oYxK17XorP71SorJwUotXcU/cV5Vy2gB9h5VjK0qxlB4TqiN+k+fBV6qbHe32VUUf240eTOU2qdydrteK8DNKUzZVfJ9L299bh5+Sk+lMFPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494212; c=relaxed/simple; bh=2nkDTHcHmB33a/QQu7UDTZHWtSxxacimi3tTiZ9a1sQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qXFcI0h8Kid3AswO8iAgnQgGdW1nIz/Mxp4Cn1ELEEfNjmYJpZQcW8sbH9uu2Rvah300aqkoXPAPZuhkBXUuKcVl4PRLpcvgihu3L+tCY800DG+6OI12uKKTPW31HkuJbep3tyuqRbEaGq7hgExbEDGUOoUXsRKF5fG1CBkzdns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=kWM4CnAI; arc=none smtp.client-ip=91.218.175.185 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="kWM4CnAI" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8nd2SXFQ1C1LlktYqUUc4CI4OG59iyq71GzSyfZOU20=; b=kWM4CnAIhzzDn6hcA5UWEWtiSn+E/I2Euqe4c3srTEF4gOULTNs5ApWtlcLfPBFH8QZZ7I 3Zvffct7NXAKoZXaEN6FPXMNl+/PinjhM/iFTxpVAUNvJCWvLZv9aJMYYAbWSXFDcfTRsh dCaDPDNwqHphtjV5ajguseu6IlopyMY= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 05/10] KVM: arm64: vgic: Get rid of the LPI linked-list Date: Wed, 21 Feb 2024 05:42:48 +0000 Message-ID: <20240221054253.3848076-6-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" All readers of LPI configuration have been transitioned to use the LPI xarray. Get rid of the linked-list altogether. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-init.c | 1 - arch/arm64/kvm/vgic/vgic-its.c | 8 ++------ arch/arm64/kvm/vgic/vgic.c | 1 - include/kvm/arm_vgic.h | 2 -- 4 files changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-ini= t.c index 411719053107..e25672d6e846 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -53,7 +53,6 @@ void kvm_vgic_early_init(struct kvm *kvm) { struct vgic_dist *dist =3D &kvm->arch.vgic; =20 - INIT_LIST_HEAD(&dist->lpi_list_head); INIT_LIST_HEAD(&dist->lpi_translation_cache); raw_spin_lock_init(&dist->lpi_list_lock); xa_init_flags(&dist->lpi_xa, XA_FLAGS_LOCK_IRQ); diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index b9874dc04608..3d0208162bcd 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -58,7 +58,6 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32= intid, return ERR_PTR(ret); } =20 - INIT_LIST_HEAD(&irq->lpi_list); INIT_LIST_HEAD(&irq->ap_list); raw_spin_lock_init(&irq->irq_lock); =20 @@ -74,10 +73,8 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u3= 2 intid, * There could be a race with another vgic_add_lpi(), so we need to * check that we don't add a second list entry with the same LPI. */ - list_for_each_entry(oldirq, &dist->lpi_list_head, lpi_list) { - if (oldirq->intid !=3D intid) - continue; - + oldirq =3D xa_load(&dist->lpi_xa, intid); + if (oldirq) { /* Someone was faster with adding this LPI, lets use that. */ kfree(irq); irq =3D oldirq; @@ -99,7 +96,6 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32= intid, goto out_unlock; } =20 - list_add_tail(&irq->lpi_list, &dist->lpi_list_head); dist->lpi_list_count++; =20 out_unlock: diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 7d17dbc8f5dc..6240faab0127 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -122,7 +122,6 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic= _irq *irq) if (!kref_put(&irq->refcount, vgic_irq_release)) return; =20 - list_del(&irq->lpi_list); xa_erase(&dist->lpi_xa, irq->intid); dist->lpi_list_count--; =20 diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 795b35656b54..aeff363e3ba6 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -117,7 +117,6 @@ struct irq_ops { =20 struct vgic_irq { raw_spinlock_t irq_lock; /* Protects the content of the struct */ - struct list_head lpi_list; /* Used to link all LPIs together */ struct list_head ap_list; =20 struct kvm_vcpu *vcpu; /* SGIs and PPIs: The VCPU @@ -277,7 +276,6 @@ struct vgic_dist { /* Protects the lpi_list and the count value below. */ raw_spinlock_t lpi_list_lock; struct xarray lpi_xa; - struct list_head lpi_list_head; int lpi_list_count; =20 /* LPI translation cache */ --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-180.mta0.migadu.com (out-180.mta0.migadu.com [91.218.175.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE9663A8E5 for ; Wed, 21 Feb 2024 05:43:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494213; cv=none; b=d+Z4cmhnQhNpublbzzYzu0larUrgdDx1Mib9O0iLJk/I9PWPbOlI6H0pHBJNGR2Avm3GNCjf+vMeVPjvv5FteY+5QnuxwDZnsW64zta9npmz9E/FUqDAU/60dIZ+Tg/T4w+UgXz9fHJOmfafv60S44gt5etiaxtuPp3qtqupjy0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494213; c=relaxed/simple; bh=i1Z17y1L2zZUqDCMRJDQXDFZa8iLbDZzznncMJH6hZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j6csHFl7jWTbDhq2ulvowNe5WvT97dqwLDH3NwDJB/DGm17RnLCxF7ZKaHXoYPkWkuPjK/uboNgPTlVpapS2pWQkPavODgzyeDUak/QU2WHUC1nIbKBUbj4HhDqMaLkl2zkGP9VHXKJRdjRdCiK/fbyRZD2J0x7VZaYuv6jBlwc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=JOdXlq8W; arc=none smtp.client-ip=91.218.175.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="JOdXlq8W" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494210; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OSxryl4GJz7jglvRnwO6HB5bazTdcs2fEdVdtMmeB8s=; b=JOdXlq8WUZfIK0n17nGijHUe5mPeuDqw2JyGKCciNDJMECFQT4kEAYdfH3vwhxjHvWm+Qb pb7a+3tuYFEeojv3/zOuWw6jNPAcD0AK+UZkb4eA8f0unN752nf2as6SvuFCUq9uK7Omv5 pcjH9bPZKe2pu2gtm7P8FjZrjXkjBb0= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 06/10] KVM: arm64: vgic: Use atomics to count LPIs Date: Wed, 21 Feb 2024 05:42:49 +0000 Message-ID: <20240221054253.3848076-7-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Switch to using atomics for LPI accounting, allowing vgic_irq references to be dropped in parallel. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-debug.c | 2 +- arch/arm64/kvm/vgic/vgic-its.c | 4 ++-- arch/arm64/kvm/vgic/vgic.c | 2 +- include/kvm/arm_vgic.h | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-debug.c b/arch/arm64/kvm/vgic/vgic-de= bug.c index 85606a531dc3..389025ce7749 100644 --- a/arch/arm64/kvm/vgic/vgic-debug.c +++ b/arch/arm64/kvm/vgic/vgic-debug.c @@ -149,7 +149,7 @@ static void print_dist_state(struct seq_file *s, struct= vgic_dist *dist) seq_printf(s, "vgic_model:\t%s\n", v3 ? "GICv3" : "GICv2"); seq_printf(s, "nr_spis:\t%d\n", dist->nr_spis); if (v3) - seq_printf(s, "nr_lpis:\t%d\n", dist->lpi_list_count); + seq_printf(s, "nr_lpis:\t%d\n", atomic_read(&dist->lpi_count)); seq_printf(s, "enabled:\t%d\n", dist->enabled); seq_printf(s, "\n"); =20 diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 3d0208162bcd..0be3c33676c3 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -96,7 +96,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32= intid, goto out_unlock; } =20 - dist->lpi_list_count++; + atomic_inc(&dist->lpi_count); =20 out_unlock: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); @@ -346,7 +346,7 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu= *vcpu, u32 **intid_ptr) * command). If coming from another path (such as enabling LPIs), * we must be careful not to overrun the array. */ - irq_count =3D READ_ONCE(dist->lpi_list_count); + irq_count =3D atomic_read(&dist->lpi_count); intids =3D kmalloc_array(irq_count, sizeof(intids[0]), GFP_KERNEL_ACCOUNT= ); if (!intids) return -ENOMEM; diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 6240faab0127..15dbd17b3a9e 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -123,7 +123,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic= _irq *irq) return; =20 xa_erase(&dist->lpi_xa, irq->intid); - dist->lpi_list_count--; + atomic_dec(&dist->lpi_count); =20 kfree(irq); } diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index aeff363e3ba6..71e9d719533b 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -273,10 +273,10 @@ struct vgic_dist { */ u64 propbaser; =20 - /* Protects the lpi_list and the count value below. */ + /* Protects the lpi_list. */ raw_spinlock_t lpi_list_lock; struct xarray lpi_xa; - int lpi_list_count; + atomic_t lpi_count; =20 /* LPI translation cache */ struct list_head lpi_translation_cache; --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-189.mta0.migadu.com (out-189.mta0.migadu.com [91.218.175.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 670C63A1C7; Wed, 21 Feb 2024 05:43:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494215; cv=none; b=Pu56ponqHgCc/vwDKL68Pc6nfyyNN+6UpJzCgahjr2CEs9A4VmkUgUa8D2u5PTcIHZl75RLPY1p2qMD8JAXSvBdqVBs8GsFw8N4ZilsBvFFv1YrjeXBWqcJ7r2wH14QHdg06ta9iUq8BsKO5L1BWSYq7lTKRmEc5NbB4hiaDa74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494215; c=relaxed/simple; bh=iPMx6to6nitv27V003CljDT9TCjGzeZ9T5PREHKTqXY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u1nNYgTX7iawtHklM1McCvSnuuRPf7V4kCinMjdCbinlHOTybJ+3VINq6ryu0xLK7uTNbxsRStm+9TCiFfo2ucq5Iangpw9TA0YtU0/jpDgPimcXfSskyTg6ZLXHbJELK6TVqu2sbCwv8NKWoqqkMxmQ+9uhlYsYUSXqNHiGMuo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=sdaejefD; arc=none smtp.client-ip=91.218.175.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="sdaejefD" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w2vP8fjlFG48ofYu0phwr6P5tTp39+9IxC+FJcXQ4f8=; b=sdaejefD5Jik4CTUWmfbhv0AYU/Q05Y8Op4W9bykUArkL/dz+jcAyW8mywmef8uZMcnfDU UIRvZ/bQ86t1OfXpl0cxcOBySejSehv3d6tkIZpwMvuPSuTYx748F88BSGe7BdJWFS/uzi AIMGLQHhX8DNtbIksD7DZa4YvK5oWhs= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 07/10] KVM: arm64: vgic: Free LPI vgic_irq structs in an RCU-safe manner Date: Wed, 21 Feb 2024 05:42:50 +0000 Message-ID: <20240221054253.3848076-8-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Free the vgic_irq structs in an RCU-safe manner to allow reads of the LPI configuration data to happen in parallel with the release of LPIs. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic.c | 2 +- include/kvm/arm_vgic.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 15dbd17b3a9e..3fedc58e663a 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -125,7 +125,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic= _irq *irq) xa_erase(&dist->lpi_xa, irq->intid); atomic_dec(&dist->lpi_count); =20 - kfree(irq); + kfree_rcu(irq, rcu); } =20 void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 71e9d719533b..47035946648e 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -117,6 +117,7 @@ struct irq_ops { =20 struct vgic_irq { raw_spinlock_t irq_lock; /* Protects the content of the struct */ + struct rcu_head rcu; struct list_head ap_list; =20 struct kvm_vcpu *vcpu; /* SGIs and PPIs: The VCPU --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-174.mta0.migadu.com (out-174.mta0.migadu.com [91.218.175.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58A7C3CF6C for ; Wed, 21 Feb 2024 05:43:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494217; cv=none; b=KPk0fnRD6QW2lI94H2bLGjgGhe793t4vI3/wCUotTyVqW/e+Hb+ILHRkQs6ifd3143Mhw9ub5RTlvlwmYiKc60uZHzOsHLmEpKwDlQ3/LIB5T/bWJiptRnqczq6ozqocWb5ZRoMGY3NyYlawfJ2V+qjYqRld0dpDDnCPV2vRCm8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494217; c=relaxed/simple; bh=w2mKAUpjDefBH8ZRTYVpf6wUaoHL7tLPKvgNGeP/7Pc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZTsEGKeBol5R8edRpHguD+Dyn314/eV+OKyEho6cmk/OBc8s2Lcrdgp4DZ8PXBFe9r5+v6dIpxho1mT0Xsrh/nU9JFKIag/N0GLWViu9Xhwv1lORw7hbjaQR9zQvvsoXZjvh5DOkUqfL0LLpOMITJpahYhF4APrynYq5XU3jA2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=UhQFNMtJ; arc=none smtp.client-ip=91.218.175.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="UhQFNMtJ" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BJ4oP9M3Q0IijgdCm1rTjm/2Zq88g2zw/bm3B6utCQw=; b=UhQFNMtJMJzvESlIIp0h2831VVZci4irnCeQlhWnqIopdwDfPiffDEsl1FdQD/dUj82J4M ai73hWEqta6WhHvDHXn6FLl1i57BCdfuXEx6luFE3soK+eVjBFfYjSCh9j3BoY4wgq0fDC cziW4XfD063WKUDH4KIEB9TmWsv5rVM= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 08/10] KVM: arm64: vgic: Rely on RCU protection in vgic_get_lpi() Date: Wed, 21 Feb 2024 05:42:51 +0000 Message-ID: <20240221054253.3848076-9-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Stop acquiring the lpi_list_lock in favor of RCU for protecting the read-side critical section in vgic_get_lpi(). In order for this to be safe, we also need to be careful not to take a reference on an irq with a refcount of 0, as it is about to be freed. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic.c | 9 ++++----- arch/arm64/kvm/vgic/vgic.h | 14 +++++++++++--- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 3fedc58e663a..76abf3d946fe 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -63,15 +63,14 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u= 32 intid) { struct vgic_dist *dist =3D &kvm->arch.vgic; struct vgic_irq *irq =3D NULL; - unsigned long flags; =20 - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); + rcu_read_lock(); =20 irq =3D xa_load(&dist->lpi_xa, intid); - if (irq) - vgic_get_irq_kref(irq); + if (!vgic_try_get_irq_kref(irq)) + irq =3D NULL; =20 - raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + rcu_read_unlock(); =20 return irq; } diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h index 8d134569d0a1..f874b9932c5a 100644 --- a/arch/arm64/kvm/vgic/vgic.h +++ b/arch/arm64/kvm/vgic/vgic.h @@ -220,12 +220,20 @@ void vgic_v2_vmcr_sync(struct kvm_vcpu *vcpu); void vgic_v2_save_state(struct kvm_vcpu *vcpu); void vgic_v2_restore_state(struct kvm_vcpu *vcpu); =20 -static inline void vgic_get_irq_kref(struct vgic_irq *irq) +static inline bool vgic_try_get_irq_kref(struct vgic_irq *irq) { + if (!irq) + return false; + if (irq->intid < VGIC_MIN_LPI) - return; + return true; =20 - kref_get(&irq->refcount); + return kref_get_unless_zero(&irq->refcount); +} + +static inline void vgic_get_irq_kref(struct vgic_irq *irq) +{ + WARN_ON_ONCE(!vgic_try_get_irq_kref(irq)); } =20 void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu); --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [91.218.175.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F2E23D0C6 for ; Wed, 21 Feb 2024 05:43:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494220; cv=none; b=nts3WQmI7Hj10QJ6wCKaxaQ+hjeZ6bfINOQG8WNOHH3GYfGC6aFZNE47RoQpsucsRgnlbabHlFgRLGxOmTR+yvRBHbbs72Q2kEwIXwxQaE7u0rlMR2E4iSrE8LNfBGtiUD4oVcv5oaIjW3qyC8Bs/PWiLoiw6TFoaAG5vZkZjGM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494220; c=relaxed/simple; bh=ohZ5HNh3+AbhZtUPEYzsTEwYsRO0b4zg7BkWRCA1M6o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LX6dUhjfgPEjgsnJRgOhiwDVvh7Nmfr3pJVG5uhLSZSdNx3nFjOdUtCyz6FQMF5dixsjDMtElFmrZl+plircMN6wEXUDHTGaPVyd2ET8bvWRU/rlX8DSUDIfo7pwQU1YVu/Lwr+BCIOyOShSluQQ1a0XwtgLj9V61S6WmAOmevU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=oJYMuUDa; arc=none smtp.client-ip=91.218.175.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="oJYMuUDa" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QeMz6ixCWY/1TUllIvS9OwAED3X7tPlGIFKged/im1A=; b=oJYMuUDa4lorjzjLyP4W/y7ThUhnzjPTg1eB4MUsxt75hClMvcOy7W5XhtgPpwI/Z4vK9f Q2cvCYDT3VIRIO3EH6bfRd2UeplzRCqfbqNE5BAQq8jYAUnSfYsZMCdvCrM4hYZDZyHB3Q QI7HCZ7WCRhcFu35cRe8lW/4RIxoxpY= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 09/10] KVM: arm64: vgic: Ensure the irq refcount is nonzero when taking a ref Date: Wed, 21 Feb 2024 05:42:52 +0000 Message-ID: <20240221054253.3848076-10-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" It will soon be possible for get() and put() calls to happen in parallel, which means in most cases we must ensure the refcount is nonzero when taking a new reference. Switch to using vgic_try_get_irq_kref() where necessary, and document the few conditions where an IRQ's refcount is guaranteed to be nonzero. Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-its.c | 18 ++++++++---------- arch/arm64/kvm/vgic/vgic.c | 3 ++- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 0be3c33676c3..dad6f0ee7c49 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -74,18 +74,11 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u= 32 intid, * check that we don't add a second list entry with the same LPI. */ oldirq =3D xa_load(&dist->lpi_xa, intid); - if (oldirq) { + if (vgic_try_get_irq_kref(oldirq)) { /* Someone was faster with adding this LPI, lets use that. */ kfree(irq); irq =3D oldirq; =20 - /* - * This increases the refcount, the caller is expected to - * call vgic_put_irq() on the returned pointer once it's - * finished with the IRQ. - */ - vgic_get_irq_kref(irq); - goto out_unlock; } =20 @@ -611,8 +604,8 @@ static struct vgic_irq *vgic_its_check_cache(struct kvm= *kvm, phys_addr_t db, raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); =20 irq =3D __vgic_its_check_cache(dist, db, devid, eventid); - if (irq) - vgic_get_irq_kref(irq); + if (!vgic_try_get_irq_kref(irq)) + irq =3D NULL; =20 raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); =20 @@ -658,6 +651,11 @@ static void vgic_its_cache_translation(struct kvm *kvm= , struct vgic_its *its, if (cte->irq) __vgic_put_lpi_locked(kvm, cte->irq); =20 + /* + * The irq refcount is guaranteed to be nonzero while holding the + * its_lock, as the ITE (and the reference it holds) cannot be freed. + */ + lockdep_assert_held(&its->its_lock); vgic_get_irq_kref(irq); =20 cte->db =3D db; diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 76abf3d946fe..df9e1aa1956c 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -395,7 +395,8 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic= _irq *irq, =20 /* * Grab a reference to the irq to reflect the fact that it is - * now in the ap_list. + * now in the ap_list. This is safe as the caller must already hold a + * reference on the irq. */ vgic_get_irq_kref(irq); list_add_tail(&irq->ap_list, &vcpu->arch.vgic_cpu.ap_list_head); --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 11:30:51 2026 Received: from out-170.mta0.migadu.com (out-170.mta0.migadu.com [91.218.175.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C6953A268 for ; Wed, 21 Feb 2024 05:43:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494222; cv=none; b=rkxGO46GGPCPs0qzZGOiNODYZZx22qMabdPc4HwP0dOQ/hbrBWLzelgJAT5vrwUwnV5vEZfUsyRCkNnG7c9iesz3Ortpl1BfpEIatNrco/koE9xibaMFi8FmrvNjoZGGxrfv1nnPtkoFJM3PdI7fQTx6A+1iLrS3OZtjbG8OYJ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494222; c=relaxed/simple; bh=huN1PqJtmqytpL0vyqUMZqGEJFXVuxAo91FZdlEJvZw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ANrTd1w0r8MGB6wJu2cg3bWDC+q9JOtrD3YWHLUpRLR0z3pFx2XNV1lsOrqeUIvX1kAFTC/W1pHOmTvCu18afoZXnj+y5ToRTlGN9sDxP+ImaX8uGcgLSNaMfrPlmtrtHWntasxslp7zkZ1QUbiZoNxgGUDar+H67Rj9F2GYWmg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=RK/jSFpK; arc=none smtp.client-ip=91.218.175.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="RK/jSFpK" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7uZnxcLXKLqyPCNevo5l7NpGLhMeejXcsQuYRK9QxO0=; b=RK/jSFpKYJqpDZ2ouKFiZ2I/rdOcsLx5lKza9bI7O/DEvmkWAWOwRt7MBFuz/AgkDXmv1z 1DsW4AldOgmJ72zxrmxbd18hjzeuCR1L2nPxHUhyTOjfZ4E7Nnk5ArYF9wBFEWr2ZBsot/ q64aIxAUJxhWVqMsKVznWHHqpDmDvVA= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 10/10] KVM: arm64: vgic: Don't acquire the lpi_list_lock in vgic_put_irq() Date: Wed, 21 Feb 2024 05:42:53 +0000 Message-ID: <20240221054253.3848076-11-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" The LPI xarray's xa_lock is sufficient for synchronizing writers when freeing a given LPI. Furthermore, readers can only take a new reference on an IRQ if it was already nonzero. Stop taking the lpi_list_lock unnecessarily and get rid of __vgic_put_lpi_locked(). Signed-off-by: Oliver Upton Reviewed-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-its.c | 4 ++-- arch/arm64/kvm/vgic/vgic.c | 28 +++++++++------------------- arch/arm64/kvm/vgic/vgic.h | 1 - 3 files changed, 11 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index dad6f0ee7c49..f6025886071c 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -649,7 +649,7 @@ static void vgic_its_cache_translation(struct kvm *kvm,= struct vgic_its *its, * was in the cache, and increment it on the new interrupt. */ if (cte->irq) - __vgic_put_lpi_locked(kvm, cte->irq); + vgic_put_irq(kvm, cte->irq); =20 /* * The irq refcount is guaranteed to be nonzero while holding the @@ -686,7 +686,7 @@ void vgic_its_invalidate_cache(struct kvm *kvm) if (!cte->irq) break; =20 - __vgic_put_lpi_locked(kvm, cte->irq); + vgic_put_irq(kvm, cte->irq); cte->irq =3D NULL; } =20 diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index df9e1aa1956c..f963f410788a 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -111,22 +111,6 @@ static void vgic_irq_release(struct kref *ref) { } =20 -/* - * Drop the refcount on the LPI. Must be called with lpi_list_lock held. - */ -void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) -{ - struct vgic_dist *dist =3D &kvm->arch.vgic; - - if (!kref_put(&irq->refcount, vgic_irq_release)) - return; - - xa_erase(&dist->lpi_xa, irq->intid); - atomic_dec(&dist->lpi_count); - - kfree_rcu(irq, rcu); -} - void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) { struct vgic_dist *dist =3D &kvm->arch.vgic; @@ -135,9 +119,15 @@ void vgic_put_irq(struct kvm *kvm, struct vgic_irq *ir= q) if (irq->intid < VGIC_MIN_LPI) return; =20 - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - __vgic_put_lpi_locked(kvm, irq); - raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + if (!kref_put(&irq->refcount, vgic_irq_release)) + return; + + xa_lock_irqsave(&dist->lpi_xa, flags); + __xa_erase(&dist->lpi_xa, irq->intid); + xa_unlock_irqrestore(&dist->lpi_xa, flags); + + atomic_dec(&dist->lpi_count); + kfree_rcu(irq, rcu); } =20 void vgic_flush_pending_lpis(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h index f874b9932c5a..0c2b82de8fa3 100644 --- a/arch/arm64/kvm/vgic/vgic.h +++ b/arch/arm64/kvm/vgic/vgic.h @@ -180,7 +180,6 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic= _io_device *iodev, gpa_t addr, int len); struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 intid); -void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq); void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq); bool vgic_get_phys_line_level(struct vgic_irq *irq); void vgic_irq_set_phys_pending(struct vgic_irq *irq, bool pending); --=20 2.44.0.rc0.258.g7320e95886-goog