From nobody Mon Sep 28 21:08:22 2026 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E98FD3E95AB for ; Mon, 17 Aug 2026 14:28:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786976910; cv=none; b=Ojth7KwRbwgNjKhwSCImfLWJP8O9xKOaJlglxFJF9s3y66L32FwOvmzIoMFjwOBXJDMNIYAXkSKfEV+ti3slXjmxGOLK/TAuHKWmWeVvRuQO9nwFQI4JhCAEPkzhxu20uhvbv6Y4uVCaaLgIfaiOEGVdSA6bFufzumAVXmc/Tsk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786976910; c=relaxed/simple; bh=6ZEVi/aOJyaJhuddMfCe08+BHazf/2XCD9aBzk4QniM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pXZHbLm3IM3p1WqikChfh+M4IGWBQdjiJe3GkXvZJj6za/UIcE0g/QkpJZSefheaJD1JN1muY1Y1ueXdmDN5JRKZaVjKVVgNwo1R/tm/MFWPDBe5pgtYZg39wvm1O5t7fzys/OqYVdWJfSGZnVWXIgvIv6gtW3xxQqtqn261Ay8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=KF1g1Lxa; arc=none smtp.client-ip=115.124.30.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="KF1g1Lxa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1786976904; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=Zhv3H2+2w0gZKGeUibRr/8VGF24nes4VuRka42uR/+4=; b=KF1g1Lxa+hfU51rJXttz2hh2LzK0mSj/p8eZZesztngKCnh843owqZ6XlbVVpQ1RboUBX0l9Bxlkdy08zXG3DhWUDD3YIeeUG3NdI1l+K2xfMEaYLNQTUUUYRSeJ/GoFOgBpHAKFX8sbwvYhXOH3RgiTQuWuzTKl4GDFPzhjZkE= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033032089153;MF=fangyu.yu@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0X9A9H-f_1786976902; Received: from localhost.localdomain(mailfrom:fangyu.yu@linux.alibaba.com fp:SMTPD_---0X9A9H-f_1786976902 cluster:ay36) by smtp.aliyun-inc.com; Mon, 17 Aug 2026 22:28:23 +0800 From: fangyu.yu@linux.alibaba.com To: tomasz.jeznach@linux.dev, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, baolu.lu@linux.intel.com, jroedel@suse.de, zong.li@sifive.com Cc: fangyu.yu@linux.alibaba.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [PATCH 1/3] iommu/riscv: Add command queue lock Date: Mon, 17 Aug 2026 22:27:26 +0800 Message-Id: <20260817142728.61783-2-fangyu.yu@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260817142728.61783-1-fangyu.yu@linux.alibaba.com> References: <20260817142728.61783-1-fangyu.yu@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fangyu Yu Add a raw spinlock to the RISC-V IOMMU queue state so command queue publishing can be serialized by a later change. Fixes: 856c0cfe5c5f ("iommu/riscv: Command and fault queue support") Signed-off-by: Fangyu Yu --- drivers/iommu/riscv/iommu.c | 1 + drivers/iommu/riscv/iommu.h | 2 ++ 2 files changed, 3 insertions(+) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index cec3ddd7ab10..2c0dcc90cf85 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1560,6 +1560,7 @@ int riscv_iommu_init(struct riscv_iommu_device *iommu) int rc; =20 RISCV_IOMMU_QUEUE_INIT(&iommu->cmdq, CQ); + raw_spin_lock_init(&iommu->cmdq.lock); RISCV_IOMMU_QUEUE_INIT(&iommu->fltq, FQ); =20 rc =3D riscv_iommu_init_check(iommu); diff --git a/drivers/iommu/riscv/iommu.h b/drivers/iommu/riscv/iommu.h index 46df79dd5495..5676001548cc 100644 --- a/drivers/iommu/riscv/iommu.h +++ b/drivers/iommu/riscv/iommu.h @@ -12,6 +12,7 @@ #define _RISCV_IOMMU_H_ =20 #include +#include #include #include =20 @@ -23,6 +24,7 @@ struct riscv_iommu_queue { atomic_t prod; /* unbounded producer allocation index */ atomic_t head; /* unbounded shadow ring buffer consumer index */ atomic_t tail; /* unbounded shadow ring buffer producer index */ + raw_spinlock_t lock; /* serialize queue publishing */ unsigned int mask; /* index mask, queue length - 1 */ unsigned int irq; /* allocated interrupt number */ struct riscv_iommu_device *iommu; /* iommu device handling the queue when= active */ --=20 2.50.1 From nobody Mon Sep 28 21:08:22 2026 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BBB94307BC for ; Mon, 17 Aug 2026 14:28:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.112 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786976918; cv=none; b=JbIrXobEKJm6X2P15y+kCyhq1V9ZjR4yICA/U614LxmxtGf39NCb7zdtuMBabBMQR7ndatuu8DvNwM6Py8FRpnrXWrGoAcuye6iRwLEcX4gzLOQkALVxGTsKLpt2ny6BeMW1GoOmZwG1PLS33ViqwFpE+0jSFW9tWLmLutHWQgo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786976918; c=relaxed/simple; bh=Jtvo19rIBm9yNZwd+I4NM0mPW5LObGvu96WwM6ZZW78=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fXIOB6LawO1A1+UHe8OLnL5hbI+M23m3WtVEFNPXNvJVoLQEO6Zm6VL8fvhvucBZFVD1bCbBvd5C/MgFQ8Z9OQlDPMBUVXQRAZ1QeBSuf1ZwpGeLLPUG+T4vA4BumDi93BFhkk7PbEW240zTIuD49L76dReiYkGE3p1dGxDcAMo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=IHiFtKA8; arc=none smtp.client-ip=115.124.30.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="IHiFtKA8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1786976906; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=aP1uKS6O6CEvKFWqd9T2SU+tlBniH5mDxsxhyUw4zbc=; b=IHiFtKA8w6olpkVM0qdBRA+R7SpdDVodjx++F3+xzfKgd5j51fk+nU97qrxR8ZYhtjdBRX7BY2N3T7AhYB9cgHbv92STAdT3odVxLgjI2/b0ZIjQcNrIfmCzuBcCK2MKQxiQzBtWY/finM/hzeKW0wjPmWhbd3PO0h2PxPTbMtI= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037009110;MF=fangyu.yu@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0X9A9H-t_1786976903; Received: from localhost.localdomain(mailfrom:fangyu.yu@linux.alibaba.com fp:SMTPD_---0X9A9H-t_1786976903 cluster:ay36) by smtp.aliyun-inc.com; Mon, 17 Aug 2026 22:28:24 +0800 From: fangyu.yu@linux.alibaba.com To: tomasz.jeznach@linux.dev, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, baolu.lu@linux.intel.com, jroedel@suse.de, zong.li@sifive.com Cc: fangyu.yu@linux.alibaba.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [PATCH 2/3] iommu/riscv: Serialize command queue publishing Date: Mon, 17 Aug 2026 22:27:27 +0800 Message-Id: <20260817142728.61783-3-fangyu.yu@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260817142728.61783-1-fangyu.yu@linux.alibaba.com> References: <20260817142728.61783-1-fangyu.yu@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fangyu Yu Serialize command queue publishing so software producer state advances only after a command is written and the hardware tail is updated. Wait for hardware consumption outside the queue lock when the command queue is full so other CPUs are not blocked behind a long poll. Fixes: 856c0cfe5c5f ("iommu/riscv: Command and fault queue support") Signed-off-by: Fangyu Yu --- drivers/iommu/riscv/iommu.c | 104 +++++++++++++++++++++--------------- 1 file changed, 62 insertions(+), 42 deletions(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 2c0dcc90cf85..86cec408be5b 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -382,77 +382,97 @@ static int riscv_iommu_queue_wait(struct riscv_iommu_= queue *queue, (int)(cons - index) > 0, 0, timeout_us); } =20 -/* Enqueue an entry and wait to be processed if timeout_us > 0 - * - * Error handling for IOMMU hardware not responding in reasonable time - * will be added as separate patch series along with other RAS features. - * For now, only report hardware failure and continue. - */ +static int riscv_iommu_queue_wait_for_space(struct riscv_iommu_queue *queu= e, + unsigned int last) +{ + unsigned int head; + unsigned int tail; + unsigned int hw_head; + unsigned long flags; + int ret; + + ret =3D riscv_iommu_readl_timeout(queue->iommu, Q_HEAD(queue), hw_head, + !(hw_head & ~queue->mask) && hw_head !=3D last, + 0, RISCV_IOMMU_QUEUE_TIMEOUT); + if (ret) + return ret; + + raw_spin_lock_irqsave(&queue->lock, flags); + head =3D atomic_read(&queue->head); + tail =3D atomic_read(&queue->tail); + if ((tail - head) >=3D queue->mask) { + last =3D Q_ITEM(queue, head); + /* + * Re-read hw_head under the lock so that it is consistent with + * the freshly computed 'last'. Using the pre-lock snapshot + * could produce a stale value that wraps around relative to the + * new 'last', advancing the shadow head past entries that have + * not yet been consumed by the hardware. + */ + hw_head =3D riscv_iommu_readl(queue->iommu, Q_HEAD(queue)); + if (!(hw_head & ~queue->mask) && hw_head !=3D last) + atomic_add((hw_head - last) & queue->mask, &queue->head); + } + raw_spin_unlock_irqrestore(&queue->lock, flags); + + return 0; +} + +/* Enqueue an entry and publish it to the hardware queue. */ static unsigned int riscv_iommu_queue_send(struct riscv_iommu_queue *queue, void *entry, size_t entry_size) { unsigned int prod; unsigned int head; - unsigned int tail; unsigned long flags; + int ret; =20 - /* Do not preempt submission flow. */ - local_irq_save(flags); + /* 1. Wait for space availability and reserve the next slot. */ + for (;;) { + raw_spin_lock_irqsave(&queue->lock, flags); =20 - /* 1. Allocate some space in the queue */ - prod =3D atomic_inc_return(&queue->prod) - 1; - head =3D atomic_read(&queue->head); + prod =3D atomic_read(&queue->tail); + head =3D atomic_read(&queue->head); =20 - /* 2. Wait for space availability. */ - if ((prod - head) > queue->mask) { - if (readx_poll_timeout(atomic_read, &queue->head, - head, (prod - head) < queue->mask, - 0, RISCV_IOMMU_QUEUE_TIMEOUT)) - goto err_busy; - } else if ((prod - head) =3D=3D queue->mask) { - const unsigned int last =3D Q_ITEM(queue, head); + if ((prod - head) < queue->mask) + break; + + head =3D Q_ITEM(queue, head); + raw_spin_unlock_irqrestore(&queue->lock, flags); =20 - if (riscv_iommu_readl_timeout(queue->iommu, Q_HEAD(queue), head, - !(head & ~queue->mask) && head !=3D last, - 0, RISCV_IOMMU_QUEUE_TIMEOUT)) + ret =3D riscv_iommu_queue_wait_for_space(queue, head); + if (ret) { + raw_spin_lock_irqsave(&queue->lock, flags); + prod =3D atomic_read(&queue->tail); goto err_busy; - atomic_add((head - last) & queue->mask, &queue->head); + } } =20 - /* 3. Store entry in the ring buffer */ + /* 2. Store entry in the ring buffer. */ memcpy(queue->base + Q_ITEM(queue, prod) * entry_size, entry, entry_size); =20 - /* 4. Wait for all previous entries to be ready */ - if (readx_poll_timeout(atomic_read, &queue->tail, tail, prod =3D=3D tail, - 0, RISCV_IOMMU_QUEUE_TIMEOUT)) - goto err_busy; - - /* - * 5. Make sure the ring buffer update (whether in normal or I/O memory) = is - * completed and visible before signaling the tail doorbell to fetch - * the next command. 'fence ow, ow' - */ + /* 3. Make sure the entry is visible before updating the queue tail. */ dma_wmb(); riscv_iommu_writel(queue->iommu, Q_TAIL(queue), Q_ITEM(queue, prod + 1)); =20 /* - * 6. Make sure the doorbell write to the device has finished before upda= ting - * the shadow tail index in normal memory. 'fence o, w' + * 4. Make sure the doorbell write to the device has finished before + * updating the shadow tail index in normal memory. 'fence o, w' */ #ifdef CONFIG_MMIOWB mmiowb(); #endif - atomic_inc(&queue->tail); + atomic_set(&queue->tail, prod + 1); + atomic_set(&queue->prod, prod + 1); =20 - /* 7. Complete submission and restore local interrupts */ - local_irq_restore(flags); + raw_spin_unlock_irqrestore(&queue->lock, flags); =20 return prod; =20 err_busy: - local_irq_restore(flags); + raw_spin_unlock_irqrestore(&queue->lock, flags); + /* Report the failure and continue; full RAS recovery is not implemented.= */ dev_err_once(queue->iommu->dev, "Hardware error: command enqueue failed\n= "); - return prod; } =20 --=20 2.50.1 From nobody Mon Sep 28 21:08:22 2026 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE5A5420471 for ; Mon, 17 Aug 2026 14:28:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786976913; cv=none; b=s2rq0ghSIlDm87o+1UlSTJ9917FAUZzqZCk/ZTifFLzY7+nn3gpe6TpTeKDDGYhvohjHTZYOFwj/nA6agpPi5/Ac+MBEiQi2zY7N/BhYZJt3xF+OJssKbDDNbkxpeV4EsIDQV9NRgMv8g009oS5eC5aq1cCWzl6tGQQLFS2YLvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786976913; c=relaxed/simple; bh=drFqM0UHHD3hILUHh08HSPHe3MqcsQM5z31wtT/ohlE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RWjOmMKv0wqc+0BGP26MueLkFsz7dnLtGGmHqbElKbIUb8X/ZfZj70aOWrfOp7RFafEe/3dQWR/w5wxDB/jqBUrffdMOAdVgZBKuhZ2u/M8weXpGwDCm01dendC7+Phh1COEaumUoN48Vrid5gI/66HbChXb6orrWA/2ULbW4Bg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=esvbCK+5; arc=none smtp.client-ip=115.124.30.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="esvbCK+5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1786976907; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=elSQcfZeblurM3/OL+mlSXqqGCSbGGx49NKsTqTo6v4=; b=esvbCK+5p3mzHXgt3lsXQMcXtO7fjCX1fZjAVWqub6FiIoqyVqsi6bGIAEBknXg5zdX/w32Sz78WJ/CEZ1QU9N0O3EA1eV2bswJPdbsaSGbOOKRYlzg4MWOZvjS3W0f3/xxmZUR7nprv1NVyF4hYu3h7qGDbO2yb2xr4XpOy1n0= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045098064;MF=fangyu.yu@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0X9A9H.G_1786976904; Received: from localhost.localdomain(mailfrom:fangyu.yu@linux.alibaba.com fp:SMTPD_---0X9A9H.G_1786976904 cluster:ay36) by smtp.aliyun-inc.com; Mon, 17 Aug 2026 22:28:25 +0800 From: fangyu.yu@linux.alibaba.com To: tomasz.jeznach@linux.dev, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, baolu.lu@linux.intel.com, jroedel@suse.de, zong.li@sifive.com Cc: fangyu.yu@linux.alibaba.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [PATCH 3/3] iommu/riscv: Avoid waiting on failed command enqueue Date: Mon, 17 Aug 2026 22:27:28 +0800 Message-Id: <20260817142728.61783-4-fangyu.yu@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260817142728.61783-1-fangyu.yu@linux.alibaba.com> References: <20260817142728.61783-1-fangyu.yu@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fangyu Yu Do not wait for IOFENCE.C completion when the command failed to enter the queue. The command was not published to hardware, so waiting for its producer index can only report a misleading execution timeout. Fixes: 856c0cfe5c5f ("iommu/riscv: Command and fault queue support") Signed-off-by: Fangyu Yu --- drivers/iommu/riscv/iommu.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 86cec408be5b..2b3dc63536d5 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -419,8 +419,9 @@ static int riscv_iommu_queue_wait_for_space(struct risc= v_iommu_queue *queue, } =20 /* Enqueue an entry and publish it to the hardware queue. */ -static unsigned int riscv_iommu_queue_send(struct riscv_iommu_queue *queue, - void *entry, size_t entry_size) +static int riscv_iommu_queue_send(struct riscv_iommu_queue *queue, + void *entry, size_t entry_size, + unsigned int *out_prod) { unsigned int prod; unsigned int head; @@ -465,15 +466,17 @@ static unsigned int riscv_iommu_queue_send(struct ris= cv_iommu_queue *queue, atomic_set(&queue->tail, prod + 1); atomic_set(&queue->prod, prod + 1); =20 - raw_spin_unlock_irqrestore(&queue->lock, flags); + if (out_prod) + *out_prod =3D prod; =20 - return prod; + raw_spin_unlock_irqrestore(&queue->lock, flags); + return 0; =20 err_busy: raw_spin_unlock_irqrestore(&queue->lock, flags); /* Report the failure and continue; full RAS recovery is not implemented.= */ dev_err_once(queue->iommu->dev, "Hardware error: command enqueue failed\n= "); - return prod; + return ret; } =20 /* @@ -512,7 +515,7 @@ static irqreturn_t riscv_iommu_cmdq_process(int irq, vo= id *data) static void riscv_iommu_cmd_send(struct riscv_iommu_device *iommu, struct riscv_iommu_command *cmd) { - riscv_iommu_queue_send(&iommu->cmdq, cmd, sizeof(*cmd)); + riscv_iommu_queue_send(&iommu->cmdq, cmd, sizeof(*cmd), NULL); } =20 /* Send IOFENCE.C command and wait for all scheduled commands to complete.= */ @@ -521,9 +524,12 @@ static void riscv_iommu_cmd_sync(struct riscv_iommu_de= vice *iommu, { struct riscv_iommu_command cmd; unsigned int prod; + int ret; =20 riscv_iommu_cmd_iofence(&cmd); - prod =3D riscv_iommu_queue_send(&iommu->cmdq, &cmd, sizeof(cmd)); + ret =3D riscv_iommu_queue_send(&iommu->cmdq, &cmd, sizeof(cmd), &prod); + if (ret) + return; =20 if (!timeout_us) return; --=20 2.50.1