From nobody Sat Sep 26 16:40:14 2026 Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 778AD3546D0 for ; Tue, 1 Sep 2026 13:39:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788269974; cv=none; b=Cy0LZB8AnIvFqgTpWesuig//XjmxXO6bm1X8bC+2Dm89DUTtWatT9406MjN+u4BJpN4HcUyBK6JkN6WLPf4U/mNWfDqgQobOUvWYmbShSJYetGuedWewmwq7rSRogtV3ONT2qDezGSfwwPp5w2bYu9ezIQNsK25SbXZ+0gg3zAw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788269974; c=relaxed/simple; bh=6ZEVi/aOJyaJhuddMfCe08+BHazf/2XCD9aBzk4QniM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OMJyaGetDLtvA6XjPBZpfHYQOCz1m19x0WPFPRsys1b4k+GiJNe/Oo1nANq8S3kE/pB8HrD/LPGKUJARyansIboiU4y1B6/d9MXtkmQR4btt4cbuu/Wb8WOZFBkIu8Jc7DmaTrwpr9izKbAyvYxNqp0tNbUZbd0eFROEoojzmjw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=jqdzBQnA; arc=none smtp.client-ip=115.124.30.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="jqdzBQnA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1788269967; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=Zhv3H2+2w0gZKGeUibRr/8VGF24nes4VuRka42uR/+4=; b=jqdzBQnAW5zLQ/f07uzqoJ7+3MytExawkSvMuK5bn3tZASmMH3lB0RUh4x1EUyNhKr5ieGSKysdSjBh6W3L0Zk6R5KsP+4lu3fj/j/8vUT7ikf76NZtn7KGMPY/kuHWUsm64gKYVRIovtEd6Rem8O282EAKlLvzTleW04c7IQo0= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045133197;MF=fangyu.yu@linux.alibaba.com;NM=1;PH=DS;RN=20;SR=0;TI=SMTPD_---0XA9NaYb_1788269964; Received: from localhost.localdomain(mailfrom:fangyu.yu@linux.alibaba.com fp:SMTPD_---0XA9NaYb_1788269964 cluster:ay36) by smtp.aliyun-inc.com; Tue, 01 Sep 2026 21:39:25 +0800 From: fangyu.yu@linux.alibaba.com To: tomasz.jeznach@linux.dev, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, baolu.lu@linux.intel.com, jroedel@suse.de, zong.li@sifive.com, andrew.jones@oss.qualcomm.com, anup@brainfault.org, jgg@nvidia.com, jgg@ziepe.ca Cc: fangyu.yu@linux.alibaba.com, guoren@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [PATCH v2 1/3] iommu/riscv: Add command queue lock Date: Tue, 1 Sep 2026 21:39:18 +0800 Message-Id: <20260901133920.14550-2-fangyu.yu@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260901133920.14550-1-fangyu.yu@linux.alibaba.com> References: <20260901133920.14550-1-fangyu.yu@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fangyu Yu Add a raw spinlock to the RISC-V IOMMU queue state so command queue publishing can be serialized by a later change. Fixes: 856c0cfe5c5f ("iommu/riscv: Command and fault queue support") Signed-off-by: Fangyu Yu --- drivers/iommu/riscv/iommu.c | 1 + drivers/iommu/riscv/iommu.h | 2 ++ 2 files changed, 3 insertions(+) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index cec3ddd7ab10..2c0dcc90cf85 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1560,6 +1560,7 @@ int riscv_iommu_init(struct riscv_iommu_device *iommu) int rc; =20 RISCV_IOMMU_QUEUE_INIT(&iommu->cmdq, CQ); + raw_spin_lock_init(&iommu->cmdq.lock); RISCV_IOMMU_QUEUE_INIT(&iommu->fltq, FQ); =20 rc =3D riscv_iommu_init_check(iommu); diff --git a/drivers/iommu/riscv/iommu.h b/drivers/iommu/riscv/iommu.h index 46df79dd5495..5676001548cc 100644 --- a/drivers/iommu/riscv/iommu.h +++ b/drivers/iommu/riscv/iommu.h @@ -12,6 +12,7 @@ #define _RISCV_IOMMU_H_ =20 #include +#include #include #include =20 @@ -23,6 +24,7 @@ struct riscv_iommu_queue { atomic_t prod; /* unbounded producer allocation index */ atomic_t head; /* unbounded shadow ring buffer consumer index */ atomic_t tail; /* unbounded shadow ring buffer producer index */ + raw_spinlock_t lock; /* serialize queue publishing */ unsigned int mask; /* index mask, queue length - 1 */ unsigned int irq; /* allocated interrupt number */ struct riscv_iommu_device *iommu; /* iommu device handling the queue when= active */ --=20 2.50.1 From nobody Sat Sep 26 16:40:14 2026 Received: from out30-111.freemail.mail.aliyun.com (out30-111.freemail.mail.aliyun.com [115.124.30.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC67646D2AF for ; Tue, 1 Sep 2026 13:39:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.111 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788269978; cv=none; b=oQlITikgDGvJpuBed58lOGR8Y12T3ZdOReBounwz1EoUTVwV4kq0nqa6TrERuTO9kvEfKdEJpANkwnQH/m64k4icXilngRQTDt4vgBo2OwSOv8QdOqRiXiyYK4FoLc4xSjSShSr1NSuWGW2LtcRgVkjqKEk8VsEX+mUEUCyampE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788269978; c=relaxed/simple; bh=FuvdXz5e2hxRMllpzuLnqvrHUalGtCL2az/Qy0Ugh0c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cskaffUBwrffA798W3pkNQhql8ECaeGGojrVIveXUUkMbK0uxJ0rZ2ImTixiQt8MGGEo5SruGywAoTjO9DIhC7zw6kPPWqUc4nzJGsGVYn+ShdtB0VoELbT0oasHdpHVOcHnn00oHHkOOmZ8iNrtQG3ju6P1Qd/y5o0oFQk02Dk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=nthgCvR7; arc=none smtp.client-ip=115.124.30.111 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="nthgCvR7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1788269968; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=zyLF35runNtHobgU5o7kJ9boPye3AhCPnim8elazGm0=; b=nthgCvR7YgvZiqI5Kp+eDC4TP6rXpDzddgd1CDbFnWQ3Y70+gCB5Z2hD2VWnx1d8r5sIJFzKmwkQCEHtjArrz35/ZMVVsESZapbyPj94kcZ80VwVhAW13+RScdlbwxukFPESkW145NNn5aPezTVNvrivcg21ISIJyqOE5I74UwI= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045098064;MF=fangyu.yu@linux.alibaba.com;NM=1;PH=DS;RN=20;SR=0;TI=SMTPD_---0XA9NaZ8_1788269965; Received: from localhost.localdomain(mailfrom:fangyu.yu@linux.alibaba.com fp:SMTPD_---0XA9NaZ8_1788269965 cluster:ay36) by smtp.aliyun-inc.com; Tue, 01 Sep 2026 21:39:26 +0800 From: fangyu.yu@linux.alibaba.com To: tomasz.jeznach@linux.dev, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, baolu.lu@linux.intel.com, jroedel@suse.de, zong.li@sifive.com, andrew.jones@oss.qualcomm.com, anup@brainfault.org, jgg@nvidia.com, jgg@ziepe.ca Cc: fangyu.yu@linux.alibaba.com, guoren@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [PATCH v2 2/3] iommu/riscv: Serialize command queue publishing Date: Tue, 1 Sep 2026 21:39:19 +0800 Message-Id: <20260901133920.14550-3-fangyu.yu@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260901133920.14550-1-fangyu.yu@linux.alibaba.com> References: <20260901133920.14550-1-fangyu.yu@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fangyu Yu Serialize command queue publishing so software producer state advances only after a command is written and the hardware tail is updated. Wait for hardware consumption outside the queue lock when the command queue is full so other CPUs are not blocked behind a long poll. Fixes: 856c0cfe5c5f ("iommu/riscv: Command and fault queue support") Signed-off-by: Fangyu Yu --- drivers/iommu/riscv/iommu.c | 100 +++++++++++++++++++++--------------- 1 file changed, 58 insertions(+), 42 deletions(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 2c0dcc90cf85..e335beb70e42 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -382,77 +382,93 @@ static int riscv_iommu_queue_wait(struct riscv_iommu_= queue *queue, (int)(cons - index) > 0, 0, timeout_us); } =20 -/* Enqueue an entry and wait to be processed if timeout_us > 0 - * - * Error handling for IOMMU hardware not responding in reasonable time - * will be added as separate patch series along with other RAS features. - * For now, only report hardware failure and continue. - */ +static int riscv_iommu_queue_wait_for_space(struct riscv_iommu_queue *queu= e, + unsigned int last) +{ + unsigned int head; + unsigned int tail; + unsigned int hw_head; + unsigned long flags; + int ret; + + ret =3D riscv_iommu_readl_timeout(queue->iommu, Q_HEAD(queue), hw_head, + !(hw_head & ~queue->mask) && hw_head !=3D last, + 0, RISCV_IOMMU_QUEUE_TIMEOUT); + if (ret) + return ret; + + raw_spin_lock_irqsave(&queue->lock, flags); + head =3D atomic_read(&queue->head); + tail =3D atomic_read(&queue->tail); + if ((tail - head) >=3D queue->mask) { + last =3D Q_ITEM(queue, head); + /* + * Re-read hw_head under the lock so that it is consistent with + * the freshly computed 'last'. Using the pre-lock snapshot + * could produce a stale value that wraps around relative to the + * new 'last', advancing the shadow head past entries that have + * not yet been consumed by the hardware. + */ + hw_head =3D riscv_iommu_readl(queue->iommu, Q_HEAD(queue)); + if (!(hw_head & ~queue->mask) && hw_head !=3D last) + atomic_add((hw_head - last) & queue->mask, &queue->head); + } + raw_spin_unlock_irqrestore(&queue->lock, flags); + + return 0; +} + +/* Enqueue an entry and publish it to the hardware queue. */ static unsigned int riscv_iommu_queue_send(struct riscv_iommu_queue *queue, void *entry, size_t entry_size) { unsigned int prod; unsigned int head; - unsigned int tail; unsigned long flags; + int ret; =20 - /* Do not preempt submission flow. */ - local_irq_save(flags); + /* 1. Wait for space availability and reserve the next slot. */ + for (;;) { + raw_spin_lock_irqsave(&queue->lock, flags); =20 - /* 1. Allocate some space in the queue */ - prod =3D atomic_inc_return(&queue->prod) - 1; - head =3D atomic_read(&queue->head); + prod =3D atomic_read(&queue->tail); + head =3D atomic_read(&queue->head); =20 - /* 2. Wait for space availability. */ - if ((prod - head) > queue->mask) { - if (readx_poll_timeout(atomic_read, &queue->head, - head, (prod - head) < queue->mask, - 0, RISCV_IOMMU_QUEUE_TIMEOUT)) - goto err_busy; - } else if ((prod - head) =3D=3D queue->mask) { - const unsigned int last =3D Q_ITEM(queue, head); + if ((prod - head) < queue->mask) + break; + + head =3D Q_ITEM(queue, head); + raw_spin_unlock_irqrestore(&queue->lock, flags); =20 - if (riscv_iommu_readl_timeout(queue->iommu, Q_HEAD(queue), head, - !(head & ~queue->mask) && head !=3D last, - 0, RISCV_IOMMU_QUEUE_TIMEOUT)) + ret =3D riscv_iommu_queue_wait_for_space(queue, head); + if (ret) goto err_busy; - atomic_add((head - last) & queue->mask, &queue->head); } =20 - /* 3. Store entry in the ring buffer */ + /* 2. Store entry in the ring buffer. */ memcpy(queue->base + Q_ITEM(queue, prod) * entry_size, entry, entry_size); =20 - /* 4. Wait for all previous entries to be ready */ - if (readx_poll_timeout(atomic_read, &queue->tail, tail, prod =3D=3D tail, - 0, RISCV_IOMMU_QUEUE_TIMEOUT)) - goto err_busy; - - /* - * 5. Make sure the ring buffer update (whether in normal or I/O memory) = is - * completed and visible before signaling the tail doorbell to fetch - * the next command. 'fence ow, ow' - */ + /* 3. Make sure the entry is visible before updating the queue tail. */ dma_wmb(); riscv_iommu_writel(queue->iommu, Q_TAIL(queue), Q_ITEM(queue, prod + 1)); =20 /* - * 6. Make sure the doorbell write to the device has finished before upda= ting - * the shadow tail index in normal memory. 'fence o, w' + * 4. Make sure the doorbell write to the device has finished before + * updating the shadow tail index in normal memory. 'fence o, w' */ #ifdef CONFIG_MMIOWB mmiowb(); #endif - atomic_inc(&queue->tail); + atomic_set(&queue->tail, prod + 1); + atomic_set(&queue->prod, prod + 1); =20 - /* 7. Complete submission and restore local interrupts */ - local_irq_restore(flags); + raw_spin_unlock_irqrestore(&queue->lock, flags); =20 return prod; =20 err_busy: - local_irq_restore(flags); + /* Report the failure and continue; full RAS recovery is not implemented.= */ dev_err_once(queue->iommu->dev, "Hardware error: command enqueue failed\n= "); - return prod; } =20 --=20 2.50.1 From nobody Sat Sep 26 16:40:14 2026 Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDE0C371053 for ; Tue, 1 Sep 2026 13:39:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.119 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788269974; cv=none; b=g8rMZFoueyqvJZtBxxEtVHt1r8+RmIeqz8/v2SL12gtZhHtrkTycd0hI+tOtjAppdwJtxINm12j6/YtdLRvBiDLfMcGMWgbu4oncRk7Z/vtMCL3tShrRISyGuqp2jqLd74rK2jcHFub1696+MarAdNoKatRh1/0s1BD1idhNmBA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788269974; c=relaxed/simple; bh=ai20KLKiOYNguIr/aic80m0d2gkI0atucc+KfaGJgrc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VEeUtIU9qW1WvvpSv8fNlA79GY4rAAF9o+Q3w/AWuxMZnHdOYqwiudNYwCxB03NEYdod03gje14mAHBxsWV8D3ADV3fLjASzfzTowT09nd5CGz1lKeMnb/Mme2e7tv2X6WLVyqlYBvWMN6wxsciDRGBUUDpDGsZ1xN56WZ+9HME= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=Qo4vsBwu; arc=none smtp.client-ip=115.124.30.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="Qo4vsBwu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1788269970; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=XIEfra7Gk5GpAPhNqAO8Wx/0NjuLdwOdToBsoQSy4QA=; b=Qo4vsBwupYtn5L2kXcej+Wk2ptvHE/mdXOEtJsUrxinjdydqvRyGCv21ErBtnqMHVg8C6bGPnuJ12Af+ZfYc6OguhF6tbAbacUthBmZaaUW3lYeL65Ghqg6gCDo9ppmyjv4qKOJCuqjMYky5kv7ZTUQuhcHA0qQjP2pROyAzvtk= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045133197;MF=fangyu.yu@linux.alibaba.com;NM=1;PH=DS;RN=20;SR=0;TI=SMTPD_---0XA9NaZW_1788269967; Received: from localhost.localdomain(mailfrom:fangyu.yu@linux.alibaba.com fp:SMTPD_---0XA9NaZW_1788269967 cluster:ay36) by smtp.aliyun-inc.com; Tue, 01 Sep 2026 21:39:28 +0800 From: fangyu.yu@linux.alibaba.com To: tomasz.jeznach@linux.dev, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, baolu.lu@linux.intel.com, jroedel@suse.de, zong.li@sifive.com, andrew.jones@oss.qualcomm.com, anup@brainfault.org, jgg@nvidia.com, jgg@ziepe.ca Cc: fangyu.yu@linux.alibaba.com, guoren@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [PATCH v2 3/3] iommu/riscv: Avoid waiting on failed command enqueue Date: Tue, 1 Sep 2026 21:39:20 +0800 Message-Id: <20260901133920.14550-4-fangyu.yu@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260901133920.14550-1-fangyu.yu@linux.alibaba.com> References: <20260901133920.14550-1-fangyu.yu@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fangyu Yu Do not wait for IOFENCE.C completion when the command failed to enter the queue. The command was not published to hardware, so waiting for its producer index can only report a misleading execution timeout. Fixes: 856c0cfe5c5f ("iommu/riscv: Command and fault queue support") Signed-off-by: Fangyu Yu --- drivers/iommu/riscv/iommu.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index e335beb70e42..fe8e6d0f8a23 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -419,8 +419,9 @@ static int riscv_iommu_queue_wait_for_space(struct risc= v_iommu_queue *queue, } =20 /* Enqueue an entry and publish it to the hardware queue. */ -static unsigned int riscv_iommu_queue_send(struct riscv_iommu_queue *queue, - void *entry, size_t entry_size) +static int riscv_iommu_queue_send(struct riscv_iommu_queue *queue, + void *entry, size_t entry_size, + unsigned int *out_prod) { unsigned int prod; unsigned int head; @@ -462,14 +463,16 @@ static unsigned int riscv_iommu_queue_send(struct ris= cv_iommu_queue *queue, atomic_set(&queue->tail, prod + 1); atomic_set(&queue->prod, prod + 1); =20 - raw_spin_unlock_irqrestore(&queue->lock, flags); + if (out_prod) + *out_prod =3D prod; =20 - return prod; + raw_spin_unlock_irqrestore(&queue->lock, flags); + return 0; =20 err_busy: /* Report the failure and continue; full RAS recovery is not implemented.= */ dev_err_once(queue->iommu->dev, "Hardware error: command enqueue failed\n= "); - return prod; + return ret; } =20 /* @@ -508,7 +511,7 @@ static irqreturn_t riscv_iommu_cmdq_process(int irq, vo= id *data) static void riscv_iommu_cmd_send(struct riscv_iommu_device *iommu, struct riscv_iommu_command *cmd) { - riscv_iommu_queue_send(&iommu->cmdq, cmd, sizeof(*cmd)); + riscv_iommu_queue_send(&iommu->cmdq, cmd, sizeof(*cmd), NULL); } =20 /* Send IOFENCE.C command and wait for all scheduled commands to complete.= */ @@ -517,9 +520,12 @@ static void riscv_iommu_cmd_sync(struct riscv_iommu_de= vice *iommu, { struct riscv_iommu_command cmd; unsigned int prod; + int ret; =20 riscv_iommu_cmd_iofence(&cmd); - prod =3D riscv_iommu_queue_send(&iommu->cmdq, &cmd, sizeof(cmd)); + ret =3D riscv_iommu_queue_send(&iommu->cmdq, &cmd, sizeof(cmd), &prod); + if (ret) + return; =20 if (!timeout_us) return; --=20 2.50.1