From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout08.his.huawei.com (canpmsgout08.his.huawei.com [113.46.200.223]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3EC617E4; Sat, 22 Nov 2025 07:49:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.223 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797764; cv=none; b=c+t+lVqsONL51IXjXj2EBPBBNBcz5sQETowAW74L+fnDLBMwsbj3jM209BirKodtHR/RO22oVtaH2lNFPUfQx3C6mM3VlmEJpTLVKSCbky1gPRnD+eP5uIXD2ensuB9quYZQQIJMclTyYQZnW/TrP9pf59MAK6OxILQ7IEaV9tw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797764; c=relaxed/simple; bh=W5bdK9LMLKNosAGIK8uSbxEQ1gvRjelbXW1hrGlC//I=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tgmgwwNfcA9V72l99x0wrlEefCT/sP5wZb1C9Y/9iUOm6giZIEjq2egYbs4xYOmJvfSIHcDxDbzLaPdUbDefW6BFqb+x+d5bsMrroVM8B+v7d1eV0cLf8b6CQW4LEeKHcv56rvCopKBV3ADm15aA1+H2ALANQRFKlWqXdDlUuOg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=MEI0p+N0; arc=none smtp.client-ip=113.46.200.223 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="MEI0p+N0" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=bdfIvkbhbhQHaorwRUFkOOSMlOD4Ol0bssvKbV67JkE=; b=MEI0p+N0wKv0YLtqMBVidlecEl0C7HmO1Dzj922FM8/6Ie/12ButDx91AMqBaovkGWuaSFpOl D3mPMf74CrSGqTkSnBAWRmDJifYLmauytsMVfeCXkfmmoZTQWIWWnjiTKZwMV5DGGz8RlfYa/KU X3TA4CQHqT8uroO8ntMPCgE= Received: from mail.maildlp.com (unknown [172.19.162.112]) by canpmsgout08.his.huawei.com (SkyGuard) with ESMTPS id 4dD41c3DZyzmVYC; Sat, 22 Nov 2025 15:47:32 +0800 (CST) Received: from dggemv706-chm.china.huawei.com (unknown [10.3.19.33]) by mail.maildlp.com (Postfix) with ESMTPS id 210FC140203; Sat, 22 Nov 2025 15:49:18 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv706-chm.china.huawei.com (10.3.19.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:17 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:17 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 01/11] crypto: hisilicon/zip - adjust the way to obtain the req in the callback function Date: Sat, 22 Nov 2025 15:49:06 +0800 Message-ID: <20251122074916.2793717-2-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" In the shared queue design, multiple tfms use same qp, and one qp need to corresponds to multiple qp_ctx. So use tag to obtain the req virtual address. Build a one-to-one relationship between tfm and qp_ctx. finaly remove the old get_tag operation. Fixes: 2bcf36348ce5 ("crypto: hisilicon/zip - initialize operations about '= sqe' in 'acomp_alg.init'") Signed-off-by: Chenghai Huang Signed-off-by: Weili Qian --- drivers/crypto/hisilicon/zip/zip_crypto.c | 24 +++++++++-------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/drivers/crypto/hisilicon/zip/zip_crypto.c b/drivers/crypto/his= ilicon/zip/zip_crypto.c index b97513981a3b..b4a656e0177d 100644 --- a/drivers/crypto/hisilicon/zip/zip_crypto.c +++ b/drivers/crypto/hisilicon/zip/zip_crypto.c @@ -39,6 +39,7 @@ enum { HZIP_CTX_Q_NUM }; =20 +#define GET_REQ_FROM_SQE(sqe) ((u64)(sqe)->dw26 | (u64)(sqe)->dw27 << 32) #define COMP_NAME_TO_TYPE(alg_name) \ (!strcmp((alg_name), "deflate") ? HZIP_ALG_TYPE_DEFLATE : 0) =20 @@ -48,6 +49,7 @@ struct hisi_zip_req { struct hisi_acc_hw_sgl *hw_dst; dma_addr_t dma_src; dma_addr_t dma_dst; + struct hisi_zip_qp_ctx *qp_ctx; u16 req_id; }; =20 @@ -74,7 +76,6 @@ struct hisi_zip_sqe_ops { void (*fill_req_type)(struct hisi_zip_sqe *sqe, u8 req_type); void (*fill_tag)(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req); void (*fill_sqe_type)(struct hisi_zip_sqe *sqe, u8 sqe_type); - u32 (*get_tag)(struct hisi_zip_sqe *sqe); u32 (*get_status)(struct hisi_zip_sqe *sqe); u32 (*get_dstlen)(struct hisi_zip_sqe *sqe); }; @@ -131,6 +132,7 @@ static struct hisi_zip_req *hisi_zip_create_req(struct = hisi_zip_qp_ctx *qp_ctx, req_cache =3D q + req_id; req_cache->req_id =3D req_id; req_cache->req =3D req; + req_cache->qp_ctx =3D qp_ctx; =20 return req_cache; } @@ -181,7 +183,8 @@ static void hisi_zip_fill_req_type(struct hisi_zip_sqe = *sqe, u8 req_type) =20 static void hisi_zip_fill_tag(struct hisi_zip_sqe *sqe, struct hisi_zip_re= q *req) { - sqe->dw26 =3D req->req_id; + sqe->dw26 =3D lower_32_bits((u64)req); + sqe->dw27 =3D upper_32_bits((u64)req); } =20 static void hisi_zip_fill_sqe_type(struct hisi_zip_sqe *sqe, u8 sqe_type) @@ -237,7 +240,7 @@ static int hisi_zip_do_work(struct hisi_zip_qp_ctx *qp_= ctx, &req->dma_dst, DMA_FROM_DEVICE); if (IS_ERR(req->hw_dst)) { ret =3D PTR_ERR(req->hw_dst); - dev_err(dev, "failed to map the dst buffer to hw slg (%d)!\n", + dev_err(dev, "failed to map the dst buffer to hw sgl (%d)!\n", ret); goto err_unmap_input; } @@ -265,11 +268,6 @@ static int hisi_zip_do_work(struct hisi_zip_qp_ctx *qp= _ctx, return ret; } =20 -static u32 hisi_zip_get_tag(struct hisi_zip_sqe *sqe) -{ - return sqe->dw26; -} - static u32 hisi_zip_get_status(struct hisi_zip_sqe *sqe) { return sqe->dw3 & HZIP_BD_STATUS_M; @@ -282,14 +280,12 @@ static u32 hisi_zip_get_dstlen(struct hisi_zip_sqe *s= qe) =20 static void hisi_zip_acomp_cb(struct hisi_qp *qp, void *data) { - struct hisi_zip_qp_ctx *qp_ctx =3D qp->qp_ctx; + struct hisi_zip_sqe *sqe =3D data; + struct hisi_zip_req *req =3D (struct hisi_zip_req *)GET_REQ_FROM_SQE(sqe); + struct hisi_zip_qp_ctx *qp_ctx =3D req->qp_ctx; const struct hisi_zip_sqe_ops *ops =3D qp_ctx->ctx->ops; struct hisi_zip_dfx *dfx =3D &qp_ctx->zip_dev->dfx; - struct hisi_zip_req_q *req_q =3D &qp_ctx->req_q; struct device *dev =3D &qp->qm->pdev->dev; - struct hisi_zip_sqe *sqe =3D data; - u32 tag =3D ops->get_tag(sqe); - struct hisi_zip_req *req =3D req_q->q + tag; struct acomp_req *acomp_req =3D req->req; int err =3D 0; u32 status; @@ -393,7 +389,6 @@ static const struct hisi_zip_sqe_ops hisi_zip_ops =3D { .fill_req_type =3D hisi_zip_fill_req_type, .fill_tag =3D hisi_zip_fill_tag, .fill_sqe_type =3D hisi_zip_fill_sqe_type, - .get_tag =3D hisi_zip_get_tag, .get_status =3D hisi_zip_get_status, .get_dstlen =3D hisi_zip_get_dstlen, }; @@ -581,7 +576,6 @@ static void hisi_zip_acomp_exit(struct crypto_acomp *tf= m) { struct hisi_zip_ctx *ctx =3D crypto_tfm_ctx(&tfm->base); =20 - hisi_zip_set_acomp_cb(ctx, NULL); hisi_zip_release_sgl_pool(ctx); hisi_zip_release_req_q(ctx); hisi_zip_ctx_exit(ctx); --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout07.his.huawei.com (canpmsgout07.his.huawei.com [113.46.200.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A8492D7388; Sat, 22 Nov 2025 07:49:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.222 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797769; cv=none; b=GBeCVAKTAQqDOM61eYx9cQdobxQ7zzxaewmSwUHO4AHutYx8OVjvP7+a/9Nektt3pd3FHLdscT6DQY34kbibqOvp08oJrefIpx/VJ1R/LIm1lsGxIRJK03PA9s1iad8cOPs4RgJDbFP3SdPRTcmlBZUQhMYfr0Cvm7zp9IA3qQw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797769; c=relaxed/simple; bh=mSye3qnlvOebZSXiqeWKYC+qtSaRFbrcAzQZKFLKuwM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EUMALbvMG3l4HDLXPazc445gIQrlw0MpAOHwvXZe7ft14q94v8IOJtk9KBMMFQ9g52Bzzc78NKWeRoBAJ6lvPHOVbOwtxxrqDPnErxceKZR5miGJVyBD3zrRvYjVrgQoVb0/WZcskaX+3OSBth4UOy1A/LrVIa2Q7Avwgz+UABI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=Ma3hBV4j; arc=none smtp.client-ip=113.46.200.222 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="Ma3hBV4j" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=vsyyawiBG4oO/Nn3Wp36udYzgMco5diEhLn5kz7oQSI=; b=Ma3hBV4jJBkeBLHlXLOOD56oXmDPfzRyct9U+y6lxSdx0GiNEADQqvdeWOUWgp/OsVHrmqRGj tHFcVwRnZ1Z/om9ZvGa3oiY+yWkjnyhSYWVf1z5HMzSQdLphvRxOKOGeTe6/Y0LDrIHi86KgVl5 4kNpO4ZAwC/RnVoTAnVgD8s= Received: from mail.maildlp.com (unknown [172.19.88.214]) by canpmsgout07.his.huawei.com (SkyGuard) with ESMTPS id 4dD41d3RjLzLlSC; Sat, 22 Nov 2025 15:47:33 +0800 (CST) Received: from dggemv712-chm.china.huawei.com (unknown [10.1.198.32]) by mail.maildlp.com (Postfix) with ESMTPS id A578D1A016C; Sat, 22 Nov 2025 15:49:18 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv712-chm.china.huawei.com (10.1.198.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:18 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:17 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 02/11] crypto: hisilicon/sec - move backlog management to qp and store sqe in qp for callback Date: Sat, 22 Nov 2025 15:49:07 +0800 Message-ID: <20251122074916.2793717-3-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" When multiple tfm use a same qp, the backlog data should be managed centrally by the qp, rather than in the qp_ctx of each req. Additionally, since SEC_BD_TYPE1 and SEC_BD_TYPE2 cannot use the tag of the sqe to carry the virtual address of the req, the sent sqe is stored in the qp. This allows the callback function to get the req address. To handle the differences between hardware types, the callback functions are split into two separate implementations. Fixes: f0ae287c5045 ("crypto: hisilicon/sec2 - implement full backlog mode = for sec") Signed-off-by: Chenghai Huang Signed-off-by: Weili Qian --- drivers/crypto/hisilicon/qm.c | 20 ++++- drivers/crypto/hisilicon/sec2/sec.h | 7 -- drivers/crypto/hisilicon/sec2/sec_crypto.c | 88 +++++++++++----------- include/linux/hisi_acc_qm.h | 8 ++ 4 files changed, 69 insertions(+), 54 deletions(-) diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index 56bbb46f1877..4760551d4fa3 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -2219,6 +2219,7 @@ static void qp_stop_fail_cb(struct hisi_qp *qp) for (i =3D 0; i < qp_used; i++) { pos =3D (i + cur_head) % sq_depth; qp->req_cb(qp, qp->sqe + (u32)(qm->sqe_size * pos)); + qm_cq_head_update(qp); atomic_dec(&qp->qp_status.used); } } @@ -2383,6 +2384,7 @@ int hisi_qp_send(struct hisi_qp *qp, const void *msg) return -EBUSY; =20 memcpy(sqe, msg, qp->qm->sqe_size); + qp->msg[sq_tail] =3D msg; =20 qm_db(qp->qm, qp->qp_id, QM_DOORBELL_CMD_SQ, sq_tail_next, 0); atomic_inc(&qp->qp_status.used); @@ -2919,12 +2921,13 @@ EXPORT_SYMBOL_GPL(hisi_qm_wait_task_finish); static void hisi_qp_memory_uninit(struct hisi_qm *qm, int num) { struct device *dev =3D &qm->pdev->dev; - struct qm_dma *qdma; + struct hisi_qp *qp; int i; =20 for (i =3D num - 1; i >=3D 0; i--) { - qdma =3D &qm->qp_array[i].qdma; - dma_free_coherent(dev, qdma->size, qdma->va, qdma->dma); + qp =3D &qm->qp_array[i]; + dma_free_coherent(dev, qp->qdma.size, qp->qdma.va, qp->qdma.dma); + kfree(qp->msg); kfree(qm->poll_data[i].qp_finish_id); } =20 @@ -2946,10 +2949,14 @@ static int hisi_qp_memory_init(struct hisi_qm *qm, = size_t dma_size, int id, return -ENOMEM; =20 qp =3D &qm->qp_array[id]; + qp->msg =3D kmalloc_array(sq_depth, sizeof(void *), GFP_KERNEL); + if (!qp->msg) + goto err_free_qp_finish_id; + qp->qdma.va =3D dma_alloc_coherent(dev, dma_size, &qp->qdma.dma, GFP_KERNEL); if (!qp->qdma.va) - goto err_free_qp_finish_id; + goto err_free_qp_msg; =20 qp->sqe =3D qp->qdma.va; qp->sqe_dma =3D qp->qdma.dma; @@ -2961,8 +2968,13 @@ static int hisi_qp_memory_init(struct hisi_qm *qm, s= ize_t dma_size, int id, qp->qm =3D qm; qp->qp_id =3D id; =20 + spin_lock_init(&qp->backlog.lock); + INIT_LIST_HEAD(&qp->backlog.list); + return 0; =20 +err_free_qp_msg: + kfree(qp->msg); err_free_qp_finish_id: kfree(qm->poll_data[id].qp_finish_id); return ret; diff --git a/drivers/crypto/hisilicon/sec2/sec.h b/drivers/crypto/hisilicon= /sec2/sec.h index 81d0beda93b2..0710977861f3 100644 --- a/drivers/crypto/hisilicon/sec2/sec.h +++ b/drivers/crypto/hisilicon/sec2/sec.h @@ -82,11 +82,6 @@ struct sec_aead_req { __u8 out_mac_buf[SEC_MAX_MAC_LEN]; }; =20 -struct sec_instance_backlog { - struct list_head list; - spinlock_t lock; -}; - /* SEC request of Crypto */ struct sec_req { union { @@ -112,7 +107,6 @@ struct sec_req { bool use_pbuf; =20 struct list_head list; - struct sec_instance_backlog *backlog; struct sec_request_buf buf; }; =20 @@ -172,7 +166,6 @@ struct sec_qp_ctx { spinlock_t id_lock; struct hisi_acc_sgl_pool *c_in_pool; struct hisi_acc_sgl_pool *c_out_pool; - struct sec_instance_backlog backlog; u16 send_head; }; =20 diff --git a/drivers/crypto/hisilicon/sec2/sec_crypto.c b/drivers/crypto/hi= silicon/sec2/sec_crypto.c index 31590d01139a..4e41235116e1 100644 --- a/drivers/crypto/hisilicon/sec2/sec_crypto.c +++ b/drivers/crypto/hisilicon/sec2/sec_crypto.c @@ -54,7 +54,6 @@ #define SEC_AUTH_CIPHER_V3 0x40 #define SEC_FLAG_OFFSET 7 #define SEC_FLAG_MASK 0x0780 -#define SEC_TYPE_MASK 0x0F #define SEC_DONE_MASK 0x0001 #define SEC_ICV_MASK 0x000E =20 @@ -148,7 +147,7 @@ static void sec_free_req_id(struct sec_req *req) spin_unlock_bh(&qp_ctx->id_lock); } =20 -static u8 pre_parse_finished_bd(struct bd_status *status, void *resp) +static void pre_parse_finished_bd(struct bd_status *status, void *resp) { struct sec_sqe *bd =3D resp; =20 @@ -158,11 +157,9 @@ static u8 pre_parse_finished_bd(struct bd_status *stat= us, void *resp) SEC_FLAG_MASK) >> SEC_FLAG_OFFSET; status->tag =3D le16_to_cpu(bd->type2.tag); status->err_type =3D bd->type2.error_type; - - return bd->type_cipher_auth & SEC_TYPE_MASK; } =20 -static u8 pre_parse_finished_bd3(struct bd_status *status, void *resp) +static void pre_parse_finished_bd3(struct bd_status *status, void *resp) { struct sec_sqe3 *bd3 =3D resp; =20 @@ -172,8 +169,6 @@ static u8 pre_parse_finished_bd3(struct bd_status *stat= us, void *resp) SEC_FLAG_MASK) >> SEC_FLAG_OFFSET; status->tag =3D le64_to_cpu(bd3->tag); status->err_type =3D bd3->error_type; - - return le32_to_cpu(bd3->bd_param) & SEC_TYPE_MASK; } =20 static int sec_cb_status_check(struct sec_req *req, @@ -244,7 +239,7 @@ static void sec_alg_send_backlog_soft(struct sec_ctx *c= tx, struct sec_qp_ctx *qp struct sec_req *req, *tmp; int ret; =20 - list_for_each_entry_safe(req, tmp, &qp_ctx->backlog.list, list) { + list_for_each_entry_safe(req, tmp, &qp_ctx->qp->backlog.list, list) { list_del(&req->list); ctx->req_op->buf_unmap(ctx, req); if (req->req_id >=3D 0) @@ -265,11 +260,12 @@ static void sec_alg_send_backlog_soft(struct sec_ctx = *ctx, struct sec_qp_ctx *qp =20 static void sec_alg_send_backlog(struct sec_ctx *ctx, struct sec_qp_ctx *q= p_ctx) { + struct hisi_qp *qp =3D qp_ctx->qp; struct sec_req *req, *tmp; int ret; =20 - spin_lock_bh(&qp_ctx->backlog.lock); - list_for_each_entry_safe(req, tmp, &qp_ctx->backlog.list, list) { + spin_lock_bh(&qp->backlog.lock); + list_for_each_entry_safe(req, tmp, &qp->backlog.list, list) { ret =3D qp_send_message(req); switch (ret) { case -EINPROGRESS: @@ -287,42 +283,46 @@ static void sec_alg_send_backlog(struct sec_ctx *ctx,= struct sec_qp_ctx *qp_ctx) } =20 unlock: - spin_unlock_bh(&qp_ctx->backlog.lock); + spin_unlock_bh(&qp->backlog.lock); } =20 static void sec_req_cb(struct hisi_qp *qp, void *resp) { - struct sec_qp_ctx *qp_ctx =3D qp->qp_ctx; - struct sec_dfx *dfx =3D &qp_ctx->ctx->sec->debug.dfx; - u8 type_supported =3D qp_ctx->ctx->type_supported; + const struct sec_sqe *sqe =3D qp->msg[qp->qp_status.cq_head]; + struct sec_req *req =3D container_of(sqe, struct sec_req, sec_sqe); + struct sec_ctx *ctx =3D req->ctx; + struct sec_dfx *dfx =3D &ctx->sec->debug.dfx; struct bd_status status; - struct sec_ctx *ctx; - struct sec_req *req; int err; - u8 type; =20 - if (type_supported =3D=3D SEC_BD_TYPE2) { - type =3D pre_parse_finished_bd(&status, resp); - req =3D qp_ctx->req_list[status.tag]; - } else { - type =3D pre_parse_finished_bd3(&status, resp); - req =3D (void *)(uintptr_t)status.tag; - } + pre_parse_finished_bd(&status, resp); =20 - if (unlikely(type !=3D type_supported)) { - atomic64_inc(&dfx->err_bd_cnt); - pr_err("err bd type [%u]\n", type); - return; - } + req->err_type =3D status.err_type; + err =3D sec_cb_status_check(req, &status); + if (err) + atomic64_inc(&dfx->done_flag_cnt); =20 - if (unlikely(!req)) { - atomic64_inc(&dfx->invalid_req_cnt); - atomic_inc(&qp->qp_status.used); - return; - } + atomic64_inc(&dfx->recv_cnt); =20 + ctx->req_op->buf_unmap(ctx, req); + ctx->req_op->callback(ctx, req, err); +} + +static void sec_req_cb3(struct hisi_qp *qp, void *resp) +{ + struct bd_status status; + struct sec_ctx *ctx; + struct sec_dfx *dfx; + struct sec_req *req; + int err; + + pre_parse_finished_bd3(&status, resp); + + req =3D (void *)(uintptr_t)status.tag; req->err_type =3D status.err_type; ctx =3D req->ctx; + dfx =3D &ctx->sec->debug.dfx; + err =3D sec_cb_status_check(req, &status); if (err) atomic64_inc(&dfx->done_flag_cnt); @@ -330,7 +330,6 @@ static void sec_req_cb(struct hisi_qp *qp, void *resp) atomic64_inc(&dfx->recv_cnt); =20 ctx->req_op->buf_unmap(ctx, req); - ctx->req_op->callback(ctx, req, err); } =20 @@ -348,8 +347,10 @@ static int sec_alg_send_message_retry(struct sec_req *= req) =20 static int sec_alg_try_enqueue(struct sec_req *req) { + struct hisi_qp *qp =3D req->qp_ctx->qp; + /* Check if any request is already backlogged */ - if (!list_empty(&req->backlog->list)) + if (!list_empty(&qp->backlog.list)) return -EBUSY; =20 /* Try to enqueue to HW ring */ @@ -359,17 +360,18 @@ static int sec_alg_try_enqueue(struct sec_req *req) =20 static int sec_alg_send_message_maybacklog(struct sec_req *req) { + struct hisi_qp *qp =3D req->qp_ctx->qp; int ret; =20 ret =3D sec_alg_try_enqueue(req); if (ret !=3D -EBUSY) return ret; =20 - spin_lock_bh(&req->backlog->lock); + spin_lock_bh(&qp->backlog.lock); ret =3D sec_alg_try_enqueue(req); if (ret =3D=3D -EBUSY) - list_add_tail(&req->list, &req->backlog->list); - spin_unlock_bh(&req->backlog->lock); + list_add_tail(&req->list, &qp->backlog.list); + spin_unlock_bh(&qp->backlog.lock); =20 return ret; } @@ -629,13 +631,14 @@ static int sec_create_qp_ctx(struct sec_ctx *ctx, int= qp_ctx_id) qp_ctx->qp =3D qp; qp_ctx->ctx =3D ctx; =20 - qp->req_cb =3D sec_req_cb; + if (ctx->type_supported =3D=3D SEC_BD_TYPE3) + qp->req_cb =3D sec_req_cb3; + else + qp->req_cb =3D sec_req_cb; =20 spin_lock_init(&qp_ctx->req_lock); idr_init(&qp_ctx->req_idr); - spin_lock_init(&qp_ctx->backlog.lock); spin_lock_init(&qp_ctx->id_lock); - INIT_LIST_HEAD(&qp_ctx->backlog.list); qp_ctx->send_head =3D 0; =20 ret =3D sec_alloc_qp_ctx_resource(ctx, qp_ctx); @@ -1952,7 +1955,6 @@ static int sec_request_init(struct sec_ctx *ctx, stru= ct sec_req *req) } while (req->req_id < 0 && ++i < ctx->sec->ctx_q_num); =20 req->qp_ctx =3D qp_ctx; - req->backlog =3D &qp_ctx->backlog; =20 return 0; } diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h index c4690e365ade..2d0cc61ed886 100644 --- a/include/linux/hisi_acc_qm.h +++ b/include/linux/hisi_acc_qm.h @@ -444,6 +444,11 @@ struct hisi_qp_ops { int (*fill_sqe)(void *sqe, void *q_parm, void *d_parm); }; =20 +struct instance_backlog { + struct list_head list; + spinlock_t lock; +}; + struct hisi_qp { u32 qp_id; u16 sq_depth; @@ -468,6 +473,9 @@ struct hisi_qp { bool is_in_kernel; u16 pasid; struct uacce_queue *uacce_q; + + struct instance_backlog backlog; + const void **msg; }; =20 static inline int vfs_num_set(const char *val, const struct kernel_param *= kp) --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout05.his.huawei.com (canpmsgout05.his.huawei.com [113.46.200.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A6B72D9493; Sat, 22 Nov 2025 07:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.220 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797769; cv=none; b=dOOnVzREMxe3MFFdQlDKSfE+HXy638lmNQNFufW549NFSDRLuCmjyJhB8fF7n9euskr1GnRUzbv2xB8KzJZoNUpoZCNm81o9ut1qSH2Kug/5jzL11zWo04Z6OAcjwnohvahLvlb0rifofh7NEgBGfCIVM7AkF6lCWSJz+YJGctM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797769; c=relaxed/simple; bh=ZNHCdJaA3afZTO4UAmrHRiw7G5rKI5FGEUItIxMSaww=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=m9Zi0Fqkfef61B1Th1r4qYVui6QIkqAiteNvOMa90mPjqbS+St/uyjtR97qDkrofkPB7Fp4AuX83mIOlrBwlqQtuWQoXNnwJwMiukNwS/zu1FcK0dcp34E3LZuhiEYVTICuwvZKPIXEcT1fdLAtp3Ja+Uwda0QEkQmdM9foCfdw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=cy/wPIol; arc=none smtp.client-ip=113.46.200.220 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="cy/wPIol" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=YAR4Mw5sArcxSPcXElFn5EAVOS+ZDvZEjvDw+TPN/Jo=; b=cy/wPIollasrRDWsRbvXju8fJ1JlEVXVPuvVn1V57VCpbyZMBPxTTxdm4VCu2QsM652TQfrz8 eN0tNyR7E0CWJzEMgTWIGga5dJB9B5agClHxHF8pnMoUCsJiMbBOmChdT5yzO1RL/ifOdDHDSsn ErBHKwLp2zR0dfMGY3F5C00= Received: from mail.maildlp.com (unknown [172.19.163.252]) by canpmsgout05.his.huawei.com (SkyGuard) with ESMTPS id 4dD4210hWZz12LgP; Sat, 22 Nov 2025 15:47:53 +0800 (CST) Received: from dggemv705-chm.china.huawei.com (unknown [10.3.19.32]) by mail.maildlp.com (Postfix) with ESMTPS id 12DAF180B4E; Sat, 22 Nov 2025 15:49:19 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv705-chm.china.huawei.com (10.3.19.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:18 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:18 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 03/11] crypto: hisilicon/hpre: extend tag field to 64 bits for better performance Date: Sat, 22 Nov 2025 15:49:08 +0800 Message-ID: <20251122074916.2793717-4-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" From: lizhi This commit expands the tag field in hpre_sqe structure from 16-bit to 64-bit. The change enables storing request addresses directly in the tag field, allowing callback functions to access request messages without the previous indirection mechanism. By eliminating the need for lookup tables, this modification reduces lock contention and associated overhead, leading to improved efficiency and simplified code. Fixes: c8b4b477079d ("crypto: hisilicon - add HiSilicon HPRE accelerator") Signed-off-by: lizhi Signed-off-by: Weili Qian Signed-off-by: Chenghai Huang --- drivers/crypto/hisilicon/hpre/hpre.h | 5 +- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 142 ++++---------------- 2 files changed, 25 insertions(+), 122 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre.h b/drivers/crypto/hisilico= n/hpre/hpre.h index 0f3ddbadbcf9..021dbd9a1d48 100644 --- a/drivers/crypto/hisilicon/hpre/hpre.h +++ b/drivers/crypto/hisilicon/hpre/hpre.h @@ -94,9 +94,8 @@ struct hpre_sqe { __le64 key; __le64 in; __le64 out; - __le16 tag; - __le16 resv2; -#define _HPRE_SQE_ALIGN_EXT 7 + __le64 tag; +#define _HPRE_SQE_ALIGN_EXT 6 __le32 rsvd1[_HPRE_SQE_ALIGN_EXT]; }; =20 diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/h= isilicon/hpre/hpre_crypto.c index 21ccf879f70c..4197281c8dff 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -108,12 +108,10 @@ struct hpre_ecdh_ctx { struct hpre_ctx { struct hisi_qp *qp; struct device *dev; - struct hpre_asym_request **req_list; struct hpre *hpre; spinlock_t req_lock; unsigned int key_sz; bool crt_g2_mode; - struct idr req_idr; union { struct hpre_rsa_ctx rsa; struct hpre_dh_ctx dh; @@ -136,7 +134,6 @@ struct hpre_asym_request { struct kpp_request *ecdh; } areq; int err; - int req_id; hpre_cb cb; struct timespec64 req_time; }; @@ -151,58 +148,13 @@ static inline unsigned int hpre_align_pd(void) return (hpre_align_sz() - 1) & ~(crypto_tfm_ctx_alignment() - 1); } =20 -static int hpre_alloc_req_id(struct hpre_ctx *ctx) +static void hpre_dfx_add_req_time(struct hpre_asym_request *hpre_req) { - unsigned long flags; - int id; - - spin_lock_irqsave(&ctx->req_lock, flags); - id =3D idr_alloc(&ctx->req_idr, NULL, 0, ctx->qp->sq_depth, GFP_ATOMIC); - spin_unlock_irqrestore(&ctx->req_lock, flags); - - return id; -} - -static void hpre_free_req_id(struct hpre_ctx *ctx, int req_id) -{ - unsigned long flags; - - spin_lock_irqsave(&ctx->req_lock, flags); - idr_remove(&ctx->req_idr, req_id); - spin_unlock_irqrestore(&ctx->req_lock, flags); -} - -static int hpre_add_req_to_ctx(struct hpre_asym_request *hpre_req) -{ - struct hpre_ctx *ctx; - struct hpre_dfx *dfx; - int id; - - ctx =3D hpre_req->ctx; - id =3D hpre_alloc_req_id(ctx); - if (unlikely(id < 0)) - return -EINVAL; - - ctx->req_list[id] =3D hpre_req; - hpre_req->req_id =3D id; + struct hpre_ctx *ctx =3D hpre_req->ctx; + struct hpre_dfx *dfx =3D ctx->hpre->debug.dfx; =20 - dfx =3D ctx->hpre->debug.dfx; if (atomic64_read(&dfx[HPRE_OVERTIME_THRHLD].value)) ktime_get_ts64(&hpre_req->req_time); - - return id; -} - -static void hpre_rm_req_from_ctx(struct hpre_asym_request *hpre_req) -{ - struct hpre_ctx *ctx =3D hpre_req->ctx; - int id =3D hpre_req->req_id; - - if (hpre_req->req_id >=3D 0) { - hpre_req->req_id =3D HPRE_INVLD_REQ_ID; - ctx->req_list[id] =3D NULL; - hpre_free_req_id(ctx, id); - } } =20 static struct hisi_qp *hpre_get_qp_and_start(u8 type) @@ -340,26 +292,19 @@ static void hpre_hw_data_clr_all(struct hpre_ctx *ctx, static int hpre_alg_res_post_hf(struct hpre_ctx *ctx, struct hpre_sqe *sqe, void **kreq) { - struct hpre_asym_request *req; unsigned int err, done, alg; - int id; =20 #define HPRE_NO_HW_ERR 0 #define HPRE_HW_TASK_DONE 3 #define HREE_HW_ERR_MASK GENMASK(10, 0) #define HREE_SQE_DONE_MASK GENMASK(1, 0) #define HREE_ALG_TYPE_MASK GENMASK(4, 0) - id =3D (int)le16_to_cpu(sqe->tag); - req =3D ctx->req_list[id]; - hpre_rm_req_from_ctx(req); - *kreq =3D req; + *kreq =3D (void *)le64_to_cpu(sqe->tag); =20 err =3D (le32_to_cpu(sqe->dw0) >> HPRE_SQE_ALG_BITS) & HREE_HW_ERR_MASK; - done =3D (le32_to_cpu(sqe->dw0) >> HPRE_SQE_DONE_SHIFT) & HREE_SQE_DONE_MASK; - if (likely(err =3D=3D HPRE_NO_HW_ERR && done =3D=3D HPRE_HW_TASK_DONE)) return 0; =20 @@ -370,34 +315,9 @@ static int hpre_alg_res_post_hf(struct hpre_ctx *ctx, = struct hpre_sqe *sqe, return -EINVAL; } =20 -static int hpre_ctx_set(struct hpre_ctx *ctx, struct hisi_qp *qp, int qlen) -{ - struct hpre *hpre; - - if (!ctx || !qp || qlen < 0) - return -EINVAL; - - spin_lock_init(&ctx->req_lock); - ctx->qp =3D qp; - ctx->dev =3D &qp->qm->pdev->dev; - - hpre =3D container_of(ctx->qp->qm, struct hpre, qm); - ctx->hpre =3D hpre; - ctx->req_list =3D kcalloc(qlen, sizeof(void *), GFP_KERNEL); - if (!ctx->req_list) - return -ENOMEM; - ctx->key_sz =3D 0; - ctx->crt_g2_mode =3D false; - idr_init(&ctx->req_idr); - - return 0; -} - static void hpre_ctx_clear(struct hpre_ctx *ctx, bool is_clear_all) { if (is_clear_all) { - idr_destroy(&ctx->req_idr); - kfree(ctx->req_list); hisi_qm_free_qps(&ctx->qp, 1); } =20 @@ -467,29 +387,22 @@ static void hpre_rsa_cb(struct hpre_ctx *ctx, void *r= esp) =20 static void hpre_alg_cb(struct hisi_qp *qp, void *resp) { - struct hpre_ctx *ctx =3D qp->qp_ctx; - struct hpre_dfx *dfx =3D ctx->hpre->debug.dfx; + struct hpre_asym_request *h_req; struct hpre_sqe *sqe =3D resp; - struct hpre_asym_request *req =3D ctx->req_list[le16_to_cpu(sqe->tag)]; =20 - if (unlikely(!req)) { - atomic64_inc(&dfx[HPRE_INVALID_REQ_CNT].value); + h_req =3D (struct hpre_asym_request *)le64_to_cpu(sqe->tag); + if (unlikely(!h_req)) { + pr_err("Failed to get request, and qp_id is %u\n", qp->qp_id); return; } =20 - req->cb(ctx, resp); -} - -static void hpre_stop_qp_and_put(struct hisi_qp *qp) -{ - hisi_qm_stop_qp(qp); - hisi_qm_free_qps(&qp, 1); + h_req->cb(h_req->ctx, resp); } =20 static int hpre_ctx_init(struct hpre_ctx *ctx, u8 type) { struct hisi_qp *qp; - int ret; + struct hpre *hpre; =20 qp =3D hpre_get_qp_and_start(type); if (IS_ERR(qp)) @@ -497,19 +410,21 @@ static int hpre_ctx_init(struct hpre_ctx *ctx, u8 typ= e) =20 qp->qp_ctx =3D ctx; qp->req_cb =3D hpre_alg_cb; + spin_lock_init(&ctx->req_lock); + ctx->qp =3D qp; + ctx->dev =3D &qp->qm->pdev->dev; + hpre =3D container_of(ctx->qp->qm, struct hpre, qm); + ctx->hpre =3D hpre; + ctx->key_sz =3D 0; + ctx->crt_g2_mode =3D false; =20 - ret =3D hpre_ctx_set(ctx, qp, qp->sq_depth); - if (ret) - hpre_stop_qp_and_put(qp); - - return ret; + return 0; } =20 static int hpre_msg_request_set(struct hpre_ctx *ctx, void *req, bool is_r= sa) { struct hpre_asym_request *h_req; struct hpre_sqe *msg; - int req_id; void *tmp; =20 if (is_rsa) { @@ -549,11 +464,8 @@ static int hpre_msg_request_set(struct hpre_ctx *ctx, = void *req, bool is_rsa) msg->task_len1 =3D (ctx->key_sz >> HPRE_BITS_2_BYTES_SHIFT) - 1; h_req->ctx =3D ctx; =20 - req_id =3D hpre_add_req_to_ctx(h_req); - if (req_id < 0) - return -EBUSY; - - msg->tag =3D cpu_to_le16((u16)req_id); + hpre_dfx_add_req_time(h_req); + msg->tag =3D cpu_to_le64((uintptr_t)h_req); =20 return 0; } @@ -619,7 +531,6 @@ static int hpre_dh_compute_value(struct kpp_request *re= q) return -EINPROGRESS; =20 clear_all: - hpre_rm_req_from_ctx(hpre_req); hpre_hw_data_clr_all(ctx, hpre_req, req->dst, req->src); =20 return ret; @@ -828,7 +739,6 @@ static int hpre_rsa_enc(struct akcipher_request *req) return -EINPROGRESS; =20 clear_all: - hpre_rm_req_from_ctx(hpre_req); hpre_hw_data_clr_all(ctx, hpre_req, req->dst, req->src); =20 return ret; @@ -883,7 +793,6 @@ static int hpre_rsa_dec(struct akcipher_request *req) return -EINPROGRESS; =20 clear_all: - hpre_rm_req_from_ctx(hpre_req); hpre_hw_data_clr_all(ctx, hpre_req, req->dst, req->src); =20 return ret; @@ -1346,7 +1255,7 @@ static int hpre_ecdh_set_param(struct hpre_ctx *ctx, = struct ecdh *params) return 0; } =20 -static bool hpre_key_is_zero(char *key, unsigned short key_sz) +static bool hpre_key_is_zero(const char *key, unsigned short key_sz) { int i; =20 @@ -1488,7 +1397,6 @@ static int hpre_ecdh_msg_request_set(struct hpre_ctx = *ctx, { struct hpre_asym_request *h_req; struct hpre_sqe *msg; - int req_id; void *tmp; =20 if (req->dst_len < ctx->key_sz << 1) { @@ -1510,11 +1418,8 @@ static int hpre_ecdh_msg_request_set(struct hpre_ctx= *ctx, msg->task_len1 =3D (ctx->key_sz >> HPRE_BITS_2_BYTES_SHIFT) - 1; h_req->ctx =3D ctx; =20 - req_id =3D hpre_add_req_to_ctx(h_req); - if (req_id < 0) - return -EBUSY; - - msg->tag =3D cpu_to_le16((u16)req_id); + hpre_dfx_add_req_time(h_req); + msg->tag =3D cpu_to_le64((uintptr_t)h_req); return 0; } =20 @@ -1612,7 +1517,6 @@ static int hpre_ecdh_compute_value(struct kpp_request= *req) return -EINPROGRESS; =20 clear_all: - hpre_rm_req_from_ctx(hpre_req); hpre_ecdh_hw_data_clr_all(ctx, hpre_req, req->dst, req->src); return ret; } --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout08.his.huawei.com (canpmsgout08.his.huawei.com [113.46.200.223]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACDAE1AA1D2; Sat, 22 Nov 2025 07:49:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.223 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797764; cv=none; b=BdOLFO3+4xsqLDGy5ozbhluOt/LHUY/AQ9fUigDu3ulfUOM4KnIqVhuzQaGZhdifYw4+mmmR/YG5FMj9pkKtSscsxJ4TOAFc/cuwlUQiXs3zdTpNzknPOMDZkP9saHQuXDdlfcJgcQ3KB3OrZuXgHq6KtZIhvvA0ByEdXnOj4cQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797764; c=relaxed/simple; bh=oWd+b1biUhr5+ZZe7pFkxU6fsYxi9jHmUJ5g0E0SlSg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=d+4m9CUcJCSf+hQXvP530AsNNAl0SjM7SVMqyJWMXvPQsUuTApRQHqcLBYQCmkCos/6kFk4kCughhiOUZ5qk50ZITUtqzV8jx5g6kYCqaHh+O9jbbS50vsm3pEZjp+gWk65MSbAlSdD5KqhPoRPeXa8iEbKiuDHHs2qK5/gEEwA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=YhzUe4y8; arc=none smtp.client-ip=113.46.200.223 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="YhzUe4y8" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=GXQ1eFoWjwuKcbWdnV+rMruIPun7+UFKdLNIdyMG4BU=; b=YhzUe4y84QdauGm0Pc7MtlTtlXt9kyDQjsyqnToA0dqTqV1NuZYCYNsWgGozCdBeYPJSghjBm agMiZsKz/dLqeGEMiWVZLoq65YuInzwGverus75MFTOP/EOeaESR+qASYUCek9bZmNRa9TvhYy8 58l5al8iNJbHro7d7grTAyU= Received: from mail.maildlp.com (unknown [172.19.163.17]) by canpmsgout08.his.huawei.com (SkyGuard) with ESMTPS id 4dD41d6p85zmVYC; Sat, 22 Nov 2025 15:47:33 +0800 (CST) Received: from dggemv706-chm.china.huawei.com (unknown [10.3.19.33]) by mail.maildlp.com (Postfix) with ESMTPS id 9D6B51A0188; Sat, 22 Nov 2025 15:49:19 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv706-chm.china.huawei.com (10.3.19.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:19 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:18 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 04/11] crypto: hisilicon/qm - enhance the configuration of req_type in queue attributes Date: Sat, 22 Nov 2025 15:49:09 +0800 Message-ID: <20251122074916.2793717-5-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" Originally, when a queue was requested, it could only be configured with the default algorithm type of 0. Now, when multiple tfms use the same queue, the queue must be selected based on its attributes to meet the requirements of tfm tasks. So the algorithm type attribute of queue need to be distinguished. Just like a queue used for compression in ZIP cannot be used for decompression tasks. Fixes: 3f1ec97aacf1 ("crypto: hisilicon/qm - Put device finding logic into = QM") Signed-off-by: Chenghai Huang Signed-off-by: Weili Qian --- drivers/crypto/hisilicon/hpre/hpre_main.c | 2 +- drivers/crypto/hisilicon/qm.c | 8 ++++---- drivers/crypto/hisilicon/sec2/sec_crypto.c | 1 - drivers/crypto/hisilicon/sec2/sec_main.c | 21 ++++++++++++++++----- drivers/crypto/hisilicon/zip/zip.h | 2 +- drivers/crypto/hisilicon/zip/zip_crypto.c | 13 +++++++++---- drivers/crypto/hisilicon/zip/zip_main.c | 4 ++-- include/linux/hisi_acc_qm.h | 3 +-- 8 files changed, 34 insertions(+), 20 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_main.c b/drivers/crypto/his= ilicon/hpre/hpre_main.c index b94fecd765ee..884d5d0afaf4 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_main.c +++ b/drivers/crypto/hisilicon/hpre/hpre_main.c @@ -465,7 +465,7 @@ struct hisi_qp *hpre_create_qp(u8 type) * type: 0 - RSA/DH. algorithm supported in V2, * 1 - ECC algorithm in V3. */ - ret =3D hisi_qm_alloc_qps_node(&hpre_devices, 1, type, node, &qp); + ret =3D hisi_qm_alloc_qps_node(&hpre_devices, 1, &type, node, &qp); if (!ret) return qp; =20 diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index 4760551d4fa3..50c32e69bd28 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -3595,7 +3595,7 @@ static int hisi_qm_sort_devices(int node, struct list= _head *head, * not meet the requirements will return error. */ int hisi_qm_alloc_qps_node(struct hisi_qm_list *qm_list, int qp_num, - u8 alg_type, int node, struct hisi_qp **qps) + u8 *alg_type, int node, struct hisi_qp **qps) { struct hisi_qm_resource *tmp; int ret =3D -ENODEV; @@ -3613,7 +3613,7 @@ int hisi_qm_alloc_qps_node(struct hisi_qm_list *qm_li= st, int qp_num, =20 list_for_each_entry(tmp, &head, list) { for (i =3D 0; i < qp_num; i++) { - qps[i] =3D hisi_qm_create_qp(tmp->qm, alg_type); + qps[i] =3D hisi_qm_create_qp(tmp->qm, alg_type[i]); if (IS_ERR(qps[i])) { hisi_qm_free_qps(qps, i); break; @@ -3628,8 +3628,8 @@ int hisi_qm_alloc_qps_node(struct hisi_qm_list *qm_li= st, int qp_num, =20 mutex_unlock(&qm_list->lock); if (ret) - pr_info("Failed to create qps, node[%d], alg[%u], qp[%d]!\n", - node, alg_type, qp_num); + pr_info("Failed to create qps, node[%d], qp[%d]!\n", + node, qp_num); =20 err: free_list(&head); diff --git a/drivers/crypto/hisilicon/sec2/sec_crypto.c b/drivers/crypto/hi= silicon/sec2/sec_crypto.c index 4e41235116e1..364bd69c6088 100644 --- a/drivers/crypto/hisilicon/sec2/sec_crypto.c +++ b/drivers/crypto/hisilicon/sec2/sec_crypto.c @@ -626,7 +626,6 @@ static int sec_create_qp_ctx(struct sec_ctx *ctx, int q= p_ctx_id) =20 qp_ctx =3D &ctx->qp_ctx[qp_ctx_id]; qp =3D ctx->qps[qp_ctx_id]; - qp->req_type =3D 0; qp->qp_ctx =3D qp_ctx; qp_ctx->qp =3D qp; qp_ctx->ctx =3D ctx; diff --git a/drivers/crypto/hisilicon/sec2/sec_main.c b/drivers/crypto/hisi= licon/sec2/sec_main.c index 5eb2d6820742..7dd125f5f511 100644 --- a/drivers/crypto/hisilicon/sec2/sec_main.c +++ b/drivers/crypto/hisilicon/sec2/sec_main.c @@ -417,18 +417,29 @@ struct hisi_qp **sec_create_qps(void) int node =3D cpu_to_node(raw_smp_processor_id()); u32 ctx_num =3D ctx_q_num; struct hisi_qp **qps; + u8 *type; int ret; =20 qps =3D kcalloc(ctx_num, sizeof(struct hisi_qp *), GFP_KERNEL); if (!qps) return NULL; =20 - ret =3D hisi_qm_alloc_qps_node(&sec_devices, ctx_num, 0, node, qps); - if (!ret) - return qps; + /* The type of SEC is all 0, so just allocated by kcalloc */ + type =3D kcalloc(ctx_num, sizeof(u8), GFP_KERNEL); + if (!type) { + kfree(qps); + return NULL; + } =20 - kfree(qps); - return NULL; + ret =3D hisi_qm_alloc_qps_node(&sec_devices, ctx_num, type, node, qps); + if (ret) { + kfree(type); + kfree(qps); + return NULL; + } + + kfree(type); + return qps; } =20 u64 sec_get_alg_bitmap(struct hisi_qm *qm, u32 high, u32 low) diff --git a/drivers/crypto/hisilicon/zip/zip.h b/drivers/crypto/hisilicon/= zip/zip.h index 9fb2a9c01132..b83f228281ab 100644 --- a/drivers/crypto/hisilicon/zip/zip.h +++ b/drivers/crypto/hisilicon/zip/zip.h @@ -99,7 +99,7 @@ enum zip_cap_table_type { ZIP_CORE5_BITMAP, }; =20 -int zip_create_qps(struct hisi_qp **qps, int qp_num, int node); +int zip_create_qps(struct hisi_qp **qps, int qp_num, int node, u8 *alg_typ= e); int hisi_zip_register_to_crypto(struct hisi_qm *qm); void hisi_zip_unregister_from_crypto(struct hisi_qm *qm); bool hisi_zip_alg_support(struct hisi_qm *qm, u32 alg); diff --git a/drivers/crypto/hisilicon/zip/zip_crypto.c b/drivers/crypto/his= ilicon/zip/zip_crypto.c index b4a656e0177d..8250a33ba586 100644 --- a/drivers/crypto/hisilicon/zip/zip_crypto.c +++ b/drivers/crypto/hisilicon/zip/zip_crypto.c @@ -66,6 +66,7 @@ struct hisi_zip_qp_ctx { struct hisi_acc_sgl_pool *sgl_pool; struct hisi_zip *zip_dev; struct hisi_zip_ctx *ctx; + u8 req_type; }; =20 struct hisi_zip_sqe_ops { @@ -245,7 +246,7 @@ static int hisi_zip_do_work(struct hisi_zip_qp_ctx *qp_= ctx, goto err_unmap_input; } =20 - hisi_zip_fill_sqe(qp_ctx->ctx, &zip_sqe, qp->req_type, req); + hisi_zip_fill_sqe(qp_ctx->ctx, &zip_sqe, qp_ctx->req_type, req); =20 /* send command to start a task */ atomic64_inc(&dfx->send_cnt); @@ -360,7 +361,6 @@ static int hisi_zip_start_qp(struct hisi_qp *qp, struct= hisi_zip_qp_ctx *qp_ctx, struct device *dev =3D &qp->qm->pdev->dev; int ret; =20 - qp->req_type =3D req_type; qp->alg_type =3D alg_type; qp->qp_ctx =3D qp_ctx; =20 @@ -397,10 +397,15 @@ static int hisi_zip_ctx_init(struct hisi_zip_ctx *his= i_zip_ctx, u8 req_type, int { struct hisi_qp *qps[HZIP_CTX_Q_NUM] =3D { NULL }; struct hisi_zip_qp_ctx *qp_ctx; + u8 alg_type[HZIP_CTX_Q_NUM]; struct hisi_zip *hisi_zip; int ret, i, j; =20 - ret =3D zip_create_qps(qps, HZIP_CTX_Q_NUM, node); + /* alg_type =3D 0 for compress, 1 for decompress in hw sqe */ + for (i =3D 0; i < HZIP_CTX_Q_NUM; i++) + alg_type[i] =3D i; + + ret =3D zip_create_qps(qps, HZIP_CTX_Q_NUM, node, alg_type); if (ret) { pr_err("failed to create zip qps (%d)!\n", ret); return -ENODEV; @@ -409,7 +414,6 @@ static int hisi_zip_ctx_init(struct hisi_zip_ctx *hisi_= zip_ctx, u8 req_type, int hisi_zip =3D container_of(qps[0]->qm, struct hisi_zip, qm); =20 for (i =3D 0; i < HZIP_CTX_Q_NUM; i++) { - /* alg_type =3D 0 for compress, 1 for decompress in hw sqe */ qp_ctx =3D &hisi_zip_ctx->qp_ctx[i]; qp_ctx->ctx =3D hisi_zip_ctx; ret =3D hisi_zip_start_qp(qps[i], qp_ctx, i, req_type); @@ -422,6 +426,7 @@ static int hisi_zip_ctx_init(struct hisi_zip_ctx *hisi_= zip_ctx, u8 req_type, int } =20 qp_ctx->zip_dev =3D hisi_zip; + qp_ctx->req_type =3D req_type; } =20 hisi_zip_ctx->ops =3D &hisi_zip_ops; diff --git a/drivers/crypto/hisilicon/zip/zip_main.c b/drivers/crypto/hisil= icon/zip/zip_main.c index 4fcbe6bada06..85b26ef17548 100644 --- a/drivers/crypto/hisilicon/zip/zip_main.c +++ b/drivers/crypto/hisilicon/zip/zip_main.c @@ -446,12 +446,12 @@ static const struct pci_device_id hisi_zip_dev_ids[] = =3D { }; MODULE_DEVICE_TABLE(pci, hisi_zip_dev_ids); =20 -int zip_create_qps(struct hisi_qp **qps, int qp_num, int node) +int zip_create_qps(struct hisi_qp **qps, int qp_num, int node, u8 *alg_typ= e) { if (node =3D=3D NUMA_NO_NODE) node =3D cpu_to_node(raw_smp_processor_id()); =20 - return hisi_qm_alloc_qps_node(&zip_devices, qp_num, 0, node, qps); + return hisi_qm_alloc_qps_node(&zip_devices, qp_num, alg_type, node, qps); } =20 bool hisi_zip_alg_support(struct hisi_qm *qm, u32 alg) diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h index 2d0cc61ed886..4f83f0700990 100644 --- a/include/linux/hisi_acc_qm.h +++ b/include/linux/hisi_acc_qm.h @@ -454,7 +454,6 @@ struct hisi_qp { u16 sq_depth; u16 cq_depth; u8 alg_type; - u8 req_type; =20 struct qm_dma qdma; void *sqe; @@ -580,7 +579,7 @@ struct hisi_acc_sgl_pool *hisi_acc_create_sgl_pool(stru= ct device *dev, void hisi_acc_free_sgl_pool(struct device *dev, struct hisi_acc_sgl_pool *pool); int hisi_qm_alloc_qps_node(struct hisi_qm_list *qm_list, int qp_num, - u8 alg_type, int node, struct hisi_qp **qps); + u8 *alg_type, int node, struct hisi_qp **qps); void hisi_qm_free_qps(struct hisi_qp **qps, int qp_num); void hisi_qm_dev_shutdown(struct pci_dev *pdev); void hisi_qm_wait_task_finish(struct hisi_qm *qm, struct hisi_qm_list *qm_= list); --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout01.his.huawei.com (canpmsgout01.his.huawei.com [113.46.200.216]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB1292DA771; Sat, 22 Nov 2025 07:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.216 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797770; cv=none; b=h5Bg/zPi2wyHmNeLlrhxNABxuqcpFuotcIN2L0VEw9GULdkbLr+Fem6rqbA2HUPscgJJwO/0EEmUbSva1SStpcMuF7FXveqA/r8ibT3AXg0rVR5Z6QGc11CnaAOrMENZxa5TwSBmFqaneW9X0VhXB0Z99vS8euYb+SA/Fd3lCvs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797770; c=relaxed/simple; bh=gJ8erQE+7UFercg+aoPxAHJsyx438U7oaFkZ/uujdvk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=YUB4DzGqTc8vvv2LSwFoUXt60PvzPLZX24gN9vfbgT5BVg07obO3xOW69NyvHqvi4uu1wfCFEK3rBDv++5n66k0EY2nRzGqa7v5Kif2Z7kovflQ38j5vjyW5kJ4My9jEjFgdrJ3V6z1LmrgEbZOBltqQmGgFdenlCs94Jd0QJU4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=Opg585aq; arc=none smtp.client-ip=113.46.200.216 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="Opg585aq" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=L2DZEZZNjXARR9fq2af6gMmZk/w8TLtCp1W8YSaru8M=; b=Opg585aqdki2oUV6caKdZyIQM9s2yhyh001QdLpbx1gtrdmiIH852bQx9azqJeKRZYZZ5wVmY UPeB6lBDhicUW93VS8hzc+3XGQfjKRl8KEN/gZdDNY6L7YIQrfxnECFSVrS4tCMdxN0B7OzzW/m S0sLIbC+HkaKKXWsJJ1WFNA= Received: from mail.maildlp.com (unknown [172.19.163.48]) by canpmsgout01.his.huawei.com (SkyGuard) with ESMTPS id 4dD41q0Fz6z1T4Hl; Sat, 22 Nov 2025 15:47:43 +0800 (CST) Received: from dggemv712-chm.china.huawei.com (unknown [10.1.198.32]) by mail.maildlp.com (Postfix) with ESMTPS id 29E74180479; Sat, 22 Nov 2025 15:49:20 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv712-chm.china.huawei.com (10.1.198.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:19 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:19 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 05/11] crypto: hisilicon/qm - centralize the sending locks of each module into qm Date: Sat, 22 Nov 2025 15:49:10 +0800 Message-ID: <20251122074916.2793717-6-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" When a single queue used by multiple tfms, the protection of shared resources by individual module driver programs is no longer sufficient. The hisi_qp_send needs to be ensured by the lock in qp. Fixes: 5fdb4b345cfb ("crypto: hisilicon - add a lock for the qp send operat= ion") Signed-off-by: Chenghai Huang Signed-off-by: Weili Qian --- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 4 ---- drivers/crypto/hisilicon/qm.c | 16 ++++++++++++---- drivers/crypto/hisilicon/zip/zip_crypto.c | 3 --- include/linux/hisi_acc_qm.h | 1 + 4 files changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/h= isilicon/hpre/hpre_crypto.c index 4197281c8dff..220022ae7afb 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -109,7 +109,6 @@ struct hpre_ctx { struct hisi_qp *qp; struct device *dev; struct hpre *hpre; - spinlock_t req_lock; unsigned int key_sz; bool crt_g2_mode; union { @@ -410,7 +409,6 @@ static int hpre_ctx_init(struct hpre_ctx *ctx, u8 type) =20 qp->qp_ctx =3D ctx; qp->req_cb =3D hpre_alg_cb; - spin_lock_init(&ctx->req_lock); ctx->qp =3D qp; ctx->dev =3D &qp->qm->pdev->dev; hpre =3D container_of(ctx->qp->qm, struct hpre, qm); @@ -478,9 +476,7 @@ static int hpre_send(struct hpre_ctx *ctx, struct hpre_= sqe *msg) =20 do { atomic64_inc(&dfx[HPRE_SEND_CNT].value); - spin_lock_bh(&ctx->req_lock); ret =3D hisi_qp_send(ctx->qp, msg); - spin_unlock_bh(&ctx->req_lock); if (ret !=3D -EBUSY) break; atomic64_inc(&dfx[HPRE_SEND_BUSY_CNT].value); diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index 50c32e69bd28..3965c8d0993c 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -2369,26 +2369,33 @@ EXPORT_SYMBOL_GPL(hisi_qm_stop_qp); int hisi_qp_send(struct hisi_qp *qp, const void *msg) { struct hisi_qp_status *qp_status =3D &qp->qp_status; - u16 sq_tail =3D qp_status->sq_tail; - u16 sq_tail_next =3D (sq_tail + 1) % qp->sq_depth; - void *sqe =3D qm_get_avail_sqe(qp); + u16 sq_tail, sq_tail_next; + void *sqe; =20 + spin_lock_bh(&qp->qp_lock); if (unlikely(atomic_read(&qp->qp_status.flags) =3D=3D QP_STOP || atomic_read(&qp->qm->status.flags) =3D=3D QM_STOP || qp->is_resetting)) { + spin_unlock_bh(&qp->qp_lock); dev_info_ratelimited(&qp->qm->pdev->dev, "QP is stopped or resetting\n"); return -EAGAIN; } =20 - if (!sqe) + sqe =3D qm_get_avail_sqe(qp); + if (!sqe) { + spin_unlock_bh(&qp->qp_lock); return -EBUSY; + } =20 + sq_tail =3D qp_status->sq_tail; + sq_tail_next =3D (sq_tail + 1) % qp->sq_depth; memcpy(sqe, msg, qp->qm->sqe_size); qp->msg[sq_tail] =3D msg; =20 qm_db(qp->qm, qp->qp_id, QM_DOORBELL_CMD_SQ, sq_tail_next, 0); atomic_inc(&qp->qp_status.used); qp_status->sq_tail =3D sq_tail_next; + spin_unlock_bh(&qp->qp_lock); =20 return 0; } @@ -2968,6 +2975,7 @@ static int hisi_qp_memory_init(struct hisi_qm *qm, si= ze_t dma_size, int id, qp->qm =3D qm; qp->qp_id =3D id; =20 + spin_lock_init(&qp->qp_lock); spin_lock_init(&qp->backlog.lock); INIT_LIST_HEAD(&qp->backlog.list); =20 diff --git a/drivers/crypto/hisilicon/zip/zip_crypto.c b/drivers/crypto/his= ilicon/zip/zip_crypto.c index 8250a33ba586..2f9035c016f3 100644 --- a/drivers/crypto/hisilicon/zip/zip_crypto.c +++ b/drivers/crypto/hisilicon/zip/zip_crypto.c @@ -217,7 +217,6 @@ static int hisi_zip_do_work(struct hisi_zip_qp_ctx *qp_= ctx, { struct hisi_acc_sgl_pool *pool =3D qp_ctx->sgl_pool; struct hisi_zip_dfx *dfx =3D &qp_ctx->zip_dev->dfx; - struct hisi_zip_req_q *req_q =3D &qp_ctx->req_q; struct acomp_req *a_req =3D req->req; struct hisi_qp *qp =3D qp_ctx->qp; struct device *dev =3D &qp->qm->pdev->dev; @@ -250,9 +249,7 @@ static int hisi_zip_do_work(struct hisi_zip_qp_ctx *qp_= ctx, =20 /* send command to start a task */ atomic64_inc(&dfx->send_cnt); - spin_lock_bh(&req_q->req_lock); ret =3D hisi_qp_send(qp, &zip_sqe); - spin_unlock_bh(&req_q->req_lock); if (unlikely(ret < 0)) { atomic64_inc(&dfx->send_busy_cnt); ret =3D -EAGAIN; diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h index 4f83f0700990..75ae01ddaa1a 100644 --- a/include/linux/hisi_acc_qm.h +++ b/include/linux/hisi_acc_qm.h @@ -473,6 +473,7 @@ struct hisi_qp { u16 pasid; struct uacce_queue *uacce_q; =20 + spinlock_t qp_lock; struct instance_backlog backlog; const void **msg; }; --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout12.his.huawei.com (canpmsgout12.his.huawei.com [113.46.200.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B81FD1DE2A5; Sat, 22 Nov 2025 07:49:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.227 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797772; cv=none; b=fu1UDCsfj2H5sQtcz0tyS1mNJ35ttnfiGdcS7QSn2Rx0Vx+puxn+S+79hawJP8tOfmOJDY/y/1AadIealy+HTa10cFgtFHw4U1qtwzlFt3GJFTBeEZWnw3IVHU+lH74uY22RNSY31UQjQs1bxCe47wEz8ME/bkJAcqh6RLhWaXE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797772; c=relaxed/simple; bh=Q2gVVrlQce/ejOUKLA694IvCSeb7BTJl8Q/9XxyZnJ8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Oh0nzfSnT72/yNhvrCqW09CDKfn8yOpm4GeDGIop0V/KVFDQ2l+kdug9ZvQ9VtWyHpKfAYj+6n5a/V0nQqM9FGC+7uFSQxtuYjIxKYNlmzMxGAMuoleoy++4c6I0eVULZ2GjAAzw20azl/ho0zbRh7AdB7iQD5c7urkI5QNU13k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=xRRonAxM; arc=none smtp.client-ip=113.46.200.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="xRRonAxM" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=PDgCsS0HdBW5v8MKTJqrHgteS9M6r91rTTpe+Qi/aro=; b=xRRonAxMlOhwdsVqmcIBNr2RC5eRQICArI84CRt35T2NuJnJoGgnFgRkyhCn0KPvfOclTsGWS VB2UEZDK8R2NNHWztEllw5KrLXV2qYCX44QK+u4ubKLnnpSpCFGKDRi7j3GSiP1JGmaBrS6Y8AT CGKyIvC8B6J9R3+TYYKJVMo= Received: from mail.maildlp.com (unknown [172.19.88.163]) by canpmsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4dD4225KNNznTWr; Sat, 22 Nov 2025 15:47:54 +0800 (CST) Received: from dggemv705-chm.china.huawei.com (unknown [10.3.19.32]) by mail.maildlp.com (Postfix) with ESMTPS id A058F1800CE; Sat, 22 Nov 2025 15:49:20 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv705-chm.china.huawei.com (10.3.19.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:20 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:19 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 06/11] crypto: hisilicon - consolidate qp creation and start in hisi_qm_alloc_qps_node Date: Sat, 22 Nov 2025 15:49:11 +0800 Message-ID: <20251122074916.2793717-7-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" Consolidate the creation and start of qp into the function hisi_qm_alloc_qps_node. This change eliminates the need for each module to perform these steps in two separate phases (creation and start). Signed-off-by: Chenghai Huang Signed-off-by: Weili Qian --- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 40 ++----------- drivers/crypto/hisilicon/qm.c | 65 ++++++++++++++++----- drivers/crypto/hisilicon/sec2/sec_crypto.c | 8 --- drivers/crypto/hisilicon/zip/zip_crypto.c | 40 +------------ include/linux/hisi_acc_qm.h | 1 - 5 files changed, 58 insertions(+), 96 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/h= isilicon/hpre/hpre_crypto.c index 220022ae7afb..f410e610eaba 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -156,27 +156,6 @@ static void hpre_dfx_add_req_time(struct hpre_asym_req= uest *hpre_req) ktime_get_ts64(&hpre_req->req_time); } =20 -static struct hisi_qp *hpre_get_qp_and_start(u8 type) -{ - struct hisi_qp *qp; - int ret; - - qp =3D hpre_create_qp(type); - if (!qp) { - pr_err("Can not create hpre qp!\n"); - return ERR_PTR(-ENODEV); - } - - ret =3D hisi_qm_start_qp(qp, 0); - if (ret < 0) { - hisi_qm_free_qps(&qp, 1); - pci_err(qp->qm->pdev, "Can not start qp!\n"); - return ERR_PTR(-EINVAL); - } - - return qp; -} - static int hpre_get_data_dma_addr(struct hpre_asym_request *hpre_req, struct scatterlist *data, unsigned int len, int is_src, dma_addr_t *tmp) @@ -316,9 +295,8 @@ static int hpre_alg_res_post_hf(struct hpre_ctx *ctx, s= truct hpre_sqe *sqe, =20 static void hpre_ctx_clear(struct hpre_ctx *ctx, bool is_clear_all) { - if (is_clear_all) { + if (is_clear_all) hisi_qm_free_qps(&ctx->qp, 1); - } =20 ctx->crt_g2_mode =3D false; ctx->key_sz =3D 0; @@ -403,11 +381,10 @@ static int hpre_ctx_init(struct hpre_ctx *ctx, u8 typ= e) struct hisi_qp *qp; struct hpre *hpre; =20 - qp =3D hpre_get_qp_and_start(type); - if (IS_ERR(qp)) - return PTR_ERR(qp); + qp =3D hpre_create_qp(type); + if (!qp) + return -ENODEV; =20 - qp->qp_ctx =3D ctx; qp->req_cb =3D hpre_alg_cb; ctx->qp =3D qp; ctx->dev =3D &qp->qm->pdev->dev; @@ -597,9 +574,6 @@ static void hpre_dh_clear_ctx(struct hpre_ctx *ctx, boo= l is_clear_all) struct device *dev =3D ctx->dev; unsigned int sz =3D ctx->key_sz; =20 - if (is_clear_all) - hisi_qm_stop_qp(ctx->qp); - if (ctx->dh.g) { dma_free_coherent(dev, sz, ctx->dh.g, ctx->dh.dma_g); ctx->dh.g =3D NULL; @@ -940,9 +914,6 @@ static void hpre_rsa_clear_ctx(struct hpre_ctx *ctx, bo= ol is_clear_all) unsigned int half_key_sz =3D ctx->key_sz >> 1; struct device *dev =3D ctx->dev; =20 - if (is_clear_all) - hisi_qm_stop_qp(ctx->qp); - if (ctx->rsa.pubkey) { dma_free_coherent(dev, ctx->key_sz << 1, ctx->rsa.pubkey, ctx->rsa.dma_pubkey); @@ -1112,9 +1083,6 @@ static void hpre_ecc_clear_ctx(struct hpre_ctx *ctx, = bool is_clear_all) unsigned int sz =3D ctx->key_sz; unsigned int shift =3D sz << 1; =20 - if (is_clear_all) - hisi_qm_stop_qp(ctx->qp); - if (ctx->ecdh.p) { /* ecdh: p->a->k->b */ memzero_explicit(ctx->ecdh.p + shift, sz); diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index 3965c8d0993c..28256f64aa3c 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -3528,6 +3528,14 @@ void hisi_qm_dev_err_uninit(struct hisi_qm *qm) } EXPORT_SYMBOL_GPL(hisi_qm_dev_err_uninit); =20 +static void qm_release_qp_nolock(struct hisi_qp *qp) +{ + struct hisi_qm *qm =3D qp->qm; + + qm->qp_in_used--; + idr_remove(&qm->qp_idr, qp->qp_id); +} + /** * hisi_qm_free_qps() - free multiple queue pairs. * @qps: The queue pairs need to be freed. @@ -3540,8 +3548,14 @@ void hisi_qm_free_qps(struct hisi_qp **qps, int qp_n= um) if (!qps || qp_num <=3D 0) return; =20 - for (i =3D qp_num - 1; i >=3D 0; i--) - hisi_qm_release_qp(qps[i]); + down_write(&qps[0]->qm->qps_lock); + + for (i =3D qp_num - 1; i >=3D 0; i--) { + qm_stop_qp_nolock(qps[i]); + qm_release_qp_nolock(qps[i]); + } + + up_write(&qps[0]->qm->qps_lock); } EXPORT_SYMBOL_GPL(hisi_qm_free_qps); =20 @@ -3555,6 +3569,39 @@ static void free_list(struct list_head *head) } } =20 +static int qm_get_and_start_qp(struct hisi_qm *qm, int qp_num, struct hisi= _qp **qps, u8 *alg_type) +{ + int ret =3D -ENODEV; + int i, j; + + down_write(&qm->qps_lock); + for (i =3D 0; i < qp_num; i++) { + qps[i] =3D qm_create_qp_nolock(qm, alg_type[i]); + if (IS_ERR(qps[i])) { + goto free_qp; + } + } + + for (j =3D 0; j < qp_num; j++) { + ret =3D qm_start_qp_nolock(qps[j], 0); + if (ret) + goto stop_qp; + } + up_write(&qm->qps_lock); + + return 0; + +stop_qp: + for (j--; j >=3D 0; j--) + qm_stop_qp_nolock(qps[j]); +free_qp: + for (i--; i >=3D 0; i--) + qm_release_qp_nolock(qps[i]); + up_write(&qm->qps_lock); + + return ret; +} + static int hisi_qm_sort_devices(int node, struct list_head *head, struct hisi_qm_list *qm_list) { @@ -3608,7 +3655,6 @@ int hisi_qm_alloc_qps_node(struct hisi_qm_list *qm_li= st, int qp_num, struct hisi_qm_resource *tmp; int ret =3D -ENODEV; LIST_HEAD(head); - int i; =20 if (!qps || !qm_list || qp_num <=3D 0) return -EINVAL; @@ -3620,18 +3666,9 @@ int hisi_qm_alloc_qps_node(struct hisi_qm_list *qm_l= ist, int qp_num, } =20 list_for_each_entry(tmp, &head, list) { - for (i =3D 0; i < qp_num; i++) { - qps[i] =3D hisi_qm_create_qp(tmp->qm, alg_type[i]); - if (IS_ERR(qps[i])) { - hisi_qm_free_qps(qps, i); - break; - } - } - - if (i =3D=3D qp_num) { - ret =3D 0; + ret =3D qm_get_and_start_qp(tmp->qm, qp_num, qps, alg_type); + if (!ret) break; - } } =20 mutex_unlock(&qm_list->lock); diff --git a/drivers/crypto/hisilicon/sec2/sec_crypto.c b/drivers/crypto/hi= silicon/sec2/sec_crypto.c index 364bd69c6088..d09d081f42dc 100644 --- a/drivers/crypto/hisilicon/sec2/sec_crypto.c +++ b/drivers/crypto/hisilicon/sec2/sec_crypto.c @@ -626,7 +626,6 @@ static int sec_create_qp_ctx(struct sec_ctx *ctx, int q= p_ctx_id) =20 qp_ctx =3D &ctx->qp_ctx[qp_ctx_id]; qp =3D ctx->qps[qp_ctx_id]; - qp->qp_ctx =3D qp_ctx; qp_ctx->qp =3D qp; qp_ctx->ctx =3D ctx; =20 @@ -644,14 +643,8 @@ static int sec_create_qp_ctx(struct sec_ctx *ctx, int = qp_ctx_id) if (ret) goto err_destroy_idr; =20 - ret =3D hisi_qm_start_qp(qp, 0); - if (ret < 0) - goto err_resource_free; - return 0; =20 -err_resource_free: - sec_free_qp_ctx_resource(ctx, qp_ctx); err_destroy_idr: idr_destroy(&qp_ctx->req_idr); return ret; @@ -660,7 +653,6 @@ static int sec_create_qp_ctx(struct sec_ctx *ctx, int q= p_ctx_id) static void sec_release_qp_ctx(struct sec_ctx *ctx, struct sec_qp_ctx *qp_ctx) { - hisi_qm_stop_qp(qp_ctx->qp); sec_free_qp_ctx_resource(ctx, qp_ctx); idr_destroy(&qp_ctx->req_idr); } diff --git a/drivers/crypto/hisilicon/zip/zip_crypto.c b/drivers/crypto/his= ilicon/zip/zip_crypto.c index 2f9035c016f3..e675269e2e02 100644 --- a/drivers/crypto/hisilicon/zip/zip_crypto.c +++ b/drivers/crypto/hisilicon/zip/zip_crypto.c @@ -352,32 +352,6 @@ static int hisi_zip_adecompress(struct acomp_req *acom= p_req) return ret; } =20 -static int hisi_zip_start_qp(struct hisi_qp *qp, struct hisi_zip_qp_ctx *q= p_ctx, - int alg_type, int req_type) -{ - struct device *dev =3D &qp->qm->pdev->dev; - int ret; - - qp->alg_type =3D alg_type; - qp->qp_ctx =3D qp_ctx; - - ret =3D hisi_qm_start_qp(qp, 0); - if (ret < 0) { - dev_err(dev, "failed to start qp (%d)!\n", ret); - return ret; - } - - qp_ctx->qp =3D qp; - - return 0; -} - -static void hisi_zip_release_qp(struct hisi_zip_qp_ctx *qp_ctx) -{ - hisi_qm_stop_qp(qp_ctx->qp); - hisi_qm_free_qps(&qp_ctx->qp, 1); -} - static const struct hisi_zip_sqe_ops hisi_zip_ops =3D { .sqe_type =3D 0x3, .fill_addr =3D hisi_zip_fill_addr, @@ -396,7 +370,7 @@ static int hisi_zip_ctx_init(struct hisi_zip_ctx *hisi_= zip_ctx, u8 req_type, int struct hisi_zip_qp_ctx *qp_ctx; u8 alg_type[HZIP_CTX_Q_NUM]; struct hisi_zip *hisi_zip; - int ret, i, j; + int ret, i; =20 /* alg_type =3D 0 for compress, 1 for decompress in hw sqe */ for (i =3D 0; i < HZIP_CTX_Q_NUM; i++) @@ -413,17 +387,9 @@ static int hisi_zip_ctx_init(struct hisi_zip_ctx *hisi= _zip_ctx, u8 req_type, int for (i =3D 0; i < HZIP_CTX_Q_NUM; i++) { qp_ctx =3D &hisi_zip_ctx->qp_ctx[i]; qp_ctx->ctx =3D hisi_zip_ctx; - ret =3D hisi_zip_start_qp(qps[i], qp_ctx, i, req_type); - if (ret) { - for (j =3D i - 1; j >=3D 0; j--) - hisi_qm_stop_qp(hisi_zip_ctx->qp_ctx[j].qp); - - hisi_qm_free_qps(qps, HZIP_CTX_Q_NUM); - return ret; - } - qp_ctx->zip_dev =3D hisi_zip; qp_ctx->req_type =3D req_type; + qp_ctx->qp =3D qps[i]; } =20 hisi_zip_ctx->ops =3D &hisi_zip_ops; @@ -436,7 +402,7 @@ static void hisi_zip_ctx_exit(struct hisi_zip_ctx *hisi= _zip_ctx) int i; =20 for (i =3D 0; i < HZIP_CTX_Q_NUM; i++) - hisi_zip_release_qp(&hisi_zip_ctx->qp_ctx[i]); + hisi_qm_free_qps(&hisi_zip_ctx->qp_ctx[i].qp, 1); } =20 static int hisi_zip_create_req_q(struct hisi_zip_ctx *ctx) diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h index 75ae01ddaa1a..4cf418a41fe4 100644 --- a/include/linux/hisi_acc_qm.h +++ b/include/linux/hisi_acc_qm.h @@ -463,7 +463,6 @@ struct hisi_qp { =20 struct hisi_qp_status qp_status; struct hisi_qp_ops *hw_ops; - void *qp_ctx; void (*req_cb)(struct hisi_qp *qp, void *data); void (*event_cb)(struct hisi_qp *qp); =20 --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout02.his.huawei.com (canpmsgout02.his.huawei.com [113.46.200.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C6701B85F8; Sat, 22 Nov 2025 07:49:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.217 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797766; cv=none; b=IWH/Ep2jtdgd0sNpH9jVunghwgPlkVZopeQUeFAwidWbyHalN9bj9xyZDBPiRzk1w9PAjj1fQUtRNYjBqXJVLEfLLDspbw7QSxmL/nZ60FUwI57NimyEA9k2+Qs4Jxg6tvC3UttdR+b0dZJIz6abj43oH1LWA3YMTlTRip9g9xU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797766; c=relaxed/simple; bh=6176ZIZ5IiSfussiKGbTQdM3mXFCSDfrmxc/eiNztH0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=laKgJtU92Oei3wUSneLughNdEd/APMRqjj2OQqGr4jqnaO1ZGeQB590RbtOBD9ETJ39Atd3P44PzwB/rOj+/tOENglq5hdnXkeq2S1PE4GI5A+YHjyN7BWHRhkLn3R+K38/yVR6o+9s/c9QrxTH36MIA6tAHx26JReK4TIbmUmc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=UNmmFbWn; arc=none smtp.client-ip=113.46.200.217 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="UNmmFbWn" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=Vm6uXFXxydiiwKIsx7W8VRO5SKwRPFOKZPatyoA1SlY=; b=UNmmFbWnCsAvG2KynOxi/4GuqtIO1h1V82DJv6RsybcPSnX23Q+VQmJQ74sYjKDCbgervB/5X QUakOVguUMd4j54qZgMT57AubIZ+tGb5XKY8Y8NeuxnmxnzClbmzCf1DdnDQQuEvwe/BmHbDYg+ mUOTTtNQjGOqILeAI4uUGUk= Received: from mail.maildlp.com (unknown [172.19.88.105]) by canpmsgout02.his.huawei.com (SkyGuard) with ESMTPS id 4dD41F32rNzcZyP; Sat, 22 Nov 2025 15:47:13 +0800 (CST) Received: from dggemv706-chm.china.huawei.com (unknown [10.3.19.33]) by mail.maildlp.com (Postfix) with ESMTPS id 143521401F2; Sat, 22 Nov 2025 15:49:21 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv706-chm.china.huawei.com (10.3.19.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:20 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:20 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 07/11] crypto: hisilicon/qm - add reference counting to queues for tfm kernel reuse Date: Sat, 22 Nov 2025 15:49:12 +0800 Message-ID: <20251122074916.2793717-8-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" Add reference counting to queues. When all queues are occupied, tfm will reuse queues with the same algorithm type that have already been allocated in the kernel. The corresponding queue will be released when the reference count reaches 1. Signed-off-by: Chenghai Huang Signed-off-by: Weili Qian --- drivers/crypto/hisilicon/qm.c | 81 +++++++++++++++++++++++++++-------- include/linux/hisi_acc_qm.h | 1 + 2 files changed, 65 insertions(+), 17 deletions(-) diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index 28256f64aa3c..6ff189941300 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -2002,7 +2002,38 @@ static void hisi_qm_unset_hw_reset(struct hisi_qp *q= p) *addr =3D 0; } =20 -static struct hisi_qp *qm_create_qp_nolock(struct hisi_qm *qm, u8 alg_type) +static struct hisi_qp *find_shareable_qp(struct hisi_qm *qm, u8 alg_type, = bool is_in_kernel) +{ + struct device *dev =3D &qm->pdev->dev; + struct hisi_qp *share_qp =3D NULL; + struct hisi_qp *qp; + u32 ref_count =3D ~0; + int i; + + if (!is_in_kernel) + goto queues_busy; + + for (i =3D 0; i < qm->qp_num; i++) { + qp =3D &qm->qp_array[i]; + if (qp->is_in_kernel && qp->alg_type =3D=3D alg_type && qp->ref_count < = ref_count) { + ref_count =3D qp->ref_count; + share_qp =3D qp; + } + } + + if (share_qp) { + share_qp->ref_count++; + return share_qp; + } + +queues_busy: + dev_info_ratelimited(dev, "All %u queues of QM are busy and no shareable = queue\n", + qm->qp_num); + atomic64_inc(&qm->debug.dfx.create_qp_err_cnt); + return ERR_PTR(-EBUSY); +} + +static struct hisi_qp *qm_create_qp_nolock(struct hisi_qm *qm, u8 alg_type= , bool is_in_kernel) { struct device *dev =3D &qm->pdev->dev; struct hisi_qp *qp; @@ -2013,17 +2044,14 @@ static struct hisi_qp *qm_create_qp_nolock(struct h= isi_qm *qm, u8 alg_type) return ERR_PTR(-EPERM); } =20 - if (qm->qp_in_used =3D=3D qm->qp_num) { - dev_info_ratelimited(dev, "All %u queues of QM are busy!\n", - qm->qp_num); - atomic64_inc(&qm->debug.dfx.create_qp_err_cnt); - return ERR_PTR(-EBUSY); - } + /* Try to find a shareable queue when all queues are busy */ + if (qm->qp_in_used =3D=3D qm->qp_num) + return find_shareable_qp(qm, alg_type, is_in_kernel); =20 qp_id =3D idr_alloc_cyclic(&qm->qp_idr, NULL, 0, qm->qp_num, GFP_ATOMIC); if (qp_id < 0) { - dev_info_ratelimited(dev, "All %u queues of QM are busy!\n", - qm->qp_num); + dev_info_ratelimited(dev, "All %u queues of QM are busy, in_used =3D %u!= \n", + qm->qp_num, qm->qp_in_used); atomic64_inc(&qm->debug.dfx.create_qp_err_cnt); return ERR_PTR(-EBUSY); } @@ -2034,10 +2062,10 @@ static struct hisi_qp *qm_create_qp_nolock(struct h= isi_qm *qm, u8 alg_type) =20 qp->event_cb =3D NULL; qp->req_cb =3D NULL; - qp->qp_id =3D qp_id; qp->alg_type =3D alg_type; - qp->is_in_kernel =3D true; + qp->is_in_kernel =3D is_in_kernel; qm->qp_in_used++; + qp->ref_count =3D 1; =20 return qp; } @@ -2059,7 +2087,7 @@ static struct hisi_qp *hisi_qm_create_qp(struct hisi_= qm *qm, u8 alg_type) return ERR_PTR(ret); =20 down_write(&qm->qps_lock); - qp =3D qm_create_qp_nolock(qm, alg_type); + qp =3D qm_create_qp_nolock(qm, alg_type, false); up_write(&qm->qps_lock); =20 if (IS_ERR(qp)) @@ -2458,7 +2486,6 @@ static int hisi_qm_uacce_get_queue(struct uacce_devic= e *uacce, qp->uacce_q =3D q; qp->event_cb =3D qm_qp_event_notifier; qp->pasid =3D arg; - qp->is_in_kernel =3D false; =20 return 0; } @@ -3532,6 +3559,9 @@ static void qm_release_qp_nolock(struct hisi_qp *qp) { struct hisi_qm *qm =3D qp->qm; =20 + if (--qp->ref_count) + return; + qm->qp_in_used--; idr_remove(&qm->qp_idr, qp->qp_id); } @@ -3551,7 +3581,10 @@ void hisi_qm_free_qps(struct hisi_qp **qps, int qp_n= um) down_write(&qps[0]->qm->qps_lock); =20 for (i =3D qp_num - 1; i >=3D 0; i--) { - qm_stop_qp_nolock(qps[i]); + if (qps[i]->ref_count =3D=3D 1) { + qm_stop_qp_nolock(qps[i]); + qm_pm_put_sync(qps[i]->qm); + } qm_release_qp_nolock(qps[i]); } =20 @@ -3576,16 +3609,27 @@ static int qm_get_and_start_qp(struct hisi_qm *qm, = int qp_num, struct hisi_qp ** =20 down_write(&qm->qps_lock); for (i =3D 0; i < qp_num; i++) { - qps[i] =3D qm_create_qp_nolock(qm, alg_type[i]); + qps[i] =3D qm_create_qp_nolock(qm, alg_type[i], true); if (IS_ERR(qps[i])) { goto free_qp; } } =20 for (j =3D 0; j < qp_num; j++) { + if (qps[j]->ref_count !=3D 1) + continue; + + ret =3D qm_pm_get_sync(qm); + if (ret) { + ret =3D -EINVAL; + goto stop_qp; + } + ret =3D qm_start_qp_nolock(qps[j], 0); - if (ret) + if (ret) { + qm_pm_put_sync(qm); goto stop_qp; + } } up_write(&qm->qps_lock); =20 @@ -3593,7 +3637,10 @@ static int qm_get_and_start_qp(struct hisi_qm *qm, i= nt qp_num, struct hisi_qp ** =20 stop_qp: for (j--; j >=3D 0; j--) - qm_stop_qp_nolock(qps[j]); + if (qps[j]->ref_count =3D=3D 1) { + qm_stop_qp_nolock(qps[j]); + qm_pm_put_sync(qm); + } free_qp: for (i--; i >=3D 0; i--) qm_release_qp_nolock(qps[i]); diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h index 4cf418a41fe4..26032d98e9bd 100644 --- a/include/linux/hisi_acc_qm.h +++ b/include/linux/hisi_acc_qm.h @@ -472,6 +472,7 @@ struct hisi_qp { u16 pasid; struct uacce_queue *uacce_q; =20 + u32 ref_count; spinlock_t qp_lock; struct instance_backlog backlog; const void **msg; --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout12.his.huawei.com (canpmsgout12.his.huawei.com [113.46.200.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 006122EAB6F; Sat, 22 Nov 2025 07:49:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.227 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797772; cv=none; b=ZzSq2Ax/NM/MkrZ9fdrY0J/GXkZsyOgaBhxGQEXW7dEDhb0WytIE6k4RM+ifVg3LfGJIdrjEmt0hmJLWS5OQzLYpnU3R4GbjKMxO1Xm0OZ5F+bKgUz+XMjv3VwiPIGkGsJl1l28oDo3tt+fBGd1bOrIJdEi320wdgw9oil7jxd4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797772; c=relaxed/simple; bh=ApDV33S+/zg89XOdXLh8aO65cUqiOOstQAFhRfXjd9M=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bGqG/p2BQfajBoj8w6fRTTVlkedsI5nYsZ0oUpdij0NYFw1aSeGfcw9IGvcF9mbzORROw96dAstjV6aAE9WyP57vpsO8C56OmRwyGvK3265I9ggw6kNcY+ZfYmCMbRN/rK7Td7d7ndohnRnI1lFPeIt9eVkeD1GnbSAhs9bm3vw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=l0/JZt5B; arc=none smtp.client-ip=113.46.200.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="l0/JZt5B" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=lJvBbjVBXZcQn74sG6LQqdM85jl/ycBukkm6sqBngi0=; b=l0/JZt5BG1u47YtQFXjKl/xv74vRKMlpTNqg7UOPZDcZpPyk5L87jP4a2H6B5+uXc8vIMtHQs +p09wVZXsgPoD+GEis/PX6qETngN/iIyYMaPAIkCtnpg33npDwV27uGq2lXPB338HO5coxXga6/ IIsJnjocz/WYlqEA0jLury8= Received: from mail.maildlp.com (unknown [172.19.88.234]) by canpmsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4dD4234rvcznTwN; Sat, 22 Nov 2025 15:47:55 +0800 (CST) Received: from dggemv712-chm.china.huawei.com (unknown [10.1.198.32]) by mail.maildlp.com (Postfix) with ESMTPS id 915F7140296; Sat, 22 Nov 2025 15:49:21 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv712-chm.china.huawei.com (10.1.198.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:21 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:20 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 08/11] crypto: hisilicon/qm - optimize device selection priority based on queue ref count and NUMA distance Date: Sat, 22 Nov 2025 15:49:13 +0800 Message-ID: <20251122074916.2793717-9-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" Add device sorting criteria to prioritize devices with fewer references and closer NUMA distances. Devices that are fully occupied will not be prioritized for use. Signed-off-by: Chenghai Huang Signed-off-by: Weili Qian --- drivers/crypto/hisilicon/qm.c | 35 +++++++++++++++++++++++++---------- 1 file changed, 25 insertions(+), 10 deletions(-) diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index 6ff189941300..e4f360172477 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -3592,6 +3592,20 @@ void hisi_qm_free_qps(struct hisi_qp **qps, int qp_n= um) } EXPORT_SYMBOL_GPL(hisi_qm_free_qps); =20 +static void qm_insert_sorted(struct list_head *head, struct hisi_qm_resour= ce *res) +{ + struct hisi_qm_resource *tmp; + struct list_head *n =3D head; + + list_for_each_entry(tmp, head, list) { + if (res->distance < tmp->distance) { + n =3D &tmp->list; + break; + } + } + list_add_tail(&res->list, n); +} + static void free_list(struct list_head *head) { struct hisi_qm_resource *res, *tmp; @@ -3652,11 +3666,12 @@ static int qm_get_and_start_qp(struct hisi_qm *qm, = int qp_num, struct hisi_qp ** static int hisi_qm_sort_devices(int node, struct list_head *head, struct hisi_qm_list *qm_list) { - struct hisi_qm_resource *res, *tmp; + struct hisi_qm_resource *res; struct hisi_qm *qm; - struct list_head *n; struct device *dev; int dev_node; + LIST_HEAD(non_full_list); + LIST_HEAD(full_list); =20 list_for_each_entry(qm, &qm_list->list, list) { dev =3D &qm->pdev->dev; @@ -3671,16 +3686,16 @@ static int hisi_qm_sort_devices(int node, struct li= st_head *head, =20 res->qm =3D qm; res->distance =3D node_distance(dev_node, node); - n =3D head; - list_for_each_entry(tmp, head, list) { - if (res->distance < tmp->distance) { - n =3D &tmp->list; - break; - } - } - list_add_tail(&res->list, n); + + if (qm->qp_in_used =3D=3D qm->qp_num) + qm_insert_sorted(&full_list, res); + else + qm_insert_sorted(&non_full_list, res); } =20 + list_splice_tail(&non_full_list, head); + list_splice_tail(&full_list, head); + return 0; } =20 --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout03.his.huawei.com (canpmsgout03.his.huawei.com [113.46.200.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13F562EFD9E; Sat, 22 Nov 2025 07:49:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.218 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797772; cv=none; b=CN+KonIPKu5hSOvDrV4phNpBYvYfyXXajo/MV9MZ8I5zbY4s/ppOgksZBNXu3c68v11/jpiDvbLsTH2SXDOE4mXV4Tu+rT5oxsAA4GOyTxd8LOkggB1eK5OhW+QiuW+vl/mwdKTFm0HugEcPETvxhBGbhw+CMsXkRduqs+AiMOI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797772; c=relaxed/simple; bh=K3fMXEs/AIjOZiYh+qj/M3wWSOKD5Ye/DP/wFbPfusE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QfnAZFocL/BxoCpfwoThNStuS/sspeDQesunsIuqsUkWHrc+jWzhHWRzZcwyivsOPjQc/rMntPJTZIVtrg10ew4EOK5YjsJlxztcSScYdJT5E8GBFlZeSYl1g6QyvU9eYXnlM7vcER8xdn/wy19wojkFqHXMU+b1cBJpMBDAhas= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=42vVJSDo; arc=none smtp.client-ip=113.46.200.218 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="42vVJSDo" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=7ElhesMM7hnWDMkwRJdjH2UufMRtu9tAkQWz5Sf9l/o=; b=42vVJSDo7MWf8y8c5jWzmG2HFhsY1x/b1XModt6uaiAdzNwqfY2gKRKu7gLno+2O34+wV/cJk bD1QBE5LJ3wO8wwmz2sk1FpDAF9vfSJLoX2tKqv/xbqXjassYkwbeC5YbHnG4SRv9puEJIxjQud cE/UBV5xBi40pipOd9Sd0HI= Received: from mail.maildlp.com (unknown [172.19.163.174]) by canpmsgout03.his.huawei.com (SkyGuard) with ESMTPS id 4dD41P5pcGzpStc; Sat, 22 Nov 2025 15:47:21 +0800 (CST) Received: from dggemv705-chm.china.huawei.com (unknown [10.3.19.32]) by mail.maildlp.com (Postfix) with ESMTPS id 05946140203; Sat, 22 Nov 2025 15:49:22 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv705-chm.china.huawei.com (10.3.19.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:21 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:21 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 09/11] crypto: hisilicon/zip - support fallback for zip Date: Sat, 22 Nov 2025 15:49:14 +0800 Message-ID: <20251122074916.2793717-10-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" When the hardware queue resource busy(no shareable queue) or memery alloc fail in initialization of acomp_alg, use soft algorithm to complete the work. Fixes: 1a9e6f59caee ("crypto: hisilicon/zip - remove zlib and gzip") Signed-off-by: Chenghai Huang --- drivers/crypto/hisilicon/Kconfig | 1 + drivers/crypto/hisilicon/zip/zip_crypto.c | 50 +++++++++++++++++++---- 2 files changed, 43 insertions(+), 8 deletions(-) diff --git a/drivers/crypto/hisilicon/Kconfig b/drivers/crypto/hisilicon/Kc= onfig index 4835bdebdbb3..a0cb1a8186ac 100644 --- a/drivers/crypto/hisilicon/Kconfig +++ b/drivers/crypto/hisilicon/Kconfig @@ -57,6 +57,7 @@ config CRYPTO_DEV_HISI_ZIP depends on UACCE || UACCE=3Dn depends on ACPI select CRYPTO_DEV_HISI_QM + select CRYPTO_DEFLATE help Support for HiSilicon ZIP Driver =20 diff --git a/drivers/crypto/hisilicon/zip/zip_crypto.c b/drivers/crypto/his= ilicon/zip/zip_crypto.c index e675269e2e02..e1d6a1e61724 100644 --- a/drivers/crypto/hisilicon/zip/zip_crypto.c +++ b/drivers/crypto/hisilicon/zip/zip_crypto.c @@ -84,6 +84,7 @@ struct hisi_zip_sqe_ops { struct hisi_zip_ctx { struct hisi_zip_qp_ctx qp_ctx[HZIP_CTX_Q_NUM]; const struct hisi_zip_sqe_ops *ops; + bool fallback; }; =20 static int sgl_sge_nr_set(const char *val, const struct kernel_param *kp) @@ -110,6 +111,24 @@ static u16 sgl_sge_nr =3D HZIP_SGL_SGE_NR; module_param_cb(sgl_sge_nr, &sgl_sge_nr_ops, &sgl_sge_nr, 0444); MODULE_PARM_DESC(sgl_sge_nr, "Number of sge in sgl(1-255)"); =20 +static int hisi_zip_fallback_do_work(struct acomp_req *acomp_req, bool is_= decompress) +{ + ACOMP_FBREQ_ON_STACK(fbreq, acomp_req); + int ret; + + if (!is_decompress) + ret =3D crypto_acomp_compress(fbreq); + else + ret =3D crypto_acomp_decompress(fbreq); + if (ret) { + pr_err("failed to do fallback work, ret=3D%d\n", ret); + return ret; + } + + acomp_req->dlen =3D fbreq->dlen; + return ret; +} + static struct hisi_zip_req *hisi_zip_create_req(struct hisi_zip_qp_ctx *qp= _ctx, struct acomp_req *req) { @@ -313,10 +332,15 @@ static int hisi_zip_acompress(struct acomp_req *acomp= _req) { struct hisi_zip_ctx *ctx =3D crypto_tfm_ctx(acomp_req->base.tfm); struct hisi_zip_qp_ctx *qp_ctx =3D &ctx->qp_ctx[HZIP_QPC_COMP]; - struct device *dev =3D &qp_ctx->qp->qm->pdev->dev; struct hisi_zip_req *req; + struct device *dev; int ret; =20 + if (ctx->fallback) + return hisi_zip_fallback_do_work(acomp_req, 0); + + dev =3D &qp_ctx->qp->qm->pdev->dev; + req =3D hisi_zip_create_req(qp_ctx, acomp_req); if (IS_ERR(req)) return PTR_ERR(req); @@ -334,10 +358,15 @@ static int hisi_zip_adecompress(struct acomp_req *aco= mp_req) { struct hisi_zip_ctx *ctx =3D crypto_tfm_ctx(acomp_req->base.tfm); struct hisi_zip_qp_ctx *qp_ctx =3D &ctx->qp_ctx[HZIP_QPC_DECOMP]; - struct device *dev =3D &qp_ctx->qp->qm->pdev->dev; struct hisi_zip_req *req; + struct device *dev; int ret; =20 + if (ctx->fallback) + return hisi_zip_fallback_do_work(acomp_req, 1); + + dev =3D &qp_ctx->qp->qm->pdev->dev; + req =3D hisi_zip_create_req(qp_ctx, acomp_req); if (IS_ERR(req)) return PTR_ERR(req); @@ -512,7 +541,7 @@ static int hisi_zip_acomp_init(struct crypto_acomp *tfm) ret =3D hisi_zip_ctx_init(ctx, COMP_NAME_TO_TYPE(alg_name), tfm->base.nod= e); if (ret) { pr_err("failed to init ctx (%d)!\n", ret); - return ret; + goto switch_to_soft; } =20 dev =3D &ctx->qp_ctx[0].qp->qm->pdev->dev; @@ -537,16 +566,20 @@ static int hisi_zip_acomp_init(struct crypto_acomp *t= fm) hisi_zip_release_req_q(ctx); err_ctx_exit: hisi_zip_ctx_exit(ctx); - return ret; +switch_to_soft: + ctx->fallback =3D true; + return 0; } =20 static void hisi_zip_acomp_exit(struct crypto_acomp *tfm) { struct hisi_zip_ctx *ctx =3D crypto_tfm_ctx(&tfm->base); =20 - hisi_zip_release_sgl_pool(ctx); - hisi_zip_release_req_q(ctx); - hisi_zip_ctx_exit(ctx); + if (!ctx->fallback) { + hisi_zip_release_sgl_pool(ctx); + hisi_zip_release_req_q(ctx); + hisi_zip_ctx_exit(ctx); + } } =20 static struct acomp_alg hisi_zip_acomp_deflate =3D { @@ -557,7 +590,8 @@ static struct acomp_alg hisi_zip_acomp_deflate =3D { .base =3D { .cra_name =3D "deflate", .cra_driver_name =3D "hisi-deflate-acomp", - .cra_flags =3D CRYPTO_ALG_ASYNC, + .cra_flags =3D CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, .cra_module =3D THIS_MODULE, .cra_priority =3D HZIP_ALG_PRIORITY, .cra_ctxsize =3D sizeof(struct hisi_zip_ctx), --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout06.his.huawei.com (canpmsgout06.his.huawei.com [113.46.200.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22EAC2F49FD; Sat, 22 Nov 2025 07:49:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.221 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797773; cv=none; b=D2mLgSISKttE+DJwmd2OpCwMptUqmjy7eRVyonyIQaS5ylkd8neVIJ/03Z0LXYJTc4Br367J3ViwATPS6ORh3wubdTfmzgCO4EIugzuGLvGw+l7BYePYqncwFkhdMS4ntQw6xfcgZ1Q1JLNKfF7SLO7gUPZL5RLTwNsB2YhqAhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797773; c=relaxed/simple; bh=yr0ZXn13CINwy9rHCCcnJPrmodqcDnw4nR1KXqK8+6s=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UEe1A+HjHKsCjO+AwUwmk/wqLo4wBRaHvuUGp01g+U69LwE76JiQjzWripIO2uRwB76lcFQOF62e+MQcEp33PVZNjQTToI+xtRiRTF3H0eVFJfNstHj93LK6jo0wGiJceF7ixNM4e/1TWsLE69MLwfgdXl4kURZo6vAlFuqvvgA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=WsgJUcHU; arc=none smtp.client-ip=113.46.200.221 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="WsgJUcHU" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=RvGsiRPrcfloe4O4c5Ztr1+tn4vf0Hnpkkn8mbqVKJ8=; b=WsgJUcHUTKBLjbVBIpH+Ogkhpa1psl+AN5TkKvNROYsSoXH0yoywOJux9+1WMmjhm5GDG//VV /KWdaEZ1K9UVMcZc67bJMs6hFePtqfX9zQbIVhuoW+29PQCJ/sAZvzZCl3QLUcE9voAvUfQj7Lb IfPnromUBOTQfKDscA07A44= Received: from mail.maildlp.com (unknown [172.19.88.105]) by canpmsgout06.his.huawei.com (SkyGuard) with ESMTPS id 4dD41j1fNPzRhRN; Sat, 22 Nov 2025 15:47:37 +0800 (CST) Received: from dggemv706-chm.china.huawei.com (unknown [10.3.19.33]) by mail.maildlp.com (Postfix) with ESMTPS id 8F9D71401F2; Sat, 22 Nov 2025 15:49:22 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv706-chm.china.huawei.com (10.3.19.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:22 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:21 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 10/11] crypto: hisilicon/hpre - support the hpre algorithm fallback Date: Sat, 22 Nov 2025 15:49:15 +0800 Message-ID: <20251122074916.2793717-11-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" From: Weili Qian When all hardware queues are busy and no shareable queue, new processes fail to apply for queues. To avoid affecting tasks, support fallback mechanism when hardware queues are unavailable. HPRE driver supports DH algorithm, limited to prime numbers up to 4K. It supports prime numbers larger than 4K via fallback mechanism. Fixes: 05e7b906aa7c ("crypto: hisilicon/hpre - add 'ECDH' algorithm") Signed-off-by: Weili Qian Signed-off-by: Chenghai Huang --- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 238 ++++++++++++++++---- 1 file changed, 199 insertions(+), 39 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/h= isilicon/hpre/hpre_crypto.c index f410e610eaba..839c1f677143 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -93,6 +93,7 @@ struct hpre_dh_ctx { =20 char *g; /* m */ dma_addr_t dma_g; + struct crypto_kpp *soft_tfm; }; =20 struct hpre_ecdh_ctx { @@ -103,6 +104,7 @@ struct hpre_ecdh_ctx { /* low address: x->y */ unsigned char *g; dma_addr_t dma_g; + struct crypto_kpp *soft_tfm; }; =20 struct hpre_ctx { @@ -120,6 +122,7 @@ struct hpre_ctx { unsigned int curve_id; /* for high performance core */ u8 enable_hpcore; + bool fallback; }; =20 struct hpre_asym_request { @@ -382,8 +385,10 @@ static int hpre_ctx_init(struct hpre_ctx *ctx, u8 type) struct hpre *hpre; =20 qp =3D hpre_create_qp(type); - if (!qp) + if (!qp) { + ctx->qp =3D NULL; return -ENODEV; + } =20 qp->req_cb =3D hpre_alg_cb; ctx->qp =3D qp; @@ -509,6 +514,48 @@ static int hpre_dh_compute_value(struct kpp_request *r= eq) return ret; } =20 +static struct kpp_request *hpre_dh_prepare_fb_req(struct kpp_request *req) +{ + struct kpp_request *fb_req =3D kpp_request_ctx(req); + struct crypto_kpp *tfm =3D crypto_kpp_reqtfm(req); + struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); + + kpp_request_set_tfm(fb_req, ctx->dh.soft_tfm); + kpp_request_set_callback(fb_req, req->base.flags, req->base.complete, req= ->base.data); + kpp_request_set_input(fb_req, req->src, req->src_len); + kpp_request_set_output(fb_req, req->dst, req->dst_len); + + return fb_req; +} + +static int hpre_dh_generate_public_key(struct kpp_request *req) +{ + struct crypto_kpp *tfm =3D crypto_kpp_reqtfm(req); + struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); + struct kpp_request *fb_req; + + if (ctx->fallback) { + fb_req =3D hpre_dh_prepare_fb_req(req); + return crypto_kpp_generate_public_key(fb_req); + } + + return hpre_dh_compute_value(req); +} + +static int hpre_dh_compute_shared_secret(struct kpp_request *req) +{ + struct crypto_kpp *tfm =3D crypto_kpp_reqtfm(req); + struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); + struct kpp_request *fb_req; + + if (ctx->fallback) { + fb_req =3D hpre_dh_prepare_fb_req(req); + return crypto_kpp_compute_shared_secret(fb_req); + } + + return hpre_dh_compute_value(req); +} + static int hpre_is_dh_params_length_valid(unsigned int key_sz) { #define _HPRE_DH_GRP1 768 @@ -535,13 +582,6 @@ static int hpre_dh_set_params(struct hpre_ctx *ctx, st= ruct dh *params) struct device *dev =3D ctx->dev; unsigned int sz; =20 - if (params->p_size > HPRE_DH_MAX_P_SZ) - return -EINVAL; - - if (hpre_is_dh_params_length_valid(params->p_size << - HPRE_BITS_2_BYTES_SHIFT)) - return -EINVAL; - sz =3D ctx->key_sz =3D params->p_size; ctx->dh.xa_p =3D dma_alloc_coherent(dev, sz << 1, &ctx->dh.dma_xa_p, GFP_KERNEL); @@ -574,6 +614,9 @@ static void hpre_dh_clear_ctx(struct hpre_ctx *ctx, boo= l is_clear_all) struct device *dev =3D ctx->dev; unsigned int sz =3D ctx->key_sz; =20 + if (!ctx->qp) + return; + if (ctx->dh.g) { dma_free_coherent(dev, sz, ctx->dh.g, ctx->dh.dma_g); ctx->dh.g =3D NULL; @@ -599,6 +642,13 @@ static int hpre_dh_set_secret(struct crypto_kpp *tfm, = const void *buf, if (crypto_dh_decode_key(buf, len, ¶ms) < 0) return -EINVAL; =20 + if (!ctx->qp) + goto set_soft_secret; + + if (hpre_is_dh_params_length_valid(params.p_size << + HPRE_BITS_2_BYTES_SHIFT)) + goto set_soft_secret; + /* Free old secret if any */ hpre_dh_clear_ctx(ctx, false); =20 @@ -609,27 +659,55 @@ static int hpre_dh_set_secret(struct crypto_kpp *tfm,= const void *buf, memcpy(ctx->dh.xa_p + (ctx->key_sz - params.key_size), params.key, params.key_size); =20 + ctx->fallback =3D false; return 0; =20 err_clear_ctx: hpre_dh_clear_ctx(ctx, false); return ret; +set_soft_secret: + ctx->fallback =3D true; + return crypto_kpp_set_secret(ctx->dh.soft_tfm, buf, len); } =20 static unsigned int hpre_dh_max_size(struct crypto_kpp *tfm) { struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); =20 + if (ctx->fallback) + return crypto_kpp_maxsize(ctx->dh.soft_tfm); + return ctx->key_sz; } =20 static int hpre_dh_init_tfm(struct crypto_kpp *tfm) { struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); + const char *alg =3D kpp_alg_name(tfm); + unsigned int reqsize; + int ret; + + ctx->dh.soft_tfm =3D crypto_alloc_kpp(alg, 0, CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->dh.soft_tfm)) { + pr_err("Failed to alloc dh tfm!\n"); + return PTR_ERR(ctx->dh.soft_tfm); + } + + crypto_kpp_set_flags(ctx->dh.soft_tfm, crypto_kpp_get_flags(tfm)); + + reqsize =3D max(sizeof(struct hpre_asym_request) + hpre_align_pd(), + sizeof(struct kpp_request) + crypto_kpp_reqsize(ctx->dh.soft_tfm)); + kpp_set_reqsize(tfm, reqsize); =20 - kpp_set_reqsize(tfm, sizeof(struct hpre_asym_request) + hpre_align_pd()); + ret =3D hpre_ctx_init(ctx, HPRE_V2_ALG_TYPE); + if (ret && ret !=3D -ENODEV) { + crypto_free_kpp(ctx->dh.soft_tfm); + return ret; + } else if (ret =3D=3D -ENODEV) { + ctx->fallback =3D true; + } =20 - return hpre_ctx_init(ctx, HPRE_V2_ALG_TYPE); + return 0; } =20 static void hpre_dh_exit_tfm(struct crypto_kpp *tfm) @@ -637,6 +715,7 @@ static void hpre_dh_exit_tfm(struct crypto_kpp *tfm) struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); =20 hpre_dh_clear_ctx(ctx, true); + crypto_free_kpp(ctx->dh.soft_tfm); } =20 static void hpre_rsa_drop_leading_zeros(const char **ptr, size_t *len) @@ -676,9 +755,8 @@ static int hpre_rsa_enc(struct akcipher_request *req) struct hpre_sqe *msg =3D &hpre_req->req; int ret; =20 - /* For 512 and 1536 bits key size, use soft tfm instead */ - if (ctx->key_sz =3D=3D HPRE_RSA_512BITS_KSZ || - ctx->key_sz =3D=3D HPRE_RSA_1536BITS_KSZ) { + /* For unsupported key size and unavailable devices, use soft tfm instead= */ + if (ctx->fallback) { akcipher_request_set_tfm(req, ctx->rsa.soft_tfm); ret =3D crypto_akcipher_encrypt(req); akcipher_request_set_tfm(req, tfm); @@ -723,9 +801,8 @@ static int hpre_rsa_dec(struct akcipher_request *req) struct hpre_sqe *msg =3D &hpre_req->req; int ret; =20 - /* For 512 and 1536 bits key size, use soft tfm instead */ - if (ctx->key_sz =3D=3D HPRE_RSA_512BITS_KSZ || - ctx->key_sz =3D=3D HPRE_RSA_1536BITS_KSZ) { + /* For unsupported key size and unavailable devices, use soft tfm instead= */ + if (ctx->fallback) { akcipher_request_set_tfm(req, ctx->rsa.soft_tfm); ret =3D crypto_akcipher_decrypt(req); akcipher_request_set_tfm(req, tfm); @@ -778,8 +855,10 @@ static int hpre_rsa_set_n(struct hpre_ctx *ctx, const = char *value, ctx->key_sz =3D vlen; =20 /* if invalid key size provided, we use software tfm */ - if (!hpre_rsa_key_size_is_support(ctx->key_sz)) + if (!hpre_rsa_key_size_is_support(ctx->key_sz)) { + ctx->fallback =3D true; return 0; + } =20 ctx->rsa.pubkey =3D dma_alloc_coherent(ctx->dev, vlen << 1, &ctx->rsa.dma_pubkey, @@ -914,6 +993,9 @@ static void hpre_rsa_clear_ctx(struct hpre_ctx *ctx, bo= ol is_clear_all) unsigned int half_key_sz =3D ctx->key_sz >> 1; struct device *dev =3D ctx->dev; =20 + if (!ctx->qp) + return; + if (ctx->rsa.pubkey) { dma_free_coherent(dev, ctx->key_sz << 1, ctx->rsa.pubkey, ctx->rsa.dma_pubkey); @@ -993,6 +1075,7 @@ static int hpre_rsa_setkey(struct hpre_ctx *ctx, const= void *key, goto free; } =20 + ctx->fallback =3D false; return 0; =20 free: @@ -1010,6 +1093,9 @@ static int hpre_rsa_setpubkey(struct crypto_akcipher = *tfm, const void *key, if (ret) return ret; =20 + if (!ctx->qp) + return 0; + return hpre_rsa_setkey(ctx, key, keylen, false); } =20 @@ -1023,6 +1109,9 @@ static int hpre_rsa_setprivkey(struct crypto_akcipher= *tfm, const void *key, if (ret) return ret; =20 + if (!ctx->qp) + return 0; + return hpre_rsa_setkey(ctx, key, keylen, true); } =20 @@ -1030,9 +1119,8 @@ static unsigned int hpre_rsa_max_size(struct crypto_a= kcipher *tfm) { struct hpre_ctx *ctx =3D akcipher_tfm_ctx(tfm); =20 - /* For 512 and 1536 bits key size, use soft tfm instead */ - if (ctx->key_sz =3D=3D HPRE_RSA_512BITS_KSZ || - ctx->key_sz =3D=3D HPRE_RSA_1536BITS_KSZ) + /* For unsupported key size and unavailable devices, use soft tfm instead= */ + if (ctx->fallback) return crypto_akcipher_maxsize(ctx->rsa.soft_tfm); =20 return ctx->key_sz; @@ -1053,10 +1141,14 @@ static int hpre_rsa_init_tfm(struct crypto_akcipher= *tfm) hpre_align_pd()); =20 ret =3D hpre_ctx_init(ctx, HPRE_V2_ALG_TYPE); - if (ret) + if (ret && ret !=3D -ENODEV) { crypto_free_akcipher(ctx->rsa.soft_tfm); + return ret; + } else if (ret =3D=3D -ENODEV) { + ctx->fallback =3D true; + } =20 - return ret; + return 0; } =20 static void hpre_rsa_exit_tfm(struct crypto_akcipher *tfm) @@ -1260,6 +1352,9 @@ static int hpre_ecdh_set_secret(struct crypto_kpp *tf= m, const void *buf, struct ecdh params; int ret; =20 + if (ctx->fallback) + return crypto_kpp_set_secret(ctx->ecdh.soft_tfm, buf, len); + if (crypto_ecdh_decode_key(buf, len, ¶ms) < 0) { dev_err(dev, "failed to decode ecdh key!\n"); return -EINVAL; @@ -1485,23 +1580,82 @@ static int hpre_ecdh_compute_value(struct kpp_reque= st *req) return ret; } =20 +static int hpre_ecdh_generate_public_key(struct kpp_request *req) +{ + struct crypto_kpp *tfm =3D crypto_kpp_reqtfm(req); + struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); + int ret; + + if (ctx->fallback) { + kpp_request_set_tfm(req, ctx->ecdh.soft_tfm); + ret =3D crypto_kpp_generate_public_key(req); + kpp_request_set_tfm(req, tfm); + return ret; + } + + return hpre_ecdh_compute_value(req); +} + +static int hpre_ecdh_compute_shared_secret(struct kpp_request *req) +{ + struct crypto_kpp *tfm =3D crypto_kpp_reqtfm(req); + struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); + int ret; + + if (ctx->fallback) { + kpp_request_set_tfm(req, ctx->ecdh.soft_tfm); + ret =3D crypto_kpp_compute_shared_secret(req); + kpp_request_set_tfm(req, tfm); + return ret; + } + + return hpre_ecdh_compute_value(req); +} + static unsigned int hpre_ecdh_max_size(struct crypto_kpp *tfm) { struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); =20 + if (ctx->fallback) + return crypto_kpp_maxsize(ctx->ecdh.soft_tfm); + /* max size is the pub_key_size, include x and y */ return ctx->key_sz << 1; } =20 +static int hpre_ecdh_init_tfm(struct crypto_kpp *tfm) +{ + struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); + const char *alg =3D kpp_alg_name(tfm); + int ret; + + ret =3D hpre_ctx_init(ctx, HPRE_V3_ECC_ALG_TYPE); + if (!ret) { + kpp_set_reqsize(tfm, sizeof(struct hpre_asym_request) + hpre_align_pd()); + return 0; + } else if (ret && ret !=3D -ENODEV) { + return ret; + } + + ctx->ecdh.soft_tfm =3D crypto_alloc_kpp(alg, 0, CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->ecdh.soft_tfm)) { + pr_err("Failed to alloc %s tfm!\n", alg); + return PTR_ERR(ctx->ecdh.soft_tfm); + } + + crypto_kpp_set_flags(ctx->ecdh.soft_tfm, crypto_kpp_get_flags(tfm)); + ctx->fallback =3D true; + + return 0; +} + static int hpre_ecdh_nist_p192_init_tfm(struct crypto_kpp *tfm) { struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); =20 ctx->curve_id =3D ECC_CURVE_NIST_P192; =20 - kpp_set_reqsize(tfm, sizeof(struct hpre_asym_request) + hpre_align_pd()); - - return hpre_ctx_init(ctx, HPRE_V3_ECC_ALG_TYPE); + return hpre_ecdh_init_tfm(tfm); } =20 static int hpre_ecdh_nist_p256_init_tfm(struct crypto_kpp *tfm) @@ -1511,9 +1665,7 @@ static int hpre_ecdh_nist_p256_init_tfm(struct crypto= _kpp *tfm) ctx->curve_id =3D ECC_CURVE_NIST_P256; ctx->enable_hpcore =3D 1; =20 - kpp_set_reqsize(tfm, sizeof(struct hpre_asym_request) + hpre_align_pd()); - - return hpre_ctx_init(ctx, HPRE_V3_ECC_ALG_TYPE); + return hpre_ecdh_init_tfm(tfm); } =20 static int hpre_ecdh_nist_p384_init_tfm(struct crypto_kpp *tfm) @@ -1522,15 +1674,18 @@ static int hpre_ecdh_nist_p384_init_tfm(struct cryp= to_kpp *tfm) =20 ctx->curve_id =3D ECC_CURVE_NIST_P384; =20 - kpp_set_reqsize(tfm, sizeof(struct hpre_asym_request) + hpre_align_pd()); - - return hpre_ctx_init(ctx, HPRE_V3_ECC_ALG_TYPE); + return hpre_ecdh_init_tfm(tfm); } =20 static void hpre_ecdh_exit_tfm(struct crypto_kpp *tfm) { struct hpre_ctx *ctx =3D kpp_tfm_ctx(tfm); =20 + if (ctx->fallback) { + crypto_free_kpp(ctx->ecdh.soft_tfm); + return; + } + hpre_ecc_clear_ctx(ctx, true); } =20 @@ -1548,13 +1703,14 @@ static struct akcipher_alg rsa =3D { .cra_name =3D "rsa", .cra_driver_name =3D "hpre-rsa", .cra_module =3D THIS_MODULE, + .cra_flags =3D CRYPTO_ALG_NEED_FALLBACK, }, }; =20 static struct kpp_alg dh =3D { .set_secret =3D hpre_dh_set_secret, - .generate_public_key =3D hpre_dh_compute_value, - .compute_shared_secret =3D hpre_dh_compute_value, + .generate_public_key =3D hpre_dh_generate_public_key, + .compute_shared_secret =3D hpre_dh_compute_shared_secret, .max_size =3D hpre_dh_max_size, .init =3D hpre_dh_init_tfm, .exit =3D hpre_dh_exit_tfm, @@ -1564,14 +1720,15 @@ static struct kpp_alg dh =3D { .cra_name =3D "dh", .cra_driver_name =3D "hpre-dh", .cra_module =3D THIS_MODULE, + .cra_flags =3D CRYPTO_ALG_NEED_FALLBACK, }, }; =20 static struct kpp_alg ecdh_curves[] =3D { { .set_secret =3D hpre_ecdh_set_secret, - .generate_public_key =3D hpre_ecdh_compute_value, - .compute_shared_secret =3D hpre_ecdh_compute_value, + .generate_public_key =3D hpre_ecdh_generate_public_key, + .compute_shared_secret =3D hpre_ecdh_compute_shared_secret, .max_size =3D hpre_ecdh_max_size, .init =3D hpre_ecdh_nist_p192_init_tfm, .exit =3D hpre_ecdh_exit_tfm, @@ -1581,11 +1738,12 @@ static struct kpp_alg ecdh_curves[] =3D { .cra_name =3D "ecdh-nist-p192", .cra_driver_name =3D "hpre-ecdh-nist-p192", .cra_module =3D THIS_MODULE, + .cra_flags =3D CRYPTO_ALG_NEED_FALLBACK, }, }, { .set_secret =3D hpre_ecdh_set_secret, - .generate_public_key =3D hpre_ecdh_compute_value, - .compute_shared_secret =3D hpre_ecdh_compute_value, + .generate_public_key =3D hpre_ecdh_generate_public_key, + .compute_shared_secret =3D hpre_ecdh_compute_shared_secret, .max_size =3D hpre_ecdh_max_size, .init =3D hpre_ecdh_nist_p256_init_tfm, .exit =3D hpre_ecdh_exit_tfm, @@ -1595,11 +1753,12 @@ static struct kpp_alg ecdh_curves[] =3D { .cra_name =3D "ecdh-nist-p256", .cra_driver_name =3D "hpre-ecdh-nist-p256", .cra_module =3D THIS_MODULE, + .cra_flags =3D CRYPTO_ALG_NEED_FALLBACK, }, }, { .set_secret =3D hpre_ecdh_set_secret, - .generate_public_key =3D hpre_ecdh_compute_value, - .compute_shared_secret =3D hpre_ecdh_compute_value, + .generate_public_key =3D hpre_ecdh_generate_public_key, + .compute_shared_secret =3D hpre_ecdh_compute_shared_secret, .max_size =3D hpre_ecdh_max_size, .init =3D hpre_ecdh_nist_p384_init_tfm, .exit =3D hpre_ecdh_exit_tfm, @@ -1609,6 +1768,7 @@ static struct kpp_alg ecdh_curves[] =3D { .cra_name =3D "ecdh-nist-p384", .cra_driver_name =3D "hpre-ecdh-nist-p384", .cra_module =3D THIS_MODULE, + .cra_flags =3D CRYPTO_ALG_NEED_FALLBACK, }, } }; --=20 2.33.0 From nobody Tue Dec 2 01:05:44 2025 Received: from canpmsgout06.his.huawei.com (canpmsgout06.his.huawei.com [113.46.200.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2B7A2F39BD; Sat, 22 Nov 2025 07:49:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.221 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797773; cv=none; b=BM77CpfPA9faFYHOu3JoPQVAa4dwf1WAPt/V4vsNslps5+4fRZN6DYyl0qMNKjr43VLzKGVgCUzRPQpsESZbgmBpbVjOeqrrQggnIEas7QJUtd4wq0rIO8irxc/egfxP8vPO5t9HnVuMdlw02Vp8Z4CtbY804S5zFdefl7LMm4Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763797773; c=relaxed/simple; bh=iu+yAidXrHEug3y3V9cSpWGIIW9hNPdKZFe5IKKkHfE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nBM/IqKLvazfqfjPfJudH9/9hp1qhmKVM9CcYoZRQXhCq5Uh9K0IteUIqRYIg1gzW16N99zvmUxA06U8zP7O4mulARpqO1QInP3EaEAND3YXM/1kZPZDgPG3MQb8w1EZFMNE7GbKUObG7miN2LhjKdXYY7CMkmnyd5mE89L7owA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=aNm0EI8b; arc=none smtp.client-ip=113.46.200.221 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="aNm0EI8b" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=JoI7cyZ+NnFFAdScLBhjUEGoGhJJ+1wW9RIBEXNcA3Y=; b=aNm0EI8brhTaJm3jSNjdbHdNkb5ow1UZcG99+MwrSeJbIZ9cCva6u/8z8fb3ji99BZdEvtqyH UY/qJ5bgt8MNwI1wTeX2HUZBhqJkdxP6FJsOzP+y6/QGTYomapIoBNxu/Hi/W0wDoh5Yfla0I0+ Dv6UUONR6PwJDEna5Ct8oH8= Received: from mail.maildlp.com (unknown [172.19.163.174]) by canpmsgout06.his.huawei.com (SkyGuard) with ESMTPS id 4dD41j4kMkzRhRN; Sat, 22 Nov 2025 15:47:37 +0800 (CST) Received: from dggemv712-chm.china.huawei.com (unknown [10.1.198.32]) by mail.maildlp.com (Postfix) with ESMTPS id 08522140203; Sat, 22 Nov 2025 15:49:23 +0800 (CST) Received: from kwepemq200001.china.huawei.com (7.202.195.16) by dggemv712-chm.china.huawei.com (10.1.198.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:22 +0800 Received: from localhost.huawei.com (10.90.31.46) by kwepemq200001.china.huawei.com (7.202.195.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 22 Nov 2025 15:49:22 +0800 From: Chenghai Huang To: , CC: , , , , , , , , Subject: [PATCH v3 11/11] crypto: hisilicon/sec2 - support skcipher/aead fallback for hardware queue unavailable Date: Sat, 22 Nov 2025 15:49:16 +0800 Message-ID: <20251122074916.2793717-12-huangchenghai2@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20251122074916.2793717-1-huangchenghai2@huawei.com> References: <20251122074916.2793717-1-huangchenghai2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To kwepemq200001.china.huawei.com (7.202.195.16) Content-Type: text/plain; charset="utf-8" From: Qi Tao When all hardware queues are busy and no shareable queue, new processes fail to apply for queues. To avoid affecting tasks, support fallback mechanism when hardware queues are unavailable. Fixes: c16a70c1f253 ("crypto: hisilicon/sec - add new algorithm mode for AE= AD") Signed-off-by: Qi Tao Signed-off-by: Chenghai Huang --- drivers/crypto/hisilicon/sec2/sec_crypto.c | 62 ++++++++++++++++------ 1 file changed, 47 insertions(+), 15 deletions(-) diff --git a/drivers/crypto/hisilicon/sec2/sec_crypto.c b/drivers/crypto/hi= silicon/sec2/sec_crypto.c index d09d081f42dc..c462b58d3034 100644 --- a/drivers/crypto/hisilicon/sec2/sec_crypto.c +++ b/drivers/crypto/hisilicon/sec2/sec_crypto.c @@ -663,10 +663,8 @@ static int sec_ctx_base_init(struct sec_ctx *ctx) int i, ret; =20 ctx->qps =3D sec_create_qps(); - if (!ctx->qps) { - pr_err("Can not create sec qps!\n"); + if (!ctx->qps) return -ENODEV; - } =20 sec =3D container_of(ctx->qps[0]->qm, struct sec_dev, qm); ctx->sec =3D sec; @@ -702,6 +700,9 @@ static void sec_ctx_base_uninit(struct sec_ctx *ctx) { int i; =20 + if (!ctx->qps) + return; + for (i =3D 0; i < ctx->sec->ctx_q_num; i++) sec_release_qp_ctx(ctx, &ctx->qp_ctx[i]); =20 @@ -713,6 +714,9 @@ static int sec_cipher_init(struct sec_ctx *ctx) { struct sec_cipher_ctx *c_ctx =3D &ctx->c_ctx; =20 + if (!ctx->qps) + return 0; + c_ctx->c_key =3D dma_alloc_coherent(ctx->dev, SEC_MAX_KEY_SIZE, &c_ctx->c_key_dma, GFP_KERNEL); if (!c_ctx->c_key) @@ -725,6 +729,9 @@ static void sec_cipher_uninit(struct sec_ctx *ctx) { struct sec_cipher_ctx *c_ctx =3D &ctx->c_ctx; =20 + if (!ctx->qps) + return; + memzero_explicit(c_ctx->c_key, SEC_MAX_KEY_SIZE); dma_free_coherent(ctx->dev, SEC_MAX_KEY_SIZE, c_ctx->c_key, c_ctx->c_key_dma); @@ -746,6 +753,9 @@ static void sec_auth_uninit(struct sec_ctx *ctx) { struct sec_auth_ctx *a_ctx =3D &ctx->a_ctx; =20 + if (!ctx->qps) + return; + memzero_explicit(a_ctx->a_key, SEC_MAX_AKEY_SIZE); dma_free_coherent(ctx->dev, SEC_MAX_AKEY_SIZE, a_ctx->a_key, a_ctx->a_key_dma); @@ -783,7 +793,7 @@ static int sec_skcipher_init(struct crypto_skcipher *tf= m) } =20 ret =3D sec_ctx_base_init(ctx); - if (ret) + if (ret && ret !=3D -ENODEV) return ret; =20 ret =3D sec_cipher_init(ctx); @@ -892,6 +902,9 @@ static int sec_skcipher_setkey(struct crypto_skcipher *= tfm, const u8 *key, struct device *dev =3D ctx->dev; int ret; =20 + if (!ctx->qps) + goto set_soft_key; + if (c_mode =3D=3D SEC_CMODE_XTS) { ret =3D xts_verify_key(tfm, key, keylen); if (ret) { @@ -922,13 +935,14 @@ static int sec_skcipher_setkey(struct crypto_skcipher= *tfm, const u8 *key, } =20 memcpy(c_ctx->c_key, key, keylen); - if (c_ctx->fbtfm) { - ret =3D crypto_sync_skcipher_setkey(c_ctx->fbtfm, key, keylen); - if (ret) { - dev_err(dev, "failed to set fallback skcipher key!\n"); - return ret; - } + +set_soft_key: + ret =3D crypto_sync_skcipher_setkey(c_ctx->fbtfm, key, keylen); + if (ret) { + dev_err(dev, "failed to set fallback skcipher key!\n"); + return ret; } + return 0; } =20 @@ -1392,6 +1406,9 @@ static int sec_aead_setkey(struct crypto_aead *tfm, c= onst u8 *key, struct crypto_authenc_keys keys; int ret; =20 + if (!ctx->qps) + return sec_aead_fallback_setkey(a_ctx, tfm, key, keylen); + ctx->a_ctx.a_alg =3D a_alg; ctx->c_ctx.c_alg =3D c_alg; c_ctx->c_mode =3D c_mode; @@ -2048,6 +2065,9 @@ static int sec_skcipher_ctx_init(struct crypto_skciph= er *tfm) if (ret) return ret; =20 + if (!ctx->qps) + return 0; + if (ctx->sec->qm.ver < QM_HW_V3) { ctx->type_supported =3D SEC_BD_TYPE2; ctx->req_op =3D &sec_skcipher_req_ops; @@ -2056,7 +2076,7 @@ static int sec_skcipher_ctx_init(struct crypto_skciph= er *tfm) ctx->req_op =3D &sec_skcipher_req_ops_v3; } =20 - return ret; + return 0; } =20 static void sec_skcipher_ctx_exit(struct crypto_skcipher *tfm) @@ -2124,7 +2144,7 @@ static int sec_aead_ctx_init(struct crypto_aead *tfm,= const char *hash_name) int ret; =20 ret =3D sec_aead_init(tfm); - if (ret) { + if (ret && ret !=3D -ENODEV) { pr_err("hisi_sec2: aead init error!\n"); return ret; } @@ -2166,7 +2186,7 @@ static int sec_aead_xcm_ctx_init(struct crypto_aead *= tfm) int ret; =20 ret =3D sec_aead_init(tfm); - if (ret) { + if (ret && ret !=3D -ENODEV) { dev_err(ctx->dev, "hisi_sec2: aead xcm init error!\n"); return ret; } @@ -2311,6 +2331,9 @@ static int sec_skcipher_crypto(struct skcipher_reques= t *sk_req, bool encrypt) bool need_fallback =3D false; int ret; =20 + if (!ctx->qps) + goto soft_crypto; + if (!sk_req->cryptlen) { if (ctx->c_ctx.c_mode =3D=3D SEC_CMODE_XTS) return -EINVAL; @@ -2328,9 +2351,12 @@ static int sec_skcipher_crypto(struct skcipher_reque= st *sk_req, bool encrypt) return -EINVAL; =20 if (unlikely(ctx->c_ctx.fallback || need_fallback)) - return sec_skcipher_soft_crypto(ctx, sk_req, encrypt); + goto soft_crypto; =20 return ctx->req_op->process(ctx, req); + +soft_crypto: + return sec_skcipher_soft_crypto(ctx, sk_req, encrypt); } =20 static int sec_skcipher_encrypt(struct skcipher_request *sk_req) @@ -2538,6 +2564,9 @@ static int sec_aead_crypto(struct aead_request *a_req= , bool encrypt) bool need_fallback =3D false; int ret; =20 + if (!ctx->qps) + goto soft_crypto; + req->flag =3D a_req->base.flags; req->aead_req.aead_req =3D a_req; req->c_req.encrypt =3D encrypt; @@ -2548,11 +2577,14 @@ static int sec_aead_crypto(struct aead_request *a_r= eq, bool encrypt) ret =3D sec_aead_param_check(ctx, req, &need_fallback); if (unlikely(ret)) { if (need_fallback) - return sec_aead_soft_crypto(ctx, a_req, encrypt); + goto soft_crypto; return -EINVAL; } =20 return ctx->req_op->process(ctx, req); + +soft_crypto: + return sec_aead_soft_crypto(ctx, a_req, encrypt); } =20 static int sec_aead_encrypt(struct aead_request *a_req) --=20 2.33.0