From nobody Mon May 11 02:06:56 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C52AC4332F for ; Mon, 18 Apr 2022 09:04:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237399AbiDRJHX (ORCPT ); Mon, 18 Apr 2022 05:07:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237132AbiDRJHO (ORCPT ); Mon, 18 Apr 2022 05:07:14 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1121D1169 for ; Mon, 18 Apr 2022 02:04:35 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id q19so17807413pgm.6 for ; Mon, 18 Apr 2022 02:04:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MMvFUIaJUZFaJh2w5O1uB3m37o1dGCP/mCJGRZ5s3NE=; b=IBECHa46OyWpM94Z/h4vDnACfq0tVl58F+SMJ9PfXQAAMiqg2Y09tHJr0N2hoPaJ5q 4bTUHlZebBHyp3V2cptNxrnvXndYD+xsIaUvqnLbOSY+1zQGy6UzHUEgb3sqkzI+2D/h uc12HSLcFjEAYO4ddKfKGjceCIXX9GTHkenXEN4DFjbjsFn+eAtdrz8CzyJMxHeFoMSR pmfiLvNEsEYi9K+5NLPHOQ7n8GdjW9WkG2NBuF8rjVzommUw87UPeqdQiTzvUo0N48TV KBnv5L4F25f/50rHEgUZY1cyyWx+SddThQZA4PEGWI0CMCFkpmnD338sddKoy2fR+U2G kirA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MMvFUIaJUZFaJh2w5O1uB3m37o1dGCP/mCJGRZ5s3NE=; b=ZmUumdibzPQUiXuqV0ub02AzcQu8Cbfa3pvVEEgQ7TVv7/cq3Hgaf7ZKSA3mUt6Iwf TwvVA81HHjJ2flam5/FwXIyi3vLIMI/wV0q81ALprzyf6Dohk6ccPZVamHXUXIwxj5/5 sk/5MlUrFMEWWt9BTa/951vuS9yrNZTmyiqU0UmB2Lk2Ybr9LuG6Hi0+xZXjsL6JsBwj DnGdie5xcskLmMttAtuRy8ujI4FF/DB/ZbPJDm52dBbKTmVjyyz3rR+AtvTzo3ayNwjF O2sE+zuXpAsAi4sE2WQ5CnO5kT4OElZlCr5Me/0eYm692lK9isa0FycwA3oXcDlCEkZd jtMg== X-Gm-Message-State: AOAM531V3AmOEKh1JAhtg4Ri6DLCPrQaXKVhx9rJNlJPOyWO0KvBDGU3 HrTNUj+CVD7bx+HQEGlvtQ7BWg== X-Google-Smtp-Source: ABdhPJz4Nb26RqRQjtNIYYqsan0fWQfU6kBj4xxwdkTln4B7PpiksAP6pXThqAg3kASRrwDF0XSBXg== X-Received: by 2002:a05:6a00:c8d:b0:50a:62e5:6d30 with SMTP id a13-20020a056a000c8d00b0050a62e56d30mr7319810pfv.47.1650272674434; Mon, 18 Apr 2022 02:04:34 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id m21-20020a17090a7f9500b001c97c6bcaf4sm16408071pjl.39.2022.04.18.02.04.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Apr 2022 02:04:33 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net, zhenwei pi Subject: [PATCH v2 1/4] virtio-crypto: wait ctrl queue instead of busy polling Date: Mon, 18 Apr 2022 17:00:48 +0800 Message-Id: <20220418090051.372803-2-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220418090051.372803-1-pizhenwei@bytedance.com> References: <20220418090051.372803-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Originally, after submitting request into virtio crypto control queue, the guest side polls the result from the virt queue. This allows all of the requests share a single request buffer in struct virtio_crypto(ctrl&input&ctrl_status). CPU0 CPU1 ... CPUx CPUy | | | | \ \ / / \--------spin_lock(&vcrypto->ctrl_lock)-------/ | virtqueue add & kick | busy poll virtqueue | spin_unlock(&vcrypto->ctrl_lock) ... There are two problems: 1, The queue depth is always 1, the performance of a virtio crypto device gets limited. Multi user processes share a single control queue, and hit spin lock race from control queue. Test on Intel Platinum 8260, a single worker gets ~35K/s create/close session operations, and 8 workers get ~40K/s operations with 800% CPU utilization. 2, The control request is supposed to get handled immediately, but in the current implementation of QEMU(v6.2), the vCPU thread kicks another thread to do this work, the latency also gets unstable. Tracking latency of virtio_crypto_alg_akcipher_close_session in 5s: usecs : count distribution 0 -> 1 : 0 | | 2 -> 3 : 7 | | 4 -> 7 : 72 | | 8 -> 15 : 186485 |************************| 16 -> 31 : 687 | | 32 -> 63 : 5 | | 64 -> 127 : 3 | | 128 -> 255 : 1 | | 256 -> 511 : 0 | | 512 -> 1023 : 0 | | 1024 -> 2047 : 0 | | 2048 -> 4095 : 0 | | 4096 -> 8191 : 0 | | 8192 -> 16383 : 2 | | This means that a CPU may hold vcrypto->ctrl_lock as long as 8192~16383u= s. To improve the performance of control queue, remove ctrl&input&ctrl_status = from struct virtio_crypto, each request allocates request buffer dynamically, wa= its completion instead of busy polling to reduce lock racing, and gets complete= d by control queue callback. CPU0 CPU1 ... CPUx CPUy | | | | \ \ / / \--------spin_lock(&vcrypto->ctrl_lock)-------/ | virtqueue add & kick | ---------spin_unlock(&vcrypto->ctrl_lock)------ / / \ \ | | | | wait wait wait wait Test this patch, the guest side get ~200K/s operations with 300% CPU utilization. Signed-off-by: zhenwei pi --- drivers/crypto/virtio/Makefile | 1 + .../virtio/virtio_crypto_akcipher_algs.c | 90 ++++++------ drivers/crypto/virtio/virtio_crypto_common.c | 61 ++++++++ drivers/crypto/virtio/virtio_crypto_common.h | 25 +++- drivers/crypto/virtio/virtio_crypto_core.c | 2 +- .../virtio/virtio_crypto_skcipher_algs.c | 133 ++++++++---------- 6 files changed, 185 insertions(+), 127 deletions(-) create mode 100644 drivers/crypto/virtio/virtio_crypto_common.c diff --git a/drivers/crypto/virtio/Makefile b/drivers/crypto/virtio/Makefile index bfa6cbae342e..49c1fa80e465 100644 --- a/drivers/crypto/virtio/Makefile +++ b/drivers/crypto/virtio/Makefile @@ -3,5 +3,6 @@ obj-$(CONFIG_CRYPTO_DEV_VIRTIO) +=3D virtio_crypto.o virtio_crypto-objs :=3D \ virtio_crypto_skcipher_algs.o \ virtio_crypto_akcipher_algs.o \ + virtio_crypto_common.o \ virtio_crypto_mgr.o \ virtio_crypto_core.o diff --git a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c b/drivers/= crypto/virtio/virtio_crypto_akcipher_algs.c index f3ec9420215e..9561bc2df62b 100644 --- a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c +++ b/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c @@ -102,8 +102,8 @@ static int virtio_crypto_alg_akcipher_init_session(stru= ct virtio_crypto_akcipher { struct scatterlist outhdr_sg, key_sg, inhdr_sg, *sgs[3]; struct virtio_crypto *vcrypto =3D ctx->vcrypto; + struct virtio_crypto_ctrl_request *vc_ctrl_req; uint8_t *pkey; - unsigned int inlen; int err; unsigned int num_out =3D 0, num_in =3D 0; =20 @@ -111,98 +111,91 @@ static int virtio_crypto_alg_akcipher_init_session(st= ruct virtio_crypto_akcipher if (!pkey) return -ENOMEM; =20 - spin_lock(&vcrypto->ctrl_lock); - memcpy(&vcrypto->ctrl.header, header, sizeof(vcrypto->ctrl.header)); - memcpy(&vcrypto->ctrl.u, para, sizeof(vcrypto->ctrl.u)); - vcrypto->input.status =3D cpu_to_le32(VIRTIO_CRYPTO_ERR); + vc_ctrl_req =3D kzalloc(sizeof(*vc_ctrl_req), GFP_KERNEL); + if (!vc_ctrl_req) { + err =3D -ENOMEM; + goto out; + } =20 - sg_init_one(&outhdr_sg, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + memcpy(&vc_ctrl_req->ctrl.header, header, sizeof(vc_ctrl_req->ctrl.header= )); + memcpy(&vc_ctrl_req->ctrl.u, para, sizeof(vc_ctrl_req->ctrl.u)); + sg_init_one(&outhdr_sg, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] =3D &outhdr_sg; =20 sg_init_one(&key_sg, pkey, keylen); sgs[num_out++] =3D &key_sg; =20 - sg_init_one(&inhdr_sg, &vcrypto->input, sizeof(vcrypto->input)); + vc_ctrl_req->input.status =3D cpu_to_le32(VIRTIO_CRYPTO_ERR); + sg_init_one(&inhdr_sg, &vc_ctrl_req->input, sizeof(vc_ctrl_req->input)); sgs[num_out + num_in++] =3D &inhdr_sg; =20 - err =3D virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, num_in, vcrypto= , GFP_ATOMIC); + err =3D virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_c= trl_req); if (err < 0) goto out; =20 - virtqueue_kick(vcrypto->ctrl_vq); - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &inlen) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); - - if (le32_to_cpu(vcrypto->input.status) !=3D VIRTIO_CRYPTO_OK) { + if (le32_to_cpu(vc_ctrl_req->input.status) !=3D VIRTIO_CRYPTO_OK) { + pr_err("virtio_crypto: Create session failed status: %u\n", + le32_to_cpu(vc_ctrl_req->input.status)); err =3D -EINVAL; goto out; } =20 - ctx->session_id =3D le64_to_cpu(vcrypto->input.session_id); + ctx->session_id =3D le64_to_cpu(vc_ctrl_req->input.session_id); ctx->session_valid =3D true; err =3D 0; =20 out: - spin_unlock(&vcrypto->ctrl_lock); + kfree(vc_ctrl_req); kfree_sensitive(pkey); =20 - if (err < 0) - pr_err("virtio_crypto: Create session failed status: %u\n", - le32_to_cpu(vcrypto->input.status)); - return err; } =20 static int virtio_crypto_alg_akcipher_close_session(struct virtio_crypto_a= kcipher_ctx *ctx) { struct scatterlist outhdr_sg, inhdr_sg, *sgs[2]; + struct virtio_crypto_ctrl_request *vc_ctrl_req; struct virtio_crypto_destroy_session_req *destroy_session; struct virtio_crypto *vcrypto =3D ctx->vcrypto; - unsigned int num_out =3D 0, num_in =3D 0, inlen; + unsigned int num_out =3D 0, num_in =3D 0; int err; =20 - spin_lock(&vcrypto->ctrl_lock); - if (!ctx->session_valid) { - err =3D 0; - goto out; - } - vcrypto->ctrl_status.status =3D VIRTIO_CRYPTO_ERR; - vcrypto->ctrl.header.opcode =3D cpu_to_le32(VIRTIO_CRYPTO_AKCIPHER_DESTRO= Y_SESSION); - vcrypto->ctrl.header.queue_id =3D 0; + if (!ctx->session_valid) + return 0; =20 - destroy_session =3D &vcrypto->ctrl.u.destroy_session; + vc_ctrl_req =3D kzalloc(sizeof(*vc_ctrl_req), GFP_KERNEL); + if (!vc_ctrl_req) + return -ENOMEM; + + vc_ctrl_req->ctrl.header.opcode =3D cpu_to_le32(VIRTIO_CRYPTO_AKCIPHER_DE= STROY_SESSION); + vc_ctrl_req->ctrl.header.queue_id =3D 0; + + destroy_session =3D &vc_ctrl_req->ctrl.u.destroy_session; destroy_session->session_id =3D cpu_to_le64(ctx->session_id); =20 - sg_init_one(&outhdr_sg, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + sg_init_one(&outhdr_sg, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] =3D &outhdr_sg; =20 - sg_init_one(&inhdr_sg, &vcrypto->ctrl_status.status, sizeof(vcrypto->ctrl= _status.status)); + vc_ctrl_req->ctrl_status.status =3D VIRTIO_CRYPTO_ERR; + sg_init_one(&inhdr_sg, &vc_ctrl_req->ctrl_status.status, + sizeof(vc_ctrl_req->ctrl_status.status)); sgs[num_out + num_in++] =3D &inhdr_sg; =20 - err =3D virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, num_in, vcrypto= , GFP_ATOMIC); + err =3D virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_c= trl_req); if (err < 0) goto out; =20 - virtqueue_kick(vcrypto->ctrl_vq); - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &inlen) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); - - if (vcrypto->ctrl_status.status !=3D VIRTIO_CRYPTO_OK) { + if (vc_ctrl_req->ctrl_status.status !=3D VIRTIO_CRYPTO_OK) { err =3D -EINVAL; + pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%l= lx\n", + vc_ctrl_req->ctrl_status.status, destroy_session->session_id); goto out; } =20 err =3D 0; ctx->session_valid =3D false; - out: - spin_unlock(&vcrypto->ctrl_lock); - if (err < 0) { - pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%l= lx\n", - vcrypto->ctrl_status.status, destroy_session->session_id); - } + kfree(vc_ctrl_req); =20 return err; } @@ -210,14 +203,11 @@ static int virtio_crypto_alg_akcipher_close_session(s= truct virtio_crypto_akciphe static int __virtio_crypto_akcipher_do_req(struct virtio_crypto_akcipher_r= equest *vc_akcipher_req, struct akcipher_request *req, struct data_queue *data_vq) { - struct virtio_crypto_akcipher_ctx *ctx =3D vc_akcipher_req->akcipher_ctx; struct virtio_crypto_request *vc_req =3D &vc_akcipher_req->base; - struct virtio_crypto *vcrypto =3D ctx->vcrypto; struct virtio_crypto_op_data_req *req_data =3D vc_req->req_data; struct scatterlist *sgs[4], outhdr_sg, inhdr_sg, srcdata_sg, dstdata_sg; void *src_buf =3D NULL, *dst_buf =3D NULL; unsigned int num_out =3D 0, num_in =3D 0; - int node =3D dev_to_node(&vcrypto->vdev->dev); unsigned long flags; int ret =3D -ENOMEM; bool verify =3D vc_akcipher_req->opcode =3D=3D VIRTIO_CRYPTO_AKCIPHER_VER= IFY; @@ -228,7 +218,7 @@ static int __virtio_crypto_akcipher_do_req(struct virti= o_crypto_akcipher_request sgs[num_out++] =3D &outhdr_sg; =20 /* src data */ - src_buf =3D kcalloc_node(src_len, 1, GFP_KERNEL, node); + src_buf =3D kcalloc(src_len, 1, GFP_KERNEL); if (!src_buf) goto err; =20 @@ -243,7 +233,7 @@ static int __virtio_crypto_akcipher_do_req(struct virti= o_crypto_akcipher_request sgs[num_out++] =3D &srcdata_sg; =20 /* dst data */ - dst_buf =3D kcalloc_node(req->dst_len, 1, GFP_KERNEL, node); + dst_buf =3D kcalloc(req->dst_len, 1, GFP_KERNEL); if (!dst_buf) goto err; =20 diff --git a/drivers/crypto/virtio/virtio_crypto_common.c b/drivers/crypto/= virtio/virtio_crypto_common.c new file mode 100644 index 000000000000..93df73c40dd3 --- /dev/null +++ b/drivers/crypto/virtio/virtio_crypto_common.c @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* Common functions and helpers + * + * Authors: zhenwei pi + * + * Copyright 2022 Bytedance CO., LTD. + */ + +#include "virtio_crypto_common.h" + +static void virtio_crypto_ctrlq_callback(struct virtio_crypto_ctrl_request= *vc_ctrl_req) +{ + complete(&vc_ctrl_req->compl); +} + +int virtio_crypto_ctrl_vq_request(struct virtio_crypto *vcrypto, struct sc= atterlist *sgs[], + unsigned int out_sgs, unsigned int in_sgs, + struct virtio_crypto_ctrl_request *vc_ctrl_req) +{ + int err; + unsigned long flags; + + init_completion(&vc_ctrl_req->compl); + vc_ctrl_req->ctrl_cb =3D virtio_crypto_ctrlq_callback; + + spin_lock_irqsave(&vcrypto->ctrl_lock, flags); + err =3D virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, out_sgs, in_sgs, vc_ctrl= _req, GFP_ATOMIC); + if (err < 0) { + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); + return err; + } + + virtqueue_kick(vcrypto->ctrl_vq); + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); + + wait_for_completion(&vc_ctrl_req->compl); + + return 0; +} + +void virtcrypto_ctrlq_callback(struct virtqueue *vq) +{ + struct virtio_crypto *vcrypto =3D vq->vdev->priv; + struct virtio_crypto_ctrl_request *vc_ctrl_req; + unsigned long flags; + unsigned int len; + + spin_lock_irqsave(&vcrypto->ctrl_lock, flags); + do { + virtqueue_disable_cb(vq); + while ((vc_ctrl_req =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); + if (vc_ctrl_req->ctrl_cb) + vc_ctrl_req->ctrl_cb(vc_ctrl_req); + spin_lock_irqsave(&vcrypto->ctrl_lock, flags); + } + if (unlikely(virtqueue_is_broken(vq))) + break; + } while (!virtqueue_enable_cb(vq)); + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); +} diff --git a/drivers/crypto/virtio/virtio_crypto_common.h b/drivers/crypto/= virtio/virtio_crypto_common.h index e693d4ee83a6..b90ac0ce30d2 100644 --- a/drivers/crypto/virtio/virtio_crypto_common.h +++ b/drivers/crypto/virtio/virtio_crypto_common.h @@ -13,6 +13,7 @@ #include #include #include +#include =20 =20 /* Internal representation of a data virtqueue */ @@ -65,11 +66,6 @@ struct virtio_crypto { /* Maximum size of per request */ u64 max_size; =20 - /* Control VQ buffers: protected by the ctrl_lock */ - struct virtio_crypto_op_ctrl_req ctrl; - struct virtio_crypto_session_input input; - struct virtio_crypto_inhdr ctrl_status; - unsigned long status; atomic_t ref_count; struct list_head list; @@ -85,6 +81,20 @@ struct virtio_crypto_sym_session_info { __u64 session_id; }; =20 +struct virtio_crypto_ctrl_request; +typedef void (*virtio_crypto_ctrl_callback) + (struct virtio_crypto_ctrl_request *vc_ctrl_req); + +/* Note: there are padding fields in request, clear them to zero before se= nding to host, + * Ex, virtio_crypto_ctrl_request::ctrl::u::destroy_session::padding[48] */ +struct virtio_crypto_ctrl_request { + struct virtio_crypto_op_ctrl_req ctrl; + struct virtio_crypto_session_input input; + struct virtio_crypto_inhdr ctrl_status; + virtio_crypto_ctrl_callback ctrl_cb; + struct completion compl; +}; + struct virtio_crypto_request; typedef void (*virtio_crypto_data_callback) (struct virtio_crypto_request *vc_req, int len); @@ -135,4 +145,9 @@ void virtio_crypto_skcipher_algs_unregister(struct virt= io_crypto *vcrypto); int virtio_crypto_akcipher_algs_register(struct virtio_crypto *vcrypto); void virtio_crypto_akcipher_algs_unregister(struct virtio_crypto *vcrypto); =20 +void virtcrypto_ctrlq_callback(struct virtqueue *vq); +int virtio_crypto_ctrl_vq_request(struct virtio_crypto *vcrypto, struct sc= atterlist *sgs[], + unsigned int out_sgs, unsigned int in_sgs, + struct virtio_crypto_ctrl_request *vc_ctrl_req); + #endif /* _VIRTIO_CRYPTO_COMMON_H */ diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/vi= rtio/virtio_crypto_core.c index c6f482db0bc0..e668d4b1bc6a 100644 --- a/drivers/crypto/virtio/virtio_crypto_core.c +++ b/drivers/crypto/virtio/virtio_crypto_core.c @@ -73,7 +73,7 @@ static int virtcrypto_find_vqs(struct virtio_crypto *vi) goto err_names; =20 /* Parameters for control virtqueue */ - callbacks[total_vqs - 1] =3D NULL; + callbacks[total_vqs - 1] =3D virtcrypto_ctrlq_callback; names[total_vqs - 1] =3D "controlq"; =20 /* Allocate/initialize parameters for data virtqueues */ diff --git a/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c b/drivers/= crypto/virtio/virtio_crypto_skcipher_algs.c index a618c46a52b8..fef355ff461c 100644 --- a/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c +++ b/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c @@ -118,11 +118,13 @@ static int virtio_crypto_alg_skcipher_init_session( int encrypt) { struct scatterlist outhdr, key_sg, inhdr, *sgs[3]; - unsigned int tmp; struct virtio_crypto *vcrypto =3D ctx->vcrypto; int op =3D encrypt ? VIRTIO_CRYPTO_OP_ENCRYPT : VIRTIO_CRYPTO_OP_DECRYPT; int err; unsigned int num_out =3D 0, num_in =3D 0; + struct virtio_crypto_ctrl_request *vc_ctrl_req; + struct virtio_crypto_ctrl_header *header; + struct virtio_crypto_sym_create_session_req *sym_create_session; =20 /* * Avoid to do DMA from the stack, switch to using @@ -133,26 +135,27 @@ static int virtio_crypto_alg_skcipher_init_session( if (!cipher_key) return -ENOMEM; =20 - spin_lock(&vcrypto->ctrl_lock); + vc_ctrl_req =3D kzalloc(sizeof(*vc_ctrl_req), GFP_KERNEL); + if (!vc_ctrl_req) { + err =3D -ENOMEM; + goto out; + } + /* Pad ctrl header */ - vcrypto->ctrl.header.opcode =3D - cpu_to_le32(VIRTIO_CRYPTO_CIPHER_CREATE_SESSION); - vcrypto->ctrl.header.algo =3D cpu_to_le32(alg); + header =3D &vc_ctrl_req->ctrl.header; + header->opcode =3D cpu_to_le32(VIRTIO_CRYPTO_CIPHER_CREATE_SESSION); + header->algo =3D cpu_to_le32(alg); /* Set the default dataqueue id to 0 */ - vcrypto->ctrl.header.queue_id =3D 0; + header->queue_id =3D 0; =20 - vcrypto->input.status =3D cpu_to_le32(VIRTIO_CRYPTO_ERR); /* Pad cipher's parameters */ - vcrypto->ctrl.u.sym_create_session.op_type =3D - cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER); - vcrypto->ctrl.u.sym_create_session.u.cipher.para.algo =3D - vcrypto->ctrl.header.algo; - vcrypto->ctrl.u.sym_create_session.u.cipher.para.keylen =3D - cpu_to_le32(keylen); - vcrypto->ctrl.u.sym_create_session.u.cipher.para.op =3D - cpu_to_le32(op); - - sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + sym_create_session =3D &vc_ctrl_req->ctrl.u.sym_create_session; + sym_create_session->op_type =3D cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER); + sym_create_session->u.cipher.para.algo =3D header->algo; + sym_create_session->u.cipher.para.keylen =3D cpu_to_le32(keylen); + sym_create_session->u.cipher.para.op =3D cpu_to_le32(op); + + sg_init_one(&outhdr, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] =3D &outhdr; =20 /* Set key */ @@ -160,45 +163,34 @@ static int virtio_crypto_alg_skcipher_init_session( sgs[num_out++] =3D &key_sg; =20 /* Return status and session id back */ - sg_init_one(&inhdr, &vcrypto->input, sizeof(vcrypto->input)); + vc_ctrl_req->input.status =3D cpu_to_le32(VIRTIO_CRYPTO_ERR); + sg_init_one(&inhdr, &vc_ctrl_req->input, sizeof(vc_ctrl_req->input)); sgs[num_out + num_in++] =3D &inhdr; =20 - err =3D virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, - num_in, vcrypto, GFP_ATOMIC); - if (err < 0) { - spin_unlock(&vcrypto->ctrl_lock); - kfree_sensitive(cipher_key); - return err; - } - virtqueue_kick(vcrypto->ctrl_vq); - - /* - * Trapping into the hypervisor, so the request should be - * handled immediately. - */ - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); + err =3D virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_c= trl_req); + if (err < 0) + goto out; =20 - if (le32_to_cpu(vcrypto->input.status) !=3D VIRTIO_CRYPTO_OK) { - spin_unlock(&vcrypto->ctrl_lock); + if (le32_to_cpu(vc_ctrl_req->input.status) !=3D VIRTIO_CRYPTO_OK) { pr_err("virtio_crypto: Create session failed status: %u\n", - le32_to_cpu(vcrypto->input.status)); - kfree_sensitive(cipher_key); - return -EINVAL; + le32_to_cpu(vc_ctrl_req->input.status)); + err =3D -EINVAL; + goto out; } =20 if (encrypt) ctx->enc_sess_info.session_id =3D - le64_to_cpu(vcrypto->input.session_id); + le64_to_cpu(vc_ctrl_req->input.session_id); else ctx->dec_sess_info.session_id =3D - le64_to_cpu(vcrypto->input.session_id); - - spin_unlock(&vcrypto->ctrl_lock); + le64_to_cpu(vc_ctrl_req->input.session_id); =20 + err =3D 0; +out: + kfree(vc_ctrl_req); kfree_sensitive(cipher_key); - return 0; + + return err; } =20 static int virtio_crypto_alg_skcipher_close_session( @@ -206,21 +198,24 @@ static int virtio_crypto_alg_skcipher_close_session( int encrypt) { struct scatterlist outhdr, status_sg, *sgs[2]; - unsigned int tmp; struct virtio_crypto_destroy_session_req *destroy_session; struct virtio_crypto *vcrypto =3D ctx->vcrypto; int err; unsigned int num_out =3D 0, num_in =3D 0; + struct virtio_crypto_ctrl_request *vc_ctrl_req; + struct virtio_crypto_ctrl_header *header; + + vc_ctrl_req =3D kzalloc(sizeof(*vc_ctrl_req), GFP_KERNEL); + if (!vc_ctrl_req) + return -ENOMEM; =20 - spin_lock(&vcrypto->ctrl_lock); - vcrypto->ctrl_status.status =3D VIRTIO_CRYPTO_ERR; /* Pad ctrl header */ - vcrypto->ctrl.header.opcode =3D - cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION); + header =3D &vc_ctrl_req->ctrl.header; + header->opcode =3D cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION); /* Set the default virtqueue id to 0 */ - vcrypto->ctrl.header.queue_id =3D 0; + header->queue_id =3D 0; =20 - destroy_session =3D &vcrypto->ctrl.u.destroy_session; + destroy_session =3D &vc_ctrl_req->ctrl.u.destroy_session; =20 if (encrypt) destroy_session->session_id =3D @@ -229,37 +224,33 @@ static int virtio_crypto_alg_skcipher_close_session( destroy_session->session_id =3D cpu_to_le64(ctx->dec_sess_info.session_id); =20 - sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + sg_init_one(&outhdr, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] =3D &outhdr; =20 /* Return status and session id back */ - sg_init_one(&status_sg, &vcrypto->ctrl_status.status, - sizeof(vcrypto->ctrl_status.status)); + vc_ctrl_req->ctrl_status.status =3D VIRTIO_CRYPTO_ERR; + sg_init_one(&status_sg, &vc_ctrl_req->ctrl_status.status, + sizeof(vc_ctrl_req->ctrl_status.status)); sgs[num_out + num_in++] =3D &status_sg; =20 - err =3D virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, - num_in, vcrypto, GFP_ATOMIC); - if (err < 0) { - spin_unlock(&vcrypto->ctrl_lock); - return err; - } - virtqueue_kick(vcrypto->ctrl_vq); - - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); + err =3D virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_c= trl_req); + if (err < 0) + goto out; =20 - if (vcrypto->ctrl_status.status !=3D VIRTIO_CRYPTO_OK) { - spin_unlock(&vcrypto->ctrl_lock); + if (vc_ctrl_req->ctrl_status.status !=3D VIRTIO_CRYPTO_OK) { pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%l= lx\n", - vcrypto->ctrl_status.status, + vc_ctrl_req->ctrl_status.status, destroy_session->session_id); =20 - return -EINVAL; + err =3D -EINVAL; + goto out; } - spin_unlock(&vcrypto->ctrl_lock); =20 - return 0; + err =3D 0; +out: + kfree(vc_ctrl_req); + + return err; } =20 static int virtio_crypto_alg_skcipher_init_sessions( --=20 2.20.1 From nobody Mon May 11 02:06:56 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F1F5C433FE for ; Mon, 18 Apr 2022 09:05:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237434AbiDRJH2 (ORCPT ); Mon, 18 Apr 2022 05:07:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237405AbiDRJHV (ORCPT ); Mon, 18 Apr 2022 05:07:21 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7554415FC8 for ; Mon, 18 Apr 2022 02:04:39 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id md4so12510590pjb.4 for ; Mon, 18 Apr 2022 02:04:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t/X33EVipdTkyOBkLhSrJ+Mg2HbAbb6q5VYpCkTBi/E=; b=P39Vpj74J1EUV1LZWKg7ijX1J7sLbeV4w1bJO63YJrnaC80C2vbK4KNvwEqhKecXkg pbTRgxKyapfmMDY2+Zl4VukLGABNVoCzDwkTugB1BU2SLJZ94zI5OFGJMVZ6XG0toF7b SsqdNp43PWEjmcVyDp1gHOToKJ2UCxYEJnBNwqJFI45n9MbOMgOdXOHldN2qz0FOjCHA U8fYoNxJD96RdFkUiSPKsKhJEgC3iYbCr3Eup0YT4UrkcR+OtXeRHbPjNBj7irixqjp6 fSEKPRV76xqpGIUj4MjzshPeMbhp2eyTNPYjwGnI39Px3wVe4miL1d/vZjhvWqepGTsR IufA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t/X33EVipdTkyOBkLhSrJ+Mg2HbAbb6q5VYpCkTBi/E=; b=1co7u9YbAazgZ7YDcEPGhRdBgtyRLO4rr5XEXNGmQihcprfuqNv2tdEwpjnU3t2LF8 E7o6AR1GICKYp4gMfhzLxN1oc8I+B57GGMUDVBFvkBlswmblQiwgXdoAUQz8e87jBZTy 6zzRVgVpLL1bIjG8mGcR5gi6OZ+5tBQCY06mrN0i/k6QaD9ypUxRfxwrdXlShfRPKCs5 hUKqD+PpI8IV11LR5bl5My/T0o3CMIbYOthODWvkqaPw6gI1RirJS4HB0OzCs7Rs6Q9t 3GMt/dBVOxM9B9+xR8IzczY6ax+0rIThxE0ttWftyrooK1ybcjEw9BvFIFmJhHxEuaH5 gk+g== X-Gm-Message-State: AOAM5305sXh5z9RbSlDSV73TisKrYeyDjRsYjt/tIxoWcgHbqB4eFcM6 Eg23/nA/U8yw5/+oPk90RyLngA== X-Google-Smtp-Source: ABdhPJxBF6Kvfmj7zAWFA5qnv4v0KI+bJQ1IuDkiWEMFez7bDk10Hxz6uXKToPPEJyvyPwKi1x2fzQ== X-Received: by 2002:a17:902:b410:b0:158:faef:79b0 with SMTP id x16-20020a170902b41000b00158faef79b0mr4310497plr.40.1650272678930; Mon, 18 Apr 2022 02:04:38 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id m21-20020a17090a7f9500b001c97c6bcaf4sm16408071pjl.39.2022.04.18.02.04.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Apr 2022 02:04:38 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net, zhenwei pi Subject: [PATCH v2 2/4] virtio-crypto: move helpers into virtio_crypto_common.c Date: Mon, 18 Apr 2022 17:00:49 +0800 Message-Id: <20220418090051.372803-3-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220418090051.372803-1-pizhenwei@bytedance.com> References: <20220418090051.372803-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Move virtcrypto_clear_request and virtcrypto_dataq_callback into virtio_crypto_common.c to make code clear. Then the xx_core.c supports: - probe/remove/irq affinity seting for a virtio device - basic virtio related operations xx_common.c supports: - common helpers/functions for algos Signed-off-by: zhenwei pi --- drivers/crypto/virtio/virtio_crypto_common.c | 31 +++++++++++++++++++ drivers/crypto/virtio/virtio_crypto_common.h | 2 ++ drivers/crypto/virtio/virtio_crypto_core.c | 32 -------------------- 3 files changed, 33 insertions(+), 32 deletions(-) diff --git a/drivers/crypto/virtio/virtio_crypto_common.c b/drivers/crypto/= virtio/virtio_crypto_common.c index 93df73c40dd3..4a23524896fe 100644 --- a/drivers/crypto/virtio/virtio_crypto_common.c +++ b/drivers/crypto/virtio/virtio_crypto_common.c @@ -8,6 +8,14 @@ =20 #include "virtio_crypto_common.h" =20 +void virtcrypto_clear_request(struct virtio_crypto_request *vc_req) +{ + if (vc_req) { + kfree_sensitive(vc_req->req_data); + kfree(vc_req->sgs); + } +} + static void virtio_crypto_ctrlq_callback(struct virtio_crypto_ctrl_request= *vc_ctrl_req) { complete(&vc_ctrl_req->compl); @@ -59,3 +67,26 @@ void virtcrypto_ctrlq_callback(struct virtqueue *vq) } while (!virtqueue_enable_cb(vq)); spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); } + +void virtcrypto_dataq_callback(struct virtqueue *vq) +{ + struct virtio_crypto *vcrypto =3D vq->vdev->priv; + struct virtio_crypto_request *vc_req; + unsigned long flags; + unsigned int len; + unsigned int qid =3D vq->index; + + spin_lock_irqsave(&vcrypto->data_vq[qid].lock, flags); + do { + virtqueue_disable_cb(vq); + while ((vc_req =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { + spin_unlock_irqrestore( + &vcrypto->data_vq[qid].lock, flags); + if (vc_req->alg_cb) + vc_req->alg_cb(vc_req, len); + spin_lock_irqsave( + &vcrypto->data_vq[qid].lock, flags); + } + } while (!virtqueue_enable_cb(vq)); + spin_unlock_irqrestore(&vcrypto->data_vq[qid].lock, flags); +} diff --git a/drivers/crypto/virtio/virtio_crypto_common.h b/drivers/crypto/= virtio/virtio_crypto_common.h index b90ac0ce30d2..b33c0d4423eb 100644 --- a/drivers/crypto/virtio/virtio_crypto_common.h +++ b/drivers/crypto/virtio/virtio_crypto_common.h @@ -150,4 +150,6 @@ int virtio_crypto_ctrl_vq_request(struct virtio_crypto = *vcrypto, struct scatterl unsigned int out_sgs, unsigned int in_sgs, struct virtio_crypto_ctrl_request *vc_ctrl_req); =20 +void virtcrypto_dataq_callback(struct virtqueue *vq); + #endif /* _VIRTIO_CRYPTO_COMMON_H */ diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/vi= rtio/virtio_crypto_core.c index e668d4b1bc6a..d8edefcb966c 100644 --- a/drivers/crypto/virtio/virtio_crypto_core.c +++ b/drivers/crypto/virtio/virtio_crypto_core.c @@ -13,38 +13,6 @@ #include "virtio_crypto_common.h" =20 =20 -void -virtcrypto_clear_request(struct virtio_crypto_request *vc_req) -{ - if (vc_req) { - kfree_sensitive(vc_req->req_data); - kfree(vc_req->sgs); - } -} - -static void virtcrypto_dataq_callback(struct virtqueue *vq) -{ - struct virtio_crypto *vcrypto =3D vq->vdev->priv; - struct virtio_crypto_request *vc_req; - unsigned long flags; - unsigned int len; - unsigned int qid =3D vq->index; - - spin_lock_irqsave(&vcrypto->data_vq[qid].lock, flags); - do { - virtqueue_disable_cb(vq); - while ((vc_req =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { - spin_unlock_irqrestore( - &vcrypto->data_vq[qid].lock, flags); - if (vc_req->alg_cb) - vc_req->alg_cb(vc_req, len); - spin_lock_irqsave( - &vcrypto->data_vq[qid].lock, flags); - } - } while (!virtqueue_enable_cb(vq)); - spin_unlock_irqrestore(&vcrypto->data_vq[qid].lock, flags); -} - static int virtcrypto_find_vqs(struct virtio_crypto *vi) { vq_callback_t **callbacks; --=20 2.20.1 From nobody Mon May 11 02:06:56 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FE3BC433EF for ; Mon, 18 Apr 2022 09:05:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237471AbiDRJHe (ORCPT ); Mon, 18 Apr 2022 05:07:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237402AbiDRJHW (ORCPT ); Mon, 18 Apr 2022 05:07:22 -0400 Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C4FF13DFC for ; Mon, 18 Apr 2022 02:04:44 -0700 (PDT) Received: by mail-pl1-x635.google.com with SMTP id t12so11867602pll.7 for ; Mon, 18 Apr 2022 02:04:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qGycGFvEc/grEUwBLjN4FrsuSMK10Ph1TJa/0rHLB00=; b=SyqgyxdXATvl1tqV7yph+r4Ov29FnV+LuEYRD3cNUQbaEnKLSOmMXI7R9KN2yxV6OR LjumbVKsA9Bzn676oEXHvjHFXu1CDjVvtFd6wJxz/NOtDrfukBxwmRP2wwHWScH+hGgh hbWTgSDqaLGqkLhNPNYBZNqiR2ZnSJ5R3Mft1zvXK05rWjuGXmJMH3srjGuX4D834G/E YrASL7u82Pj5BEv1wNW/7TMbIO+uC9JFCvPe6cwD0wYb9NXyFVAw20WUL8v7oQrlwg+y ZoF45RADFCEW+dDKIM+plRx6Bloo7DSGc6vK6v/Ckv2g3S++SN80sfYi5E0N+MDY4bSi SKsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qGycGFvEc/grEUwBLjN4FrsuSMK10Ph1TJa/0rHLB00=; b=UwKTa/1F7Bjc1CkO/eDbAPcWmDi9hcVoh0tW0KGxNgOblx1OE7ny0g1cPzyhgkQ+PJ DqGQSNiio9wa9r2CVQNYeFjbvRJI10YhuIGcJOB+g70Sf/yYOJeodb3N8JhnW5PufNZR 78XXqD5yVWgLLErLf6Wzu2QxhxPeiBHJJ7bUyv7NfqW4Zkd3iQsFka8cz2+x01xXgDFu O0zgYBmKRUrk4AF/bpt4nXOppY3P8WWRlILxfa5Vvk4ttkEJDu+TsCdjh09q/FdIIMDt SV/8WYje+6lyxnHxVFtL/6YUeRi8NKNfFgVpW6k7WS1ScRHi6BkqVbK4rt14r6M1SZu7 KOSg== X-Gm-Message-State: AOAM533F4+diTREY3SNfX9CCMgIZOZnSzAioCGUH0D+Eip6LDEfB9+PO To7SwTQMNTg91C5ajPaSXT34dg== X-Google-Smtp-Source: ABdhPJzl0keGE9CDZ9JEYQSKw/oJDMmcnJw0PJyud+pyqmk6lmwwYeDI+kw+cMA+ac6eAWEccD+O7g== X-Received: by 2002:a17:903:40ce:b0:158:8178:8563 with SMTP id t14-20020a17090340ce00b0015881788563mr9800299pld.167.1650272683819; Mon, 18 Apr 2022 02:04:43 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id m21-20020a17090a7f9500b001c97c6bcaf4sm16408071pjl.39.2022.04.18.02.04.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Apr 2022 02:04:43 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net, lei he , zhenwei pi Subject: [PATCH v2 3/4] virtio-crypto: adjust dst_len at ops callback Date: Mon, 18 Apr 2022 17:00:50 +0800 Message-Id: <20220418090051.372803-4-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220418090051.372803-1-pizhenwei@bytedance.com> References: <20220418090051.372803-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: lei he For some akcipher operations(eg, decryption of pkcs1pad(rsa)), the length of returned result maybe less than akcipher_req->dst_len, we need to recalculate the actual dst_len through the virt-queue protocol. Signed-off-by: lei he Signed-off-by: zhenwei pi --- drivers/crypto/virtio/virtio_crypto_akcipher_algs.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c b/drivers/= crypto/virtio/virtio_crypto_akcipher_algs.c index 9561bc2df62b..82db86e088c2 100644 --- a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c +++ b/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c @@ -90,9 +90,12 @@ static void virtio_crypto_dataq_akcipher_callback(struct= virtio_crypto_request * } =20 akcipher_req =3D vc_akcipher_req->akcipher_req; - if (vc_akcipher_req->opcode !=3D VIRTIO_CRYPTO_AKCIPHER_VERIFY) + if (vc_akcipher_req->opcode !=3D VIRTIO_CRYPTO_AKCIPHER_VERIFY) { + /* actuall length maybe less than dst buffer */ + akcipher_req->dst_len =3D len - sizeof(vc_req->status); sg_copy_from_buffer(akcipher_req->dst, sg_nents(akcipher_req->dst), vc_akcipher_req->dst_buf, akcipher_req->dst_len); + } virtio_crypto_akcipher_finalize_req(vc_akcipher_req, akcipher_req, error); } =20 --=20 2.20.1 From nobody Mon May 11 02:06:56 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9209C433EF for ; Mon, 18 Apr 2022 09:05:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236179AbiDRJHq (ORCPT ); Mon, 18 Apr 2022 05:07:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237443AbiDRJH3 (ORCPT ); Mon, 18 Apr 2022 05:07:29 -0400 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3536F75 for ; Mon, 18 Apr 2022 02:04:48 -0700 (PDT) Received: by mail-pg1-x52f.google.com with SMTP id t13so17797742pgn.8 for ; Mon, 18 Apr 2022 02:04:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ouz4DgKg6HX7//LGRrUD97CctBPhdyzV/9EeFTiygxk=; b=6TWHPFn0LN6u1Gtv/IJ/FZUtIkxZGYRSlYL1xUuSQwz6ytLfUxUHzEhiZogx/oPjrb 5IOYrTilRPWoHpCRSg3pD9hxYZzJxQhgFu7oMUXgRdkrpSBD8H8p9b2MjNWFFbRV/0fk JqtHVC/Atk4oTpbgL809sb6GZ48czjib99D2vLy8PKSaX5Q4H53WLL5nFIul/nggKvzB QyCC5TIlf1UwqjmQzzFfYQICmDC+k+WILzeWkCdunjIwypfSFlv0ZFqUS9k7G5oLbxB2 6oUiQeUOEd1aB+8ADGQyVZYhbF9eyXJK0CCrBZfsSUMAUUktSQsB/HwqoQFmhdVi8f4r H9mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ouz4DgKg6HX7//LGRrUD97CctBPhdyzV/9EeFTiygxk=; b=wWcTDgn61HpCDnl4VhtD6PX23c+9xsGahBjEDHdRbNK7sYbGuAhIxH3Hhsut2UobgU 4QKfmEvop2w1GnT42JPnhTsitQPKkoSBCDtD1FseRTgMRbLQnLPDyj9GOzsV4St17BIp 75cuBV1CH9uaWb7MgJpkpvJ98rF2s+S4k07mmxqwnh6T+Mj0aE2PFjvEkD/FytazkfHM gJ+Fl9RRs21Bp0nQ2u9upd9IM16ZKIPQMS7wVusvrwKIyAyl7WKD3SMpjIVemOgCXBzN 3MKPBJxuMpnmFeJ6HLi2695WGCCU1kf7gApNhPd2aelRlWiOp/uCIEPGSTk4Fa6ECgJA TXrw== X-Gm-Message-State: AOAM532ptoGCOe3XyTE/2slmcfjYBa6Fhz1ThLjZUzC/XKqEvCEc6KPu B1yTsDz8+bQQsFbvVYkK/z3huA== X-Google-Smtp-Source: ABdhPJz7+O43bQjw3Qx8+8pHjS0RYIso9TqOCGaesGbw3hn8vk87HPcB2H/bsqdEGcpfWIhWqakxhg== X-Received: by 2002:a63:34e:0:b0:39d:8d96:a947 with SMTP id 75-20020a63034e000000b0039d8d96a947mr9503958pgd.105.1650272688180; Mon, 18 Apr 2022 02:04:48 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id m21-20020a17090a7f9500b001c97c6bcaf4sm16408071pjl.39.2022.04.18.02.04.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Apr 2022 02:04:47 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net Subject: [PATCH v2 4/4] virtio-crypto: enable retry for virtio-crypto-dev Date: Mon, 18 Apr 2022 17:00:51 +0800 Message-Id: <20220418090051.372803-5-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220418090051.372803-1-pizhenwei@bytedance.com> References: <20220418090051.372803-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: lei he Enable retry for virtio-crypto-dev, so that crypto-engine can process cipher-requests parallelly. Signed-off-by: lei he --- drivers/crypto/virtio/virtio_crypto_core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/vi= rtio/virtio_crypto_core.c index d8edefcb966c..5c0d68c9e894 100644 --- a/drivers/crypto/virtio/virtio_crypto_core.c +++ b/drivers/crypto/virtio/virtio_crypto_core.c @@ -62,7 +62,8 @@ static int virtcrypto_find_vqs(struct virtio_crypto *vi) spin_lock_init(&vi->data_vq[i].lock); vi->data_vq[i].vq =3D vqs[i]; /* Initialize crypto engine */ - vi->data_vq[i].engine =3D crypto_engine_alloc_init(dev, 1); + vi->data_vq[i].engine =3D crypto_engine_alloc_init_and_set(dev, true, NU= LL, 1, + virtqueue_get_vring_size(vqs[i])); if (!vi->data_vq[i].engine) { ret =3D -ENOMEM; goto err_engine; --=20 2.20.1