From nobody Sun May 19 10:57:24 2024 Received: from mx07-00178001.pphosted.com (mx08-00178001.pphosted.com [91.207.212.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27A1713BAFE; Tue, 23 Apr 2024 15:15:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.207.212.93 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713885335; cv=none; b=FSGFzsMzGJdxX2AlEKExlL7l+oatKGbLhdtuQYCGlHVTDUISV4ZhCXqcAdtA6t1war7piPD34ImHndFrkOsI888gCoyE0lnZFn4FtqpvVI0KECrY0X86PfOtoRJPHhbB0Jj5Xf23Di9itBLogjsN+3QXSrxN9krA3Gm4axzyLGo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713885335; c=relaxed/simple; bh=8xiXjiuTl3fYLE98vBH4rNJx6fJjwP2A7LNiXM1faG8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VzTXIxVQdxQNt0no0OyK5+xB6UoITwvdyfbf7/3Z0jhymibQzZMvZUKlWbiBXWAubwdFiQ5dk11/INfVrdlTnDAVQ0tU0ZQGt26RmjEXthU13tuCfSWDKze75wYctme9nTDll/JhjvdixzjqoniHAx+YgJYR5JMhWfuvLNZEUO4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=foss.st.com; spf=pass smtp.mailfrom=foss.st.com; dkim=pass (2048-bit key) header.d=foss.st.com header.i=@foss.st.com header.b=uMjW3WxN; arc=none smtp.client-ip=91.207.212.93 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=foss.st.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=foss.st.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=foss.st.com header.i=@foss.st.com header.b="uMjW3WxN" Received: from pps.filterd (m0046661.ppops.net [127.0.0.1]) by mx07-00178001.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 43NCd6kr020338; Tue, 23 Apr 2024 17:15:00 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foss.st.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s= selector1; bh=cbrWtY4kM4DuZWl/qNMK+PX0vMMkExlCZHo21VFxfWM=; b=uM jW3WxNPaSUERLQCWS1fmSE7IkXL/guVeeX/VLTUplL80wkD53+Ixm9td27+5FFD5 cEvdAZk7j6IA8S0xIuXwBuuI+GpTvosMrgk0JTmutAWAcdowwekEN3v5IrfcKWSE 4PoofpehL6ycshvpMjemBhdulkD1YljAkekXUskhwMLIYrcV5JOr9UTtcU3k2r+y UFVrhFTIkYg2ZI1b5lXBpv9yfji4/I38aD2M01NOMRNnLm3efC+FyocbrHx34Mvc 1vhN0NVmnjZAX+JTBpyX07aDJYZfa71Z5frKOY9zN91Zb1jaXUGp3oZU2ghiDoYH DdOy+RUM1AgzDmbgtmCw== Received: from beta.dmz-ap.st.com (beta.dmz-ap.st.com [138.198.100.35]) by mx07-00178001.pphosted.com (PPS) with ESMTPS id 3xm4kb3ytr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 23 Apr 2024 17:15:00 +0200 (MEST) Received: from euls16034.sgp.st.com (euls16034.sgp.st.com [10.75.44.20]) by beta.dmz-ap.st.com (STMicroelectronics) with ESMTP id 85F4640046; Tue, 23 Apr 2024 17:14:55 +0200 (CEST) Received: from Webmail-eu.st.com (shfdag1node3.st.com [10.75.129.71]) by euls16034.sgp.st.com (STMicroelectronics) with ESMTP id 4642D223660; Tue, 23 Apr 2024 17:14:15 +0200 (CEST) Received: from localhost (10.48.86.103) by SHFDAG1NODE3.st.com (10.75.129.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 23 Apr 2024 17:14:14 +0200 From: Maxime MERE To: Herbert Xu , "David S . Miller" , Maxime Coquelin , Alexandre Torgue CC: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= , Rob Herring , , , , Subject: [PATCH v2 1/3] crypto: stm32/cryp - use dma when possible. Date: Tue, 23 Apr 2024 17:14:07 +0200 Message-ID: <20240423151409.902490-2-maxime.mere@foss.st.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240423151409.902490-1-maxime.mere@foss.st.com> References: <20240423151409.902490-1-maxime.mere@foss.st.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: EQNCAS1NODE4.st.com (10.75.129.82) To SHFDAG1NODE3.st.com (10.75.129.71) X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-23_12,2024-04-23_02,2023-05-22_02 From: Maxime M=C3=A9r=C3=A9 Use DMA when buffer are aligned and with expected size. If buffer are correctly aligned and bigger than 1KB we have some performance gain: With DMA enable: $ openssl speed -evp aes-256-cbc -engine afalg -elapsed The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 byt= es 16384 bytes aes-256-cbc 120.02k 406.78k 1588.82k 5873.32k 26020.= 52k 34258.94k Without DMA: $ openssl speed -evp aes-256-cbc -engine afalg -elapsed The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 byt= es 16384 bytes aes-256-cbc 121.06k 419.95k 1112.23k 1897.47k 2362.= 03k 2386.60k With DMA: extract of $ modprobe tcrypt mode=3D500 testing speed of async cbc(aes) (stm32-cbc-aes) encryption tcrypt: test 14 (256 bit key, 16 byte blocks): 1 operation in 1679 cycle= s (16 bytes) tcrypt: test 15 (256 bit key, 64 byte blocks): 1 operation in 1893 cycle= s (64 bytes) tcrypt: test 16 (256 bit key, 128 byte blocks): 1 operation in 1760 cycle= s (128 bytes) tcrypt: test 17 (256 bit key, 256 byte blocks): 1 operation in 2154 cycle= s (256 bytes) tcrypt: test 18 (256 bit key, 1024 byte blocks): 1 operation in 2132 cycle= s (1024 bytes) tcrypt: test 19 (256 bit key, 1424 byte blocks): 1 operation in 2466 cycle= s (1424 bytes) tcrypt: test 20 (256 bit key, 4096 byte blocks): 1 operation in 4040 cycle= s (4096 bytes) Without DMA: $ modprobe tcrypt mode=3D500 tcrypt: test 14 (256 bit key, 16 byte blocks): 1 operation in 1671 cycle= s (16 bytes) tcrypt: test 15 (256 bit key, 64 byte blocks): 1 operation in 2263 cycle= s (64 bytes) tcrypt: test 16 (256 bit key, 128 byte blocks): 1 operation in 2881 cycle= s (128 bytes) tcrypt: test 17 (256 bit key, 256 byte blocks): 1 operation in 4270 cycle= s (256 bytes) tcrypt: test 18 (256 bit key, 1024 byte blocks): 1 operation in 11537 cycle= s (1024 bytes) tcrypt: test 19 (256 bit key, 1424 byte blocks): 1 operation in 15025 cycle= s (1424 bytes) tcrypt: test 20 (256 bit key, 4096 byte blocks): 1 operation in 40747 cycle= s (4096 bytes) Signed-off-by: Alexandre Torgue Signed-off-by: Maxime M=C3=A9r=C3=A9 Signed-off-by: Nicolas Toromanoff --- drivers/crypto/stm32/stm32-cryp.c | 688 ++++++++++++++++++++++++++++-- 1 file changed, 662 insertions(+), 26 deletions(-) diff --git a/drivers/crypto/stm32/stm32-cryp.c b/drivers/crypto/stm32/stm32= -cryp.c index 11ad4ffdce0d..5b6d2fc68fea 100644 --- a/drivers/crypto/stm32/stm32-cryp.c +++ b/drivers/crypto/stm32/stm32-cryp.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include #include #include @@ -40,6 +42,8 @@ /* Mode mask =3D bits [15..0] */ #define FLG_MODE_MASK GENMASK(15, 0) /* Bit [31..16] status */ +#define FLG_IN_OUT_DMA BIT(16) +#define FLG_HEADER_DMA BIT(17) =20 /* Registers */ #define CRYP_CR 0x00000000 @@ -121,8 +125,12 @@ #define CR_PH_MASK 0x00030000 #define CR_NBPBL_SHIFT 20 =20 -#define SR_BUSY 0x00000010 -#define SR_OFNE 0x00000004 +#define SR_IFNF BIT(1) +#define SR_OFNE BIT(2) +#define SR_BUSY BIT(8) + +#define DMACR_DIEN BIT(0) +#define DMACR_DOEN BIT(1) =20 #define IMSCR_IN BIT(0) #define IMSCR_OUT BIT(1) @@ -133,7 +141,15 @@ /* Misc */ #define AES_BLOCK_32 (AES_BLOCK_SIZE / sizeof(u32)) #define GCM_CTR_INIT 2 -#define CRYP_AUTOSUSPEND_DELAY 50 +#define CRYP_AUTOSUSPEND_DELAY 50 + +#define CRYP_DMA_BURST_REG 4 + +enum stm32_dma_mode { + NO_DMA, + DMA_PLAIN_SG, + DMA_NEED_SG_TRUNC +}; =20 struct stm32_cryp_caps { bool aeads_support; @@ -146,6 +162,7 @@ struct stm32_cryp_caps { u32 sr; u32 din; u32 dout; + u32 dmacr; u32 imsc; u32 mis; u32 k1l; @@ -172,6 +189,7 @@ struct stm32_cryp { struct list_head list; struct device *dev; void __iomem *regs; + phys_addr_t phys_base; struct clk *clk; unsigned long flags; u32 irq_status; @@ -190,8 +208,20 @@ struct stm32_cryp { size_t header_in; size_t payload_out; =20 + /* DMA process fields */ + struct scatterlist *in_sg; + struct scatterlist *header_sg; struct scatterlist *out_sg; + size_t in_sg_len; + size_t header_sg_len; + size_t out_sg_len; + struct completion dma_completion; + + struct dma_chan *dma_lch_in; + struct dma_chan *dma_lch_out; + enum stm32_dma_mode dma_mode; =20 + /* IT process fields */ struct scatter_walk in_walk; struct scatter_walk out_walk; =20 @@ -291,12 +321,20 @@ static inline int stm32_cryp_wait_enable(struct stm32= _cryp *cryp) !(status & CR_CRYPEN), 10, 100000); } =20 +static inline int stm32_cryp_wait_input(struct stm32_cryp *cryp) +{ + u32 status; + + return readl_relaxed_poll_timeout_atomic(cryp->regs + cryp->caps->sr, sta= tus, + status & SR_IFNF, 1, 10); +} + static inline int stm32_cryp_wait_output(struct stm32_cryp *cryp) { u32 status; =20 - return readl_relaxed_poll_timeout(cryp->regs + cryp->caps->sr, status, - status & SR_OFNE, 10, 100000); + return readl_relaxed_poll_timeout_atomic(cryp->regs + cryp->caps->sr, sta= tus, + status & SR_OFNE, 1, 10); } =20 static inline void stm32_cryp_key_read_enable(struct stm32_cryp *cryp) @@ -311,8 +349,13 @@ static inline void stm32_cryp_key_read_disable(struct = stm32_cryp *cryp) cryp->regs + cryp->caps->cr); } =20 +static void stm32_cryp_irq_read_data(struct stm32_cryp *cryp); +static void stm32_cryp_irq_write_data(struct stm32_cryp *cryp); +static void stm32_cryp_irq_write_gcmccm_header(struct stm32_cryp *cryp); static int stm32_cryp_read_auth_tag(struct stm32_cryp *cryp); static void stm32_cryp_finish_req(struct stm32_cryp *cryp, int err); +static int stm32_cryp_dma_start(struct stm32_cryp *cryp); +static int stm32_cryp_it_start(struct stm32_cryp *cryp); =20 static struct stm32_cryp *stm32_cryp_find_dev(struct stm32_cryp_ctx *ctx) { @@ -813,11 +856,238 @@ static void stm32_cryp_finish_req(struct stm32_cryp = *cryp, int err) if (is_gcm(cryp) || is_ccm(cryp)) crypto_finalize_aead_request(cryp->engine, cryp->areq, err); else - crypto_finalize_skcipher_request(cryp->engine, cryp->req, - err); + crypto_finalize_skcipher_request(cryp->engine, cryp->req, err); +} + +static void stm32_cryp_header_dma_callback(void *param) +{ + struct stm32_cryp *cryp =3D (struct stm32_cryp *)param; + int ret; + u32 reg; + + dma_unmap_sg(cryp->dev, cryp->header_sg, cryp->header_sg_len, DMA_TO_DEVI= CE); + + reg =3D stm32_cryp_read(cryp, cryp->caps->dmacr); + stm32_cryp_write(cryp, cryp->caps->dmacr, reg & ~(DMACR_DOEN | DMACR_DIEN= )); + + kfree(cryp->header_sg); + + reg =3D stm32_cryp_read(cryp, cryp->caps->cr); + + if (cryp->header_in) { + stm32_cryp_write(cryp, cryp->caps->cr, reg | CR_CRYPEN); + + ret =3D stm32_cryp_wait_input(cryp); + if (ret) { + dev_err(cryp->dev, "input header ready timeout after dma\n"); + stm32_cryp_finish_req(cryp, ret); + return; + } + stm32_cryp_irq_write_gcmccm_header(cryp); + WARN_ON(cryp->header_in); + } + + if (stm32_cryp_get_input_text_len(cryp)) { + /* Phase 3 : payload */ + reg =3D stm32_cryp_read(cryp, cryp->caps->cr); + stm32_cryp_write(cryp, cryp->caps->cr, reg & ~CR_CRYPEN); + + reg &=3D ~CR_PH_MASK; + reg |=3D CR_PH_PAYLOAD | CR_CRYPEN; + stm32_cryp_write(cryp, cryp->caps->cr, reg); + + if (cryp->flags & FLG_IN_OUT_DMA) { + ret =3D stm32_cryp_dma_start(cryp); + if (ret) + stm32_cryp_finish_req(cryp, ret); + } else { + stm32_cryp_it_start(cryp); + } + } else { + /* + * Phase 4 : tag. + * Nothing to read, nothing to write =3D> end request + */ + stm32_cryp_finish_req(cryp, 0); + } } =20 -static int stm32_cryp_cpu_start(struct stm32_cryp *cryp) +static void stm32_cryp_dma_callback(void *param) +{ + struct stm32_cryp *cryp =3D (struct stm32_cryp *)param; + int ret; + u32 reg; + + complete(&cryp->dma_completion); /* completion to indicate no timeout */ + + dma_sync_sg_for_device(cryp->dev, cryp->out_sg, cryp->out_sg_len, DMA_FRO= M_DEVICE); + + if (cryp->in_sg !=3D cryp->out_sg) + dma_unmap_sg(cryp->dev, cryp->in_sg, cryp->in_sg_len, DMA_TO_DEVICE); + + dma_unmap_sg(cryp->dev, cryp->out_sg, cryp->out_sg_len, DMA_FROM_DEVICE); + + reg =3D stm32_cryp_read(cryp, cryp->caps->dmacr); + stm32_cryp_write(cryp, cryp->caps->dmacr, reg & ~(DMACR_DOEN | DMACR_DIEN= )); + + reg =3D stm32_cryp_read(cryp, cryp->caps->cr); + + if (is_gcm(cryp) || is_ccm(cryp)) { + kfree(cryp->in_sg); + kfree(cryp->out_sg); + } else { + if (cryp->in_sg !=3D cryp->req->src) + kfree(cryp->in_sg); + if (cryp->out_sg !=3D cryp->req->dst) + kfree(cryp->out_sg); + } + + if (cryp->payload_in) { + stm32_cryp_write(cryp, cryp->caps->cr, reg | CR_CRYPEN); + + ret =3D stm32_cryp_wait_input(cryp); + if (ret) { + dev_err(cryp->dev, "input ready timeout after dma\n"); + stm32_cryp_finish_req(cryp, ret); + return; + } + stm32_cryp_irq_write_data(cryp); + + ret =3D stm32_cryp_wait_output(cryp); + if (ret) { + dev_err(cryp->dev, "output ready timeout after dma\n"); + stm32_cryp_finish_req(cryp, ret); + return; + } + stm32_cryp_irq_read_data(cryp); + } + + stm32_cryp_finish_req(cryp, 0); +} + +static int stm32_cryp_header_dma_start(struct stm32_cryp *cryp) +{ + int ret; + struct dma_async_tx_descriptor *tx_in; + u32 reg; + size_t align_size; + + ret =3D dma_map_sg(cryp->dev, cryp->header_sg, cryp->header_sg_len, DMA_T= O_DEVICE); + if (!ret) { + dev_err(cryp->dev, "dma_map_sg() error\n"); + return -ENOMEM; + } + + dma_sync_sg_for_device(cryp->dev, cryp->header_sg, cryp->header_sg_len, D= MA_TO_DEVICE); + + tx_in =3D dmaengine_prep_slave_sg(cryp->dma_lch_in, cryp->header_sg, cryp= ->header_sg_len, + DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!tx_in) { + dev_err(cryp->dev, "IN prep_slave_sg() failed\n"); + return -EINVAL; + } + + tx_in->callback_param =3D cryp; + tx_in->callback =3D stm32_cryp_header_dma_callback; + + /* Advance scatterwalk to not DMA'ed data */ + align_size =3D ALIGN_DOWN(cryp->header_in, cryp->hw_blocksize); + scatterwalk_copychunks(NULL, &cryp->in_walk, align_size, 2); + cryp->header_in -=3D align_size; + + ret =3D dma_submit_error(dmaengine_submit(tx_in)); + if (ret < 0) { + dev_err(cryp->dev, "DMA in submit failed\n"); + return ret; + } + dma_async_issue_pending(cryp->dma_lch_in); + + reg =3D stm32_cryp_read(cryp, cryp->caps->dmacr); + stm32_cryp_write(cryp, cryp->caps->dmacr, reg | DMACR_DIEN); + + return 0; +} + +static int stm32_cryp_dma_start(struct stm32_cryp *cryp) +{ + int ret; + size_t align_size; + struct dma_async_tx_descriptor *tx_in, *tx_out; + u32 reg; + + if (cryp->in_sg !=3D cryp->out_sg) { + ret =3D dma_map_sg(cryp->dev, cryp->in_sg, cryp->in_sg_len, DMA_TO_DEVIC= E); + if (!ret) { + dev_err(cryp->dev, "dma_map_sg() error\n"); + return -ENOMEM; + } + } + + ret =3D dma_map_sg(cryp->dev, cryp->out_sg, cryp->out_sg_len, DMA_FROM_DE= VICE); + if (!ret) { + dev_err(cryp->dev, "dma_map_sg() error\n"); + return -ENOMEM; + } + + dma_sync_sg_for_device(cryp->dev, cryp->in_sg, cryp->in_sg_len, DMA_TO_DE= VICE); + + tx_in =3D dmaengine_prep_slave_sg(cryp->dma_lch_in, cryp->in_sg, cryp->in= _sg_len, + DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!tx_in) { + dev_err(cryp->dev, "IN prep_slave_sg() failed\n"); + return -EINVAL; + } + + /* No callback necessary */ + tx_in->callback_param =3D cryp; + tx_in->callback =3D NULL; + + tx_out =3D dmaengine_prep_slave_sg(cryp->dma_lch_out, cryp->out_sg, cryp-= >out_sg_len, + DMA_DEV_TO_MEM, DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!tx_out) { + dev_err(cryp->dev, "OUT prep_slave_sg() failed\n"); + return -EINVAL; + } + + reinit_completion(&cryp->dma_completion); + tx_out->callback =3D stm32_cryp_dma_callback; + tx_out->callback_param =3D cryp; + + /* Advance scatterwalk to not DMA'ed data */ + align_size =3D ALIGN_DOWN(cryp->payload_in, cryp->hw_blocksize); + scatterwalk_copychunks(NULL, &cryp->in_walk, align_size, 2); + cryp->payload_in -=3D align_size; + + ret =3D dma_submit_error(dmaengine_submit(tx_in)); + if (ret < 0) { + dev_err(cryp->dev, "DMA in submit failed\n"); + return ret; + } + dma_async_issue_pending(cryp->dma_lch_in); + + /* Advance scatterwalk to not DMA'ed data */ + scatterwalk_copychunks(NULL, &cryp->out_walk, align_size, 2); + cryp->payload_out -=3D align_size; + ret =3D dma_submit_error(dmaengine_submit(tx_out)); + if (ret < 0) { + dev_err(cryp->dev, "DMA out submit failed\n"); + return ret; + } + dma_async_issue_pending(cryp->dma_lch_out); + + reg =3D stm32_cryp_read(cryp, cryp->caps->dmacr); + stm32_cryp_write(cryp, cryp->caps->dmacr, reg | DMACR_DOEN | DMACR_DIEN); + + if (!wait_for_completion_timeout(&cryp->dma_completion, msecs_to_jiffies(= 1000))) { + dev_err(cryp->dev, "DMA out timed out\n"); + dmaengine_terminate_sync(cryp->dma_lch_out); + return -ETIMEDOUT; + } + + return 0; +} + +static int stm32_cryp_it_start(struct stm32_cryp *cryp) { /* Enable interrupt and let the IRQ handler do everything */ stm32_cryp_write(cryp, cryp->caps->imsc, IMSCR_IN | IMSCR_OUT); @@ -1149,13 +1419,256 @@ static int stm32_cryp_tdes_cbc_decrypt(struct skci= pher_request *req) return stm32_cryp_crypt(req, FLG_TDES | FLG_CBC); } =20 +static enum stm32_dma_mode stm32_cryp_dma_check_sg(struct scatterlist *tes= t_sg, size_t len, + size_t block_size) +{ + struct scatterlist *sg; + int i; + + if (len <=3D 16) + return NO_DMA; /* Faster */ + + for_each_sg(test_sg, sg, sg_nents(test_sg), i) { + if (!IS_ALIGNED(sg->length, block_size) && !sg_is_last(sg)) + return NO_DMA; + + if (sg->offset % sizeof(u32)) + return NO_DMA; + + if (sg_is_last(sg) && !IS_ALIGNED(sg->length, AES_BLOCK_SIZE)) + return DMA_NEED_SG_TRUNC; + } + + return DMA_PLAIN_SG; +} + +static enum stm32_dma_mode stm32_cryp_dma_check(struct stm32_cryp *cryp, s= truct scatterlist *in_sg, + struct scatterlist *out_sg) +{ + enum stm32_dma_mode ret =3D DMA_PLAIN_SG; + + if (!is_aes(cryp)) + return NO_DMA; + + if (!cryp->dma_lch_in || !cryp->dma_lch_out) + return NO_DMA; + + ret =3D stm32_cryp_dma_check_sg(in_sg, cryp->payload_in, AES_BLOCK_SIZE); + if (ret =3D=3D NO_DMA) + return ret; + + ret =3D stm32_cryp_dma_check_sg(out_sg, cryp->payload_out, AES_BLOCK_SIZE= ); + if (ret =3D=3D NO_DMA) + return ret; + + /* Check CTR counter overflow */ + if (is_aes(cryp) && is_ctr(cryp)) { + u32 c; + __be32 iv3; + + memcpy(&iv3, &cryp->req->iv[3 * sizeof(u32)], sizeof(iv3)); + c =3D be32_to_cpu(iv3); + if ((c + cryp->payload_in) < cryp->payload_in) + return NO_DMA; + } + + /* Workaround */ + if (is_aes(cryp) && is_ctr(cryp) && ret =3D=3D DMA_NEED_SG_TRUNC) + return NO_DMA; + + return ret; +} + +static int stm32_cryp_truncate_sg(struct scatterlist **new_sg, size_t *new= _sg_len, + struct scatterlist *sg, off_t skip, size_t size) +{ + struct scatterlist *cur; + int alloc_sg_len; + + *new_sg_len =3D 0; + + if (!sg || !size) { + *new_sg =3D NULL; + return 0; + } + + alloc_sg_len =3D sg_nents_for_len(sg, skip + size); + if (alloc_sg_len < 0) + return alloc_sg_len; + + /* We allocate to much sg entry, but it is easier */ + *new_sg =3D kmalloc_array((size_t)alloc_sg_len, sizeof(struct scatterlist= ), GFP_KERNEL); + if (!*new_sg) + return -ENOMEM; + + sg_init_table(*new_sg, (unsigned int)alloc_sg_len); + + cur =3D *new_sg; + while (sg && size) { + unsigned int len =3D sg->length; + unsigned int offset =3D sg->offset; + + if (skip > len) { + skip -=3D len; + sg =3D sg_next(sg); + continue; + } + + if (skip) { + len -=3D skip; + offset +=3D skip; + skip =3D 0; + } + + if (size < len) + len =3D size; + + if (len > 0) { + (*new_sg_len)++; + size -=3D len; + sg_set_page(cur, sg_page(sg), len, offset); + if (size =3D=3D 0) + sg_mark_end(cur); + cur =3D sg_next(cur); + } + + sg =3D sg_next(sg); + } + + return 0; +} + +static int stm32_cryp_cipher_prepare(struct stm32_cryp *cryp, struct scatt= erlist *in_sg, + struct scatterlist *out_sg) +{ + size_t align_size; + int ret; + + cryp->dma_mode =3D stm32_cryp_dma_check(cryp, in_sg, out_sg); + + scatterwalk_start(&cryp->in_walk, in_sg); + scatterwalk_start(&cryp->out_walk, out_sg); + + if (cryp->dma_mode =3D=3D NO_DMA) { + cryp->flags &=3D ~FLG_IN_OUT_DMA; + + if (is_ctr(cryp)) + memset(cryp->last_ctr, 0, sizeof(cryp->last_ctr)); + + } else if (cryp->dma_mode =3D=3D DMA_NEED_SG_TRUNC) { + + cryp->flags |=3D FLG_IN_OUT_DMA; + + align_size =3D ALIGN_DOWN(cryp->payload_in, cryp->hw_blocksize); + ret =3D stm32_cryp_truncate_sg(&cryp->in_sg, &cryp->in_sg_len, in_sg, 0,= align_size); + if (ret) + return ret; + + ret =3D stm32_cryp_truncate_sg(&cryp->out_sg, &cryp->out_sg_len, out_sg,= 0, + align_size); + if (ret) { + kfree(cryp->in_sg); + return ret; + } + } else { + cryp->flags |=3D FLG_IN_OUT_DMA; + + cryp->in_sg =3D in_sg; + cryp->out_sg =3D out_sg; + + ret =3D sg_nents_for_len(cryp->in_sg, cryp->payload_in); + if (ret < 0) + return ret; + cryp->in_sg_len =3D (size_t)ret; + + ret =3D sg_nents_for_len(out_sg, cryp->payload_out); + if (ret < 0) + return ret; + cryp->out_sg_len =3D (size_t)ret; + } + + return 0; +} + +static int stm32_cryp_aead_prepare(struct stm32_cryp *cryp, struct scatter= list *in_sg, + struct scatterlist *out_sg) +{ + size_t align_size; + off_t skip; + int ret, ret2; + + cryp->header_sg =3D NULL; + cryp->in_sg =3D NULL; + cryp->out_sg =3D NULL; + + if (!cryp->dma_lch_in || !cryp->dma_lch_out) { + cryp->dma_mode =3D NO_DMA; + cryp->flags &=3D ~(FLG_IN_OUT_DMA | FLG_HEADER_DMA); + + return 0; + } + + /* CCM hw_init may have advanced in header */ + skip =3D cryp->areq->assoclen - cryp->header_in; + + align_size =3D ALIGN_DOWN(cryp->header_in, cryp->hw_blocksize); + ret =3D stm32_cryp_truncate_sg(&cryp->header_sg, &cryp->header_sg_len, in= _sg, skip, + align_size); + if (ret) + return ret; + + ret =3D stm32_cryp_dma_check_sg(cryp->header_sg, align_size, AES_BLOCK_SI= ZE); + if (ret =3D=3D NO_DMA) { + /* We cannot DMA the header */ + kfree(cryp->header_sg); + cryp->header_sg =3D NULL; + + cryp->flags &=3D ~FLG_HEADER_DMA; + } else { + cryp->flags |=3D FLG_HEADER_DMA; + } + + /* Now skip all header to be at payload start */ + skip =3D cryp->areq->assoclen; + align_size =3D ALIGN_DOWN(cryp->payload_in, cryp->hw_blocksize); + ret =3D stm32_cryp_truncate_sg(&cryp->in_sg, &cryp->in_sg_len, in_sg, ski= p, align_size); + if (ret) { + kfree(cryp->header_sg); + return ret; + } + + /* For out buffer align_size is same as in buffer */ + ret =3D stm32_cryp_truncate_sg(&cryp->out_sg, &cryp->out_sg_len, out_sg, = skip, align_size); + if (ret) { + kfree(cryp->header_sg); + kfree(cryp->in_sg); + return ret; + } + + ret =3D stm32_cryp_dma_check_sg(cryp->in_sg, align_size, AES_BLOCK_SIZE); + ret2 =3D stm32_cryp_dma_check_sg(cryp->out_sg, align_size, AES_BLOCK_SIZE= ); + if (ret =3D=3D NO_DMA || ret2 =3D=3D NO_DMA) { + kfree(cryp->in_sg); + cryp->in_sg =3D NULL; + + kfree(cryp->out_sg); + cryp->out_sg =3D NULL; + + cryp->flags &=3D ~FLG_IN_OUT_DMA; + } else { + cryp->flags |=3D FLG_IN_OUT_DMA; + } + + return 0; +} + static int stm32_cryp_prepare_req(struct skcipher_request *req, struct aead_request *areq) { struct stm32_cryp_ctx *ctx; struct stm32_cryp *cryp; struct stm32_cryp_reqctx *rctx; - struct scatterlist *in_sg; + struct scatterlist *in_sg, *out_sg; int ret; =20 if (!req && !areq) @@ -1169,8 +1682,6 @@ static int stm32_cryp_prepare_req(struct skcipher_req= uest *req, rctx =3D req ? skcipher_request_ctx(req) : aead_request_ctx(areq); rctx->mode &=3D FLG_MODE_MASK; =20 - ctx->cryp =3D cryp; - cryp->flags =3D (cryp->flags & ~FLG_MODE_MASK) | rctx->mode; cryp->hw_blocksize =3D is_aes(cryp) ? AES_BLOCK_SIZE : DES_BLOCK_SIZE; cryp->ctx =3D ctx; @@ -1182,6 +1693,15 @@ static int stm32_cryp_prepare_req(struct skcipher_re= quest *req, cryp->payload_in =3D req->cryptlen; cryp->payload_out =3D req->cryptlen; cryp->authsize =3D 0; + + in_sg =3D req->src; + out_sg =3D req->dst; + + ret =3D stm32_cryp_cipher_prepare(cryp, in_sg, out_sg); + if (ret) + return ret; + + ret =3D stm32_cryp_hw_init(cryp); } else { /* * Length of input and output data: @@ -1211,23 +1731,22 @@ static int stm32_cryp_prepare_req(struct skcipher_r= equest *req, cryp->header_in =3D areq->assoclen; cryp->payload_out =3D cryp->payload_in; } - } - - in_sg =3D req ? req->src : areq->src; - scatterwalk_start(&cryp->in_walk, in_sg); =20 - cryp->out_sg =3D req ? req->dst : areq->dst; - scatterwalk_start(&cryp->out_walk, cryp->out_sg); + in_sg =3D areq->src; + out_sg =3D areq->dst; =20 - if (is_gcm(cryp) || is_ccm(cryp)) { + scatterwalk_start(&cryp->in_walk, in_sg); + scatterwalk_start(&cryp->out_walk, out_sg); /* In output, jump after assoc data */ scatterwalk_copychunks(NULL, &cryp->out_walk, cryp->areq->assoclen, 2); - } =20 - if (is_ctr(cryp)) - memset(cryp->last_ctr, 0, sizeof(cryp->last_ctr)); + ret =3D stm32_cryp_hw_init(cryp); + if (ret) + return ret; + + ret =3D stm32_cryp_aead_prepare(cryp, in_sg, out_sg); + } =20 - ret =3D stm32_cryp_hw_init(cryp); return ret; } =20 @@ -1239,12 +1758,35 @@ static int stm32_cryp_cipher_one_req(struct crypto_= engine *engine, void *areq) struct stm32_cryp_ctx *ctx =3D crypto_skcipher_ctx( crypto_skcipher_reqtfm(req)); struct stm32_cryp *cryp =3D ctx->cryp; + int ret; =20 if (!cryp) return -ENODEV; =20 - return stm32_cryp_prepare_req(req, NULL) ?: - stm32_cryp_cpu_start(cryp); + ret =3D stm32_cryp_prepare_req(req, NULL); + if (ret) + return ret; + + /* Avoid to use DMA if peripheral 32 bit counter is about to overflow wit= h ctr(aes) */ + if (is_aes(cryp) && is_ctr(cryp)) { + u32 iv_overflow[4]; + + memcpy(iv_overflow, req->iv, sizeof(__be32) * 4); + iv_overflow[3] =3D 0xffffffff - be32_to_cpu((__be32)iv_overflow[3]); + + if (req->src->length > iv_overflow[3]) + cryp->flags &=3D ~FLG_IN_OUT_DMA; + } + + if (cryp->flags & FLG_IN_OUT_DMA) + ret =3D stm32_cryp_dma_start(cryp); + else + ret =3D stm32_cryp_it_start(cryp); + + if (ret =3D=3D -ETIMEDOUT) + stm32_cryp_finish_req(cryp, ret); + + return ret; } =20 static int stm32_cryp_aead_one_req(struct crypto_engine *engine, void *are= q) @@ -1262,13 +1804,20 @@ static int stm32_cryp_aead_one_req(struct crypto_en= gine *engine, void *areq) if (err) return err; =20 - if (unlikely(!cryp->payload_in && !cryp->header_in)) { + if (!stm32_cryp_get_input_text_len(cryp) && !cryp->header_in && + !(cryp->flags & FLG_HEADER_DMA)) { /* No input data to process: get tag and finish */ stm32_cryp_finish_req(cryp, 0); return 0; } =20 - return stm32_cryp_cpu_start(cryp); + if (cryp->flags & FLG_HEADER_DMA) + return stm32_cryp_header_dma_start(cryp); + + if (!cryp->header_in && cryp->flags & FLG_IN_OUT_DMA) + return stm32_cryp_dma_start(cryp); + + return stm32_cryp_it_start(cryp); } =20 static int stm32_cryp_read_auth_tag(struct stm32_cryp *cryp) @@ -1680,6 +2229,65 @@ static irqreturn_t stm32_cryp_irq(int irq, void *arg) return IRQ_WAKE_THREAD; } =20 +static int stm32_cryp_dma_init(struct stm32_cryp *cryp) +{ + struct dma_slave_config dma_conf; + struct dma_chan *chan; + int ret; + + memset(&dma_conf, 0, sizeof(dma_conf)); + + dma_conf.direction =3D DMA_MEM_TO_DEV; + dma_conf.dst_addr =3D cryp->phys_base + cryp->caps->din; + dma_conf.dst_addr_width =3D DMA_SLAVE_BUSWIDTH_4_BYTES; + dma_conf.dst_maxburst =3D CRYP_DMA_BURST_REG; + dma_conf.device_fc =3D false; + + chan =3D dma_request_chan(cryp->dev, "in"); + if (IS_ERR(chan)) + return PTR_ERR(chan); + + cryp->dma_lch_in =3D chan; + ret =3D dmaengine_slave_config(cryp->dma_lch_in, &dma_conf); + if (ret) { + dma_release_channel(cryp->dma_lch_in); + cryp->dma_lch_in =3D NULL; + dev_err(cryp->dev, "Couldn't configure DMA in slave.\n"); + return ret; + } + + memset(&dma_conf, 0, sizeof(dma_conf)); + + dma_conf.direction =3D DMA_DEV_TO_MEM; + dma_conf.src_addr =3D cryp->phys_base + cryp->caps->dout; + dma_conf.src_addr_width =3D DMA_SLAVE_BUSWIDTH_4_BYTES; + dma_conf.src_maxburst =3D CRYP_DMA_BURST_REG; + dma_conf.device_fc =3D false; + + chan =3D dma_request_chan(cryp->dev, "out"); + if (IS_ERR(chan)) { + dma_release_channel(cryp->dma_lch_in); + cryp->dma_lch_in =3D NULL; + return PTR_ERR(chan); + } + + cryp->dma_lch_out =3D chan; + + ret =3D dmaengine_slave_config(cryp->dma_lch_out, &dma_conf); + if (ret) { + dma_release_channel(cryp->dma_lch_out); + cryp->dma_lch_out =3D NULL; + dev_err(cryp->dev, "Couldn't configure DMA out slave.\n"); + dma_release_channel(cryp->dma_lch_in); + cryp->dma_lch_in =3D NULL; + return ret; + } + + init_completion(&cryp->dma_completion); + + return 0; +} + static struct skcipher_engine_alg crypto_algs[] =3D { { .base =3D { @@ -1901,6 +2509,7 @@ static const struct stm32_cryp_caps ux500_data =3D { .sr =3D UX500_CRYP_SR, .din =3D UX500_CRYP_DIN, .dout =3D UX500_CRYP_DOUT, + .dmacr =3D UX500_CRYP_DMACR, .imsc =3D UX500_CRYP_IMSC, .mis =3D UX500_CRYP_MIS, .k1l =3D UX500_CRYP_K1L, @@ -1923,6 +2532,7 @@ static const struct stm32_cryp_caps f7_data =3D { .sr =3D CRYP_SR, .din =3D CRYP_DIN, .dout =3D CRYP_DOUT, + .dmacr =3D CRYP_DMACR, .imsc =3D CRYP_IMSCR, .mis =3D CRYP_MISR, .k1l =3D CRYP_K1LR, @@ -1945,6 +2555,7 @@ static const struct stm32_cryp_caps mp1_data =3D { .sr =3D CRYP_SR, .din =3D CRYP_DIN, .dout =3D CRYP_DOUT, + .dmacr =3D CRYP_DMACR, .imsc =3D CRYP_IMSCR, .mis =3D CRYP_MISR, .k1l =3D CRYP_K1LR, @@ -1985,6 +2596,8 @@ static int stm32_cryp_probe(struct platform_device *p= dev) if (IS_ERR(cryp->regs)) return PTR_ERR(cryp->regs); =20 + cryp->phys_base =3D platform_get_resource(pdev, IORESOURCE_MEM, 0)->start; + irq =3D platform_get_irq(pdev, 0); if (irq < 0) return irq; @@ -2030,6 +2643,17 @@ static int stm32_cryp_probe(struct platform_device *= pdev) =20 platform_set_drvdata(pdev, cryp); =20 + ret =3D stm32_cryp_dma_init(cryp); + switch (ret) { + case 0: + break; + case -ENODEV: + dev_dbg(dev, "DMA mode not available\n"); + break; + default: + goto err_dma; + } + spin_lock(&cryp_list.lock); list_add(&cryp->list, &cryp_list.dev_list); spin_unlock(&cryp_list.lock); @@ -2075,6 +2699,12 @@ static int stm32_cryp_probe(struct platform_device *= pdev) spin_lock(&cryp_list.lock); list_del(&cryp->list); spin_unlock(&cryp_list.lock); + + if (cryp->dma_lch_in) + dma_release_channel(cryp->dma_lch_in); + if (cryp->dma_lch_out) + dma_release_channel(cryp->dma_lch_out); +err_dma: err_rst: pm_runtime_disable(dev); pm_runtime_put_noidle(dev); @@ -2101,6 +2731,12 @@ static void stm32_cryp_remove(struct platform_device= *pdev) list_del(&cryp->list); spin_unlock(&cryp_list.lock); =20 + if (cryp->dma_lch_in) + dma_release_channel(cryp->dma_lch_in); + + if (cryp->dma_lch_out) + dma_release_channel(cryp->dma_lch_out); + pm_runtime_disable(cryp->dev); pm_runtime_put_noidle(cryp->dev); =20 --=20 2.25.1 From nobody Sun May 19 10:57:24 2024 Received: from mx08-00178001.pphosted.com (mx08-00178001.pphosted.com [91.207.212.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB66013BC29; Tue, 23 Apr 2024 15:15:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.207.212.93 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713885333; cv=none; b=g0ma/fRq/yFE3Z91WQ4q2cCXgW+Z7imyeXKcJbt9L6rd4DIYQUROZ6EP8NmyXkXxI27RxMBPUs65GNoGYFVjYvX3AEDpIoj7DlUeEl75yX7pR22YF7bldQSbFMfiy4FhdC2+SvUw+0MCndD1xzVXuEcR3ITbaQrylSlRNfSfCAc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713885333; c=relaxed/simple; bh=+7L2xHxhehvAkcUf86WO8mIMGSDiqwaar3cKR/gKO/g=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Z9T6dpiCzWVqiPbI2lJf6rKls1rj8BChgle9jRENfCXPT0QmXogfsy8QFuVFkpOK7aKUCTONQauLVoFauWkwYuvoVLCY9kOsyRTEKYQ5naPoM6q+RoE0Fk7PXNv9D+bMdXSUb+FLXHV3uo8JURldTw0KudWYsOkX6HC3KSp2e54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=foss.st.com; spf=pass smtp.mailfrom=foss.st.com; dkim=pass (2048-bit key) header.d=foss.st.com header.i=@foss.st.com header.b=0SSYQ5J8; arc=none smtp.client-ip=91.207.212.93 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=foss.st.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=foss.st.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=foss.st.com header.i=@foss.st.com header.b="0SSYQ5J8" Received: from pps.filterd (m0369457.ppops.net [127.0.0.1]) by mx07-00178001.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 43NBd3ME013106; Tue, 23 Apr 2024 17:15:00 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foss.st.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s= selector1; bh=W5kriCTlwtswYN7XF7ASYU4t8GUwuuzQNeua46yCkMs=; b=0S SYQ5J85N5KCRVpjUXd+eHOFttpIcmbuGx+SH3nzSMjrxMR2BpdlroSjMvcAviYg8 lU7oR9AxrUcY3GR1MA63vM+iFjTaX2JIsJu+1CuphywQHDJGrrJM2U/BwwthbIZU FgXHLepnK4UmIga3NvQpGpw0L+mhyfX64IJf6ICloN3Va2yjpLqohzkSejKIiq3A N+ymWZ1vSfZ8VEJYo/vG//2QoKSEJov1X3nAQrhlXAGo5DZRgn+WwvjbbO3e6RfE 2skbS0z/EjkNkmJGzECM2SfBqRPKCm6S1U9Ag4Us20PgHUc/GVKsdgN2utsbLCbK zbxlC6JsTQ4gfHre5RoQ== Received: from beta.dmz-ap.st.com (beta.dmz-ap.st.com [138.198.100.35]) by mx07-00178001.pphosted.com (PPS) with ESMTPS id 3xmrnj1q9c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 23 Apr 2024 17:15:00 +0200 (MEST) Received: from euls16034.sgp.st.com (euls16034.sgp.st.com [10.75.44.20]) by beta.dmz-ap.st.com (STMicroelectronics) with ESMTP id 2C05440044; Tue, 23 Apr 2024 17:14:52 +0200 (CEST) Received: from Webmail-eu.st.com (shfdag1node3.st.com [10.75.129.71]) by euls16034.sgp.st.com (STMicroelectronics) with ESMTP id 0A28122365F; Tue, 23 Apr 2024 17:14:16 +0200 (CEST) Received: from localhost (10.48.86.103) by SHFDAG1NODE3.st.com (10.75.129.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 23 Apr 2024 17:14:15 +0200 From: Maxime MERE To: Herbert Xu , "David S . Miller" , Maxime Coquelin , Alexandre Torgue CC: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= , Rob Herring , , , , Subject: [PATCH v2 2/3] crypto: stm32/cryp - increase priority Date: Tue, 23 Apr 2024 17:14:08 +0200 Message-ID: <20240423151409.902490-3-maxime.mere@foss.st.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240423151409.902490-1-maxime.mere@foss.st.com> References: <20240423151409.902490-1-maxime.mere@foss.st.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: EQNCAS1NODE4.st.com (10.75.129.82) To SHFDAG1NODE3.st.com (10.75.129.71) X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-23_12,2024-04-23_02,2023-05-22_02 From: Maxime M=C3=A9r=C3=A9 Increase STM32 CRYP priority, to be greater than the ARM-NEON accelerated version. Signed-of-by: Maxime M=C3=A9r=C3=A9 Signed-off-by: Nicolas Toromanoff --- drivers/crypto/stm32/stm32-cryp.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/crypto/stm32/stm32-cryp.c b/drivers/crypto/stm32/stm32= -cryp.c index 5b6d2fc68fea..36248975191c 100644 --- a/drivers/crypto/stm32/stm32-cryp.c +++ b/drivers/crypto/stm32/stm32-cryp.c @@ -2293,7 +2293,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base =3D { .base.cra_name =3D "ecb(aes)", .base.cra_driver_name =3D "stm32-ecb-aes", - .base.cra_priority =3D 200, + .base.cra_priority =3D 300, .base.cra_flags =3D CRYPTO_ALG_ASYNC, .base.cra_blocksize =3D AES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2315,7 +2315,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base =3D { .base.cra_name =3D "cbc(aes)", .base.cra_driver_name =3D "stm32-cbc-aes", - .base.cra_priority =3D 200, + .base.cra_priority =3D 300, .base.cra_flags =3D CRYPTO_ALG_ASYNC, .base.cra_blocksize =3D AES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2338,7 +2338,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base =3D { .base.cra_name =3D "ctr(aes)", .base.cra_driver_name =3D "stm32-ctr-aes", - .base.cra_priority =3D 200, + .base.cra_priority =3D 300, .base.cra_flags =3D CRYPTO_ALG_ASYNC, .base.cra_blocksize =3D 1, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2361,7 +2361,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base =3D { .base.cra_name =3D "ecb(des)", .base.cra_driver_name =3D "stm32-ecb-des", - .base.cra_priority =3D 200, + .base.cra_priority =3D 300, .base.cra_flags =3D CRYPTO_ALG_ASYNC, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2383,7 +2383,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base =3D { .base.cra_name =3D "cbc(des)", .base.cra_driver_name =3D "stm32-cbc-des", - .base.cra_priority =3D 200, + .base.cra_priority =3D 300, .base.cra_flags =3D CRYPTO_ALG_ASYNC, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2406,7 +2406,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base =3D { .base.cra_name =3D "ecb(des3_ede)", .base.cra_driver_name =3D "stm32-ecb-des3", - .base.cra_priority =3D 200, + .base.cra_priority =3D 300, .base.cra_flags =3D CRYPTO_ALG_ASYNC, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2428,7 +2428,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base =3D { .base.cra_name =3D "cbc(des3_ede)", .base.cra_driver_name =3D "stm32-cbc-des3", - .base.cra_priority =3D 200, + .base.cra_priority =3D 300, .base.cra_flags =3D CRYPTO_ALG_ASYNC, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2462,7 +2462,7 @@ static struct aead_engine_alg aead_algs[] =3D { .base.base =3D { .cra_name =3D "gcm(aes)", .cra_driver_name =3D "stm32-gcm-aes", - .cra_priority =3D 200, + .cra_priority =3D 300, .cra_flags =3D CRYPTO_ALG_ASYNC, .cra_blocksize =3D 1, .cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), @@ -2485,7 +2485,7 @@ static struct aead_engine_alg aead_algs[] =3D { .base.base =3D { .cra_name =3D "ccm(aes)", .cra_driver_name =3D "stm32-ccm-aes", - .cra_priority =3D 200, + .cra_priority =3D 300, .cra_flags =3D CRYPTO_ALG_ASYNC, .cra_blocksize =3D 1, .cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), --=20 2.25.1 From nobody Sun May 19 10:57:24 2024 Received: from mx07-00178001.pphosted.com (mx07-00178001.pphosted.com [185.132.182.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A82513C695; Tue, 23 Apr 2024 15:15:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.132.182.106 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713885325; cv=none; b=AW2Hv+C+SRDBdfwhHSQDU49R2PHascrFM110j2PqPkP056yhcDFkAZXOmvrbrxcoGmz4Anhxm5GXMbZN5aqqtfO8KVAe41JEDR9uHHicrgAu1oodrn009YGCgOJ7Y/YzAHEHAQvFJVvytEll2mQ2L2IDqtWjri2dd9Fz7M91VEE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713885325; c=relaxed/simple; bh=lnYfLz/yKVLG22Y0s2TOCf6E1sFzxpYUAzI759jEv3k=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Sk0xyE3kHgjN33nTGjtm/zywIe/H7CGpWT1aMvmiHsW4Yjw/ckjkdQklmi+JnRplAJXickog/tcA+qkUm91yLQQj3JXKNoHzhinMxjG+MyhgxSQ4ZGXLwYM7omN5Yqce5Qx6/x+cAk3MLDYcZZ29xIPsbmNwxFOjVN3MsQ3O8ZQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=foss.st.com; spf=pass smtp.mailfrom=foss.st.com; dkim=pass (2048-bit key) header.d=foss.st.com header.i=@foss.st.com header.b=iiNNHVRA; arc=none smtp.client-ip=185.132.182.106 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=foss.st.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=foss.st.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=foss.st.com header.i=@foss.st.com header.b="iiNNHVRA" Received: from pps.filterd (m0288072.ppops.net [127.0.0.1]) by mx07-00178001.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 43NCH14A018378; Tue, 23 Apr 2024 17:15:00 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foss.st.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s= selector1; bh=sORmSAf9vacArCUQo8K1St5wRolQtpbAu5JoPjjIy14=; b=ii NNHVRAEhwXhr+niTwSdwL4ktGCL2b1QSYUaSPeh4ZHBjPZLuf2J3Xx9wWzqtg5Bt O6ufbjV5uAybP3SOX7vvMkiSM0vnNppdbDpMcXf7UKDRSg4JiOixxi24Q+jRyK/w lOyKyTs3y/be+CeJEkV/FVUeXVv5Tum+FBunU9PBDcZTTuP2D7wdoFKIRPqdguac e6ap/qrbyncVrdciLBFSY2Dxk56enOgBA8YCADsIHQE+Ms2LV5RKWf5PlbrqJ1Mb nbGcXlwo5wlXWTddQw2Vl4krKG8ziaOiJfvdA4byPKOscaK40ml/7YAvvd7qTCB4 sjuwBbDnrLyP0f39ReuQ== Received: from beta.dmz-ap.st.com (beta.dmz-ap.st.com [138.198.100.35]) by mx07-00178001.pphosted.com (PPS) with ESMTPS id 3xm4cncfun-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 23 Apr 2024 17:15:00 +0200 (MEST) Received: from euls16034.sgp.st.com (euls16034.sgp.st.com [10.75.44.20]) by beta.dmz-ap.st.com (STMicroelectronics) with ESMTP id 2BFCD4002D; Tue, 23 Apr 2024 17:14:52 +0200 (CEST) Received: from Webmail-eu.st.com (shfdag1node3.st.com [10.75.129.71]) by euls16034.sgp.st.com (STMicroelectronics) with ESMTP id CE988223664; Tue, 23 Apr 2024 17:14:16 +0200 (CEST) Received: from localhost (10.48.86.103) by SHFDAG1NODE3.st.com (10.75.129.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 23 Apr 2024 17:14:16 +0200 From: Maxime MERE To: Herbert Xu , "David S . Miller" , Maxime Coquelin , Alexandre Torgue CC: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= , Rob Herring , , , , Subject: [PATCH v2 3/3] crypto: stm32/cryp - add CRYPTO_ALG_KERN_DRIVER_ONLY flag Date: Tue, 23 Apr 2024 17:14:09 +0200 Message-ID: <20240423151409.902490-4-maxime.mere@foss.st.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240423151409.902490-1-maxime.mere@foss.st.com> References: <20240423151409.902490-1-maxime.mere@foss.st.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: EQNCAS1NODE4.st.com (10.75.129.82) To SHFDAG1NODE3.st.com (10.75.129.71) X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-23_12,2024-04-23_02,2023-05-22_02 From: Maxime M=C3=A9r=C3=A9 This flag is needed to make the driver visible from openssl and cryptodev. Signed-off-by: Maxime M=C3=A9r=C3=A9 --- drivers/crypto/stm32/stm32-cryp.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/crypto/stm32/stm32-cryp.c b/drivers/crypto/stm32/stm32= -cryp.c index 36248975191c..e1a4b5d9d006 100644 --- a/drivers/crypto/stm32/stm32-cryp.c +++ b/drivers/crypto/stm32/stm32-cryp.c @@ -2294,7 +2294,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base.cra_name =3D "ecb(aes)", .base.cra_driver_name =3D "stm32-ecb-aes", .base.cra_priority =3D 300, - .base.cra_flags =3D CRYPTO_ALG_ASYNC, + .base.cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize =3D AES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .base.cra_alignmask =3D 0, @@ -2316,7 +2316,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base.cra_name =3D "cbc(aes)", .base.cra_driver_name =3D "stm32-cbc-aes", .base.cra_priority =3D 300, - .base.cra_flags =3D CRYPTO_ALG_ASYNC, + .base.cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize =3D AES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .base.cra_alignmask =3D 0, @@ -2339,7 +2339,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base.cra_name =3D "ctr(aes)", .base.cra_driver_name =3D "stm32-ctr-aes", .base.cra_priority =3D 300, - .base.cra_flags =3D CRYPTO_ALG_ASYNC, + .base.cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize =3D 1, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .base.cra_alignmask =3D 0, @@ -2362,7 +2362,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base.cra_name =3D "ecb(des)", .base.cra_driver_name =3D "stm32-ecb-des", .base.cra_priority =3D 300, - .base.cra_flags =3D CRYPTO_ALG_ASYNC, + .base.cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .base.cra_alignmask =3D 0, @@ -2384,7 +2384,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base.cra_name =3D "cbc(des)", .base.cra_driver_name =3D "stm32-cbc-des", .base.cra_priority =3D 300, - .base.cra_flags =3D CRYPTO_ALG_ASYNC, + .base.cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .base.cra_alignmask =3D 0, @@ -2407,7 +2407,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base.cra_name =3D "ecb(des3_ede)", .base.cra_driver_name =3D "stm32-ecb-des3", .base.cra_priority =3D 300, - .base.cra_flags =3D CRYPTO_ALG_ASYNC, + .base.cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .base.cra_alignmask =3D 0, @@ -2429,7 +2429,7 @@ static struct skcipher_engine_alg crypto_algs[] =3D { .base.cra_name =3D "cbc(des3_ede)", .base.cra_driver_name =3D "stm32-cbc-des3", .base.cra_priority =3D 300, - .base.cra_flags =3D CRYPTO_ALG_ASYNC, + .base.cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize =3D DES_BLOCK_SIZE, .base.cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .base.cra_alignmask =3D 0, @@ -2463,7 +2463,7 @@ static struct aead_engine_alg aead_algs[] =3D { .cra_name =3D "gcm(aes)", .cra_driver_name =3D "stm32-gcm-aes", .cra_priority =3D 300, - .cra_flags =3D CRYPTO_ALG_ASYNC, + .cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize =3D 1, .cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .cra_alignmask =3D 0, @@ -2486,7 +2486,7 @@ static struct aead_engine_alg aead_algs[] =3D { .cra_name =3D "ccm(aes)", .cra_driver_name =3D "stm32-ccm-aes", .cra_priority =3D 300, - .cra_flags =3D CRYPTO_ALG_ASYNC, + .cra_flags =3D CRYPTO_ALG_ASYNC | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize =3D 1, .cra_ctxsize =3D sizeof(struct stm32_cryp_ctx), .cra_alignmask =3D 0, --=20 2.25.1