From nobody Tue Aug 25 08:46:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA3093E8C65; Wed, 5 Aug 2026 21:20:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964815; cv=none; b=adJPNFrfpR/eMXnAOCVCBDyvl9uyKnFbgyjXSNqEcpOrWLmXb8MZ7EL8llsGRJ0C834xjT4jk67uGlo15jhgWk8iZWtEEOVDFIhu/BF40yu7fo8drOrqHQ/drdrSA1aNjkS3jCbQ9clbdz5hn5aEVjH85Mc6LFZNhTXPRpvIg0k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964815; c=relaxed/simple; bh=Bkj/TkmC38OQHd9BxPXJNS5cfJdj/J4ToKSU/+HlcOs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jXM8Hd9UNqEI4dMbjxhczc3asD4OvU1L+3+tEzKQCumdo6UUffJ7fQx2fZDlrs+D8pLS7ngDwBeoP3t9bvfUfT+NPXX4xGn5vTY5ck1yS3eTY7iXljVRHAC1GYixvLtfjaSWGa7DDyDXoBc2pAAjNCqoE6hmNyTwSEyHmSfLLfc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Ob5o5lsq; arc=none smtp.client-ip=198.175.65.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Ob5o5lsq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1785964808; x=1817500808; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Bkj/TkmC38OQHd9BxPXJNS5cfJdj/J4ToKSU/+HlcOs=; b=Ob5o5lsq//DIAFZ0odCWFPjbS0lLfucsamJBVfuC/tlBBEcNsUU/77IK NIA9dSCIRb9Q5Sma6UfWukQT5aaDSfqUWC/kksldk4n8jhSlYEezlB4zs JDgcyOWffmqhDMmkg0ObHUH3XX0biZHGBXj04Kz1pyoUTAAzgR5RqPwMy ktkYAyZj7dPc35hVcPOVtiA4XScFdh6pNupVmqJgpt8LuZcsThB8hvQbt zB6myz0Z36W1t+o84sJoJ7a63XUyBxXdMENA8lG9DFOT3pM1dGuZBQbgv qZyb6GSaMkxLoDcgsoVmwRXCWumCivN9O33YHNDEwZTAimC9c6gxNXuFL A==; X-CSE-ConnectionGUID: XQOBQVHhSOKtgRfniYf5GA== X-CSE-MsgGUID: 8brnOda1SAOJqitKBjlnUQ== X-IronPort-AV: E=McAfee;i="6800,10657,11866"; a="109341671" X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="109341671" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 X-CSE-ConnectionGUID: 56lWxJejS0ebB0duRwn5WQ== X-CSE-MsgGUID: Ufo9sTw6QJOOXYMgV3ZwBA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="300138475" Received: from vcostago-desk1.jf.intel.com (HELO [10.88.27.144]) ([10.88.27.144]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 From: Vinicius Costa Gomes Date: Wed, 05 Aug 2026 14:19:22 -0700 Subject: [PATCH v2 1/5] dmaengine: idxd: assign all engines to group 0 in IAA defaults Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260805-iaa-crypto-fixes-zswap-v2-1-55c02694f499@intel.com> References: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> In-Reply-To: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> To: Dave Jiang , Vinod Koul , Frank Li , Kristen Accardi , Herbert Xu , "David S. Miller" , Andrew Morton , Yosry Ahmed , Nhat Pham Cc: dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, Vinicius Costa Gomes , Giovanni Cabiddu X-Mailer: b4 0.16-dev-4217c X-Developer-Signature: v=1; a=ed25519-sha256; t=1785964788; l=1611; i=vinicius.gomes@intel.com; s=20230921; h=from:subject:message-id; bh=4xJecnjuJZxGefDI/e9EVeBv1K3O7sNfo98hKPVMPyU=; b=hlGRaEjiI1Q6EBmLbZjOOpCJGJjPjE5QND0ws6GpI6FZpcA5jsKRMwrlDBNNLnK6BR1Vuu5HH dRsYlnSuhTSDRxF1YEni1uh1bz7sUKrSjI977RzsHRh1F6B9ro5ajus X-Developer-Key: i=vinicius.gomes@intel.com; a=ed25519; pk=aJkrtgqgT6TZ8iIHSG8/rTPsmlYnjMrUjCsMYvCzntk= From: Giovanni Cabiddu The IAA device defaults only assigned engine 0 to group 0, leaving engines 1 through max_engines-1 unassigned (group_id =3D -1). This means that by default only a single engine processed descriptors, limiting throughput to one engine's capacity. Assign all available engines to group 0 so that the full hardware parallelism is used out of the box without requiring manual accel-config setup. Signed-off-by: Giovanni Cabiddu Signed-off-by: Vinicius Costa Gomes Reviewed-by: Dave Jiang --- drivers/dma/idxd/defaults.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/dma/idxd/defaults.c b/drivers/dma/idxd/defaults.c index 2bbbcd02a0da..26ebfa2ca144 100644 --- a/drivers/dma/idxd/defaults.c +++ b/drivers/dma/idxd/defaults.c @@ -8,6 +8,7 @@ int idxd_load_iaa_device_defaults(struct idxd_device *idxd) struct idxd_engine *engine; struct idxd_group *group; struct idxd_wq *wq; + int i; =20 if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) return 0; @@ -41,11 +42,12 @@ int idxd_load_iaa_device_defaults(struct idxd_device *i= dxd) /* set driver_name to "crypto" */ strscpy_pad(wq->driver_name, "crypto"); =20 - engine =3D idxd->engines[0]; - - /* set engine group to 0 */ - engine->group =3D idxd->groups[0]; - engine->group->num_engines++; + /* assign all engines to group 0 */ + for (i =3D 0; i < idxd->max_engines; i++) { + engine =3D idxd->engines[i]; + engine->group =3D group; + group->num_engines++; + } =20 return 0; } --=20 2.55.0 From nobody Tue Aug 25 08:46:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 764713EEAC4; Wed, 5 Aug 2026 21:20:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964819; cv=none; b=ZbEajE3jVSyTsp136qotLQlEFXZL4NXBhROgb8mN9YSnw3n3s3tLgNodrbC+tk1HVcj8EJlQ9vakoxkn/xeU4+gdm1MHE5aEWdQ86mtbSthEx+DmatR7+3nJ8r4eBCzRqz09vsgtOy4q4cAi+mS5G5TsiPr5ol80QA4Fvm1emsE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964819; c=relaxed/simple; bh=WGQ/kaRFdhmLazLFJgHnGUbRo6qOeCsrYg861dEF8FU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=hicKsaeHqNIcaaYIwMyBEchyFK0Tj/H6fm6NV4ACHJW8MMdkuiOi5DrPHZSGNj6Roc0ma/oHaBM4QgP+FGD824SIgwH9km6Q63QDxB02mCYw9PA4txuFqyxHQCKhBWzSP/lcAxJaFQ/JeoGDq6t5RFD77uwrag1BWKhWYP8rzbw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=da+IXLCD; arc=none smtp.client-ip=198.175.65.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="da+IXLCD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1785964812; x=1817500812; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=WGQ/kaRFdhmLazLFJgHnGUbRo6qOeCsrYg861dEF8FU=; b=da+IXLCDfrie3VAhyyEeCc9WA1bRVGBLulONnECMSFREHHY3OB6F7PpS faQpAVDHr1yb5NPhfmhhnQjk/4mBcLYLe/L0f7d0EDdIH1SMTAwiDamcO qmgmjZ8p1B2wdaOooWTlm+TbZnLt/e6PS/zc3aDT5QIc3f5PavRfqF/Qy UCZ03cV7CrYC32mttf1BrBE/1dWRC/aW8H0gsGlfS29S1rf4ndEuARroq gFeabfb90jBL+LQIfsqGQ8BZXNxqAFLMJ69w39Zaq4ISsNSGQiEGtpMlX nNXWqovRrWFT/CfGigPT7HMIzwurO6bBFwitLGadTPWcPxxDX0ioWkcQN Q==; X-CSE-ConnectionGUID: TEJnuGKLSO26UjM0mvAnRA== X-CSE-MsgGUID: vCOMsGxATv+UF4/xPyfc5Q== X-IronPort-AV: E=McAfee;i="6800,10657,11866"; a="109341681" X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="109341681" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 X-CSE-ConnectionGUID: J9ETO/b+QAezz81HYVNHpA== X-CSE-MsgGUID: zJT9IvCfT7mwVxP4rP67Bg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="300138478" Received: from vcostago-desk1.jf.intel.com (HELO [10.88.27.144]) ([10.88.27.144]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 From: Vinicius Costa Gomes Date: Wed, 05 Aug 2026 14:19:23 -0700 Subject: [PATCH v2 2/5] crypto: iaa - fall back to software for multi-entry scatterlists Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260805-iaa-crypto-fixes-zswap-v2-2-55c02694f499@intel.com> References: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> In-Reply-To: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> To: Dave Jiang , Vinod Koul , Frank Li , Kristen Accardi , Herbert Xu , "David S. Miller" , Andrew Morton , Yosry Ahmed , Nhat Pham Cc: dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, Vinicius Costa Gomes , Giovanni Cabiddu , stable@vger.kernel.org X-Mailer: b4 0.16-dev-4217c X-Developer-Signature: v=1; a=ed25519-sha256; t=1785964788; l=11943; i=vinicius.gomes@intel.com; s=20230921; h=from:subject:message-id; bh=s+BRcnMhBAw0HL5SYdMWjPETgpmi/Pu5LVKKe5+JoRQ=; b=RvjDJ7hOhyP+2Ixasn7PhtEUvTW96qtcjpb4ok8epXvE/ybgGeRtLKdgggGSBgTePR/+o9v6r GrqW/4VLfi5A1ZFLN9iuYKrYj6kHl5Q7Y49V4A1UZArw80DIe8Wo+71 X-Developer-Key: i=vinicius.gomes@intel.com; a=ed25519; pk=aJkrtgqgT6TZ8iIHSG8/rTPsmlYnjMrUjCsMYvCzntk= From: Giovanni Cabiddu IAA cannot process source or destination scatterlists with more than one entry directly. Instead of failing these requests, route them through a separate deflate acomp transform and keep the request alive in software. The IAA driver has never handled multi-entry scatterlists, but the limitation was latent until commit e2c3b6b21c77 ("mm: zswap: use SG list decompression APIs from zsmalloc") made zswap pass the raw zsmalloc SG list directly to crypto drivers, so objects spanning multiple pages now reach IAA as multi-entry sources and would otherwise fail decompression. Fallback to the generic DEFLATE implementation for scatterlists with more than one entry. After the multi-entry cases fall back early, simplify the DMA mapping path to a single scatterlist entry and fall back on mapping failure as well. Add counters to track the number of requests processed by the software implementation on the compression direction. Fixes: 2ec6761df889 ("crypto: iaa - Add support for deflate-iaa compression= algorithm") Fixes: e2c3b6b21c77 ("mm: zswap: use SG list decompression APIs from zsmall= oc") Cc: stable@vger.kernel.org Signed-off-by: Giovanni Cabiddu Signed-off-by: Vinicius Costa Gomes --- drivers/crypto/intel/iaa/iaa_crypto_main.c | 111 +++++++++++++++---------= ---- drivers/crypto/intel/iaa/iaa_crypto_stats.c | 9 +++ drivers/crypto/intel/iaa/iaa_crypto_stats.h | 2 + 3 files changed, 71 insertions(+), 51 deletions(-) diff --git a/drivers/crypto/intel/iaa/iaa_crypto_main.c b/drivers/crypto/in= tel/iaa/iaa_crypto_main.c index f62b994e18e5..904d9413ba18 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_main.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_main.c @@ -2,6 +2,7 @@ /* Copyright(c) 2021 Intel Corporation. All rights rsvd. */ =20 #include +#include #include #include #include @@ -996,6 +997,19 @@ static int deflate_generic_decompress(struct acomp_req= *req) return ret; } =20 +static int deflate_generic_compress(struct acomp_req *req) +{ + ACOMP_FBREQ_ON_STACK(fbreq, req); + int ret; + + ret =3D crypto_acomp_compress(fbreq); + req->dlen =3D fbreq->dlen; + + update_total_sw_comp_calls(); + + return ret; +} + static int iaa_remap_for_verify(struct device *dev, struct iaa_wq *iaa_wq, struct acomp_req *req, dma_addr_t *src_addr, dma_addr_t *dst_addr); @@ -1472,7 +1486,7 @@ static int iaa_comp_acompress(struct acomp_req *req) struct iaa_compression_ctx *compression_ctx; struct crypto_tfm *tfm =3D req->base.tfm; dma_addr_t src_addr, dst_addr; - int nr_sgs, cpu, ret =3D 0; + int cpu, ret =3D 0; struct iaa_wq *iaa_wq; struct idxd_wq *wq; struct device *dev; @@ -1484,11 +1498,15 @@ static int iaa_comp_acompress(struct acomp_req *req) return -ENODEV; } =20 - if (!req->src || !req->slen) { - pr_debug("invalid src, not compressing\n"); + if (!req->src || !req->slen || !req->dst) { + pr_debug("invalid req, not compressing\n"); return -EINVAL; } =20 + /* Fall back to software if src or dst has multiple sg entries */ + if (sg_nents(req->src) > 1 || sg_nents(req->dst) > 1) + return deflate_generic_compress(req); + cpu =3D get_cpu(); wq =3D wq_table_next_wq(cpu); put_cpu(); @@ -1507,30 +1525,25 @@ static int iaa_comp_acompress(struct acomp_req *req) =20 dev =3D &wq->idxd->pdev->dev; =20 - nr_sgs =3D dma_map_sg(dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); - if (nr_sgs <=3D 0 || nr_sgs > 1) { - dev_dbg(dev, "couldn't map src sg for iaa device %d," - " wq %d: ret=3D%d\n", iaa_wq->iaa_device->idxd->id, - iaa_wq->wq->id, ret); - ret =3D -EIO; - goto out; + if (!dma_map_sg(dev, req->src, 1, DMA_TO_DEVICE)) { + dev_dbg(dev, "couldn't map src sg for iaa device %d, wq %d\n", + iaa_wq->iaa_device->idxd->id, iaa_wq->wq->id); + iaa_wq_put(wq); + return deflate_generic_compress(req); } src_addr =3D sg_dma_address(req->src); - dev_dbg(dev, "dma_map_sg, src_addr %llx, nr_sgs %d, req->src %p," - " req->slen %d, sg_dma_len(sg) %d\n", src_addr, nr_sgs, + dev_dbg(dev, "map src %llx req->src %p slen %d sg_len %d\n", src_addr, req->src, req->slen, sg_dma_len(req->src)); =20 - nr_sgs =3D dma_map_sg(dev, req->dst, sg_nents(req->dst), DMA_FROM_DEVICE); - if (nr_sgs <=3D 0 || nr_sgs > 1) { - dev_dbg(dev, "couldn't map dst sg for iaa device %d," - " wq %d: ret=3D%d\n", iaa_wq->iaa_device->idxd->id, - iaa_wq->wq->id, ret); - ret =3D -EIO; - goto err_map_dst; + if (!dma_map_sg(dev, req->dst, 1, DMA_FROM_DEVICE)) { + dev_dbg(dev, "couldn't map dst sg for iaa device %d, wq %d\n", + iaa_wq->iaa_device->idxd->id, iaa_wq->wq->id); + dma_unmap_sg(dev, req->src, 1, DMA_TO_DEVICE); + iaa_wq_put(wq); + return deflate_generic_compress(req); } dst_addr =3D sg_dma_address(req->dst); - dev_dbg(dev, "dma_map_sg, dst_addr %llx, nr_sgs %d, req->dst %p," - " req->dlen %d, sg_dma_len(sg) %d\n", dst_addr, nr_sgs, + dev_dbg(dev, "map dst %llx req->dst %p dlen %d sg_len %d\n", dst_addr, req->dst, req->dlen, sg_dma_len(req->dst)); =20 ret =3D iaa_compress(tfm, req, wq, src_addr, req->slen, dst_addr, @@ -1550,8 +1563,8 @@ static int iaa_comp_acompress(struct acomp_req *req) if (ret) dev_dbg(dev, "asynchronous compress verification failed ret=3D%d\n", re= t); =20 - dma_unmap_sg(dev, req->dst, sg_nents(req->dst), DMA_TO_DEVICE); - dma_unmap_sg(dev, req->src, sg_nents(req->src), DMA_FROM_DEVICE); + dma_unmap_sg(dev, req->dst, 1, DMA_TO_DEVICE); + dma_unmap_sg(dev, req->src, 1, DMA_FROM_DEVICE); =20 goto out; } @@ -1559,9 +1572,8 @@ static int iaa_comp_acompress(struct acomp_req *req) if (ret) dev_dbg(dev, "asynchronous compress failed ret=3D%d\n", ret); =20 - dma_unmap_sg(dev, req->dst, sg_nents(req->dst), DMA_FROM_DEVICE); -err_map_dst: - dma_unmap_sg(dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); + dma_unmap_sg(dev, req->dst, 1, DMA_FROM_DEVICE); + dma_unmap_sg(dev, req->src, 1, DMA_TO_DEVICE); out: iaa_wq_put(wq); =20 @@ -1572,7 +1584,7 @@ static int iaa_comp_adecompress(struct acomp_req *req) { struct crypto_tfm *tfm =3D req->base.tfm; dma_addr_t src_addr, dst_addr; - int nr_sgs, cpu, ret =3D 0; + int cpu, ret =3D 0; struct iaa_wq *iaa_wq; struct device *dev; struct idxd_wq *wq; @@ -1582,11 +1594,15 @@ static int iaa_comp_adecompress(struct acomp_req *r= eq) return -ENODEV; } =20 - if (!req->src || !req->slen) { - pr_debug("invalid src, not decompressing\n"); + if (!req->src || !req->slen || !req->dst) { + pr_debug("invalid req, not decompressing\n"); return -EINVAL; } =20 + /* Fall back to software if src or dst has multiple sg entries */ + if (sg_nents(req->src) > 1 || sg_nents(req->dst) > 1) + return deflate_generic_decompress(req); + cpu =3D get_cpu(); wq =3D wq_table_next_wq(cpu); put_cpu(); @@ -1605,30 +1621,25 @@ static int iaa_comp_adecompress(struct acomp_req *r= eq) =20 dev =3D &wq->idxd->pdev->dev; =20 - nr_sgs =3D dma_map_sg(dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); - if (nr_sgs <=3D 0 || nr_sgs > 1) { - dev_dbg(dev, "couldn't map src sg for iaa device %d," - " wq %d: ret=3D%d\n", iaa_wq->iaa_device->idxd->id, - iaa_wq->wq->id, ret); - ret =3D -EIO; - goto out; + if (!dma_map_sg(dev, req->src, 1, DMA_TO_DEVICE)) { + dev_dbg(dev, "couldn't map src sg for iaa device %d, wq %d\n", + iaa_wq->iaa_device->idxd->id, iaa_wq->wq->id); + iaa_wq_put(wq); + return deflate_generic_decompress(req); } src_addr =3D sg_dma_address(req->src); - dev_dbg(dev, "dma_map_sg, src_addr %llx, nr_sgs %d, req->src %p," - " req->slen %d, sg_dma_len(sg) %d\n", src_addr, nr_sgs, + dev_dbg(dev, "map src %llx req->src %p slen %d sg_len %d\n", src_addr, req->src, req->slen, sg_dma_len(req->src)); =20 - nr_sgs =3D dma_map_sg(dev, req->dst, sg_nents(req->dst), DMA_FROM_DEVICE); - if (nr_sgs <=3D 0 || nr_sgs > 1) { - dev_dbg(dev, "couldn't map dst sg for iaa device %d," - " wq %d: ret=3D%d\n", iaa_wq->iaa_device->idxd->id, - iaa_wq->wq->id, ret); - ret =3D -EIO; - goto err_map_dst; + if (!dma_map_sg(dev, req->dst, 1, DMA_FROM_DEVICE)) { + dev_dbg(dev, "couldn't map dst sg for iaa device %d, wq %d\n", + iaa_wq->iaa_device->idxd->id, iaa_wq->wq->id); + dma_unmap_sg(dev, req->src, 1, DMA_TO_DEVICE); + iaa_wq_put(wq); + return deflate_generic_decompress(req); } dst_addr =3D sg_dma_address(req->dst); - dev_dbg(dev, "dma_map_sg, dst_addr %llx, nr_sgs %d, req->dst %p," - " req->dlen %d, sg_dma_len(sg) %d\n", dst_addr, nr_sgs, + dev_dbg(dev, "map dst %llx req->dst %p dlen %d sg_len %d\n", dst_addr, req->dst, req->dlen, sg_dma_len(req->dst)); =20 ret =3D iaa_decompress(tfm, req, wq, src_addr, req->slen, @@ -1639,10 +1650,8 @@ static int iaa_comp_adecompress(struct acomp_req *re= q) if (ret !=3D 0) dev_dbg(dev, "asynchronous decompress failed ret=3D%d\n", ret); =20 - dma_unmap_sg(dev, req->dst, sg_nents(req->dst), DMA_FROM_DEVICE); -err_map_dst: - dma_unmap_sg(dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); -out: + dma_unmap_sg(dev, req->dst, 1, DMA_FROM_DEVICE); + dma_unmap_sg(dev, req->src, 1, DMA_TO_DEVICE); iaa_wq_put(wq); =20 return ret; diff --git a/drivers/crypto/intel/iaa/iaa_crypto_stats.c b/drivers/crypto/i= ntel/iaa/iaa_crypto_stats.c index f5cc3d29ca19..2f2ed88c8812 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_stats.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_stats.c @@ -19,6 +19,7 @@ =20 static atomic64_t total_comp_calls; static atomic64_t total_decomp_calls; +static atomic64_t total_sw_comp_calls; static atomic64_t total_sw_decomp_calls; static atomic64_t total_comp_bytes_out; static atomic64_t total_decomp_bytes_in; @@ -43,6 +44,11 @@ void update_total_decomp_calls(void) atomic64_inc(&total_decomp_calls); } =20 +void update_total_sw_comp_calls(void) +{ + atomic64_inc(&total_sw_comp_calls); +} + void update_total_sw_decomp_calls(void) { atomic64_inc(&total_sw_decomp_calls); @@ -104,6 +110,7 @@ static void reset_iaa_crypto_stats(void) { atomic64_set(&total_comp_calls, 0); atomic64_set(&total_decomp_calls, 0); + atomic64_set(&total_sw_comp_calls, 0); atomic64_set(&total_sw_decomp_calls, 0); atomic64_set(&total_comp_bytes_out, 0); atomic64_set(&total_decomp_bytes_in, 0); @@ -174,6 +181,8 @@ static int global_stats_show(struct seq_file *m, void *= v) atomic64_read(&total_comp_calls)); seq_printf(m, " total_decomp_calls: %llu\n", atomic64_read(&total_decomp_calls)); + seq_printf(m, " total_sw_comp_calls: %llu\n", + atomic64_read(&total_sw_comp_calls)); seq_printf(m, " total_sw_decomp_calls: %llu\n", atomic64_read(&total_sw_decomp_calls)); seq_printf(m, " total_comp_bytes_out: %llu\n", diff --git a/drivers/crypto/intel/iaa/iaa_crypto_stats.h b/drivers/crypto/i= ntel/iaa/iaa_crypto_stats.h index 3787a5f507eb..6e0c6f9939bf 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_stats.h +++ b/drivers/crypto/intel/iaa/iaa_crypto_stats.h @@ -11,6 +11,7 @@ void iaa_crypto_debugfs_cleanup(void); void update_total_comp_calls(void); void update_total_comp_bytes_out(int n); void update_total_decomp_calls(void); +void update_total_sw_comp_calls(void); void update_total_sw_decomp_calls(void); void update_total_decomp_bytes_in(int n); void update_completion_einval_errs(void); @@ -29,6 +30,7 @@ static inline void iaa_crypto_debugfs_cleanup(void) {} static inline void update_total_comp_calls(void) {} static inline void update_total_comp_bytes_out(int n) {} static inline void update_total_decomp_calls(void) {} +static inline void update_total_sw_comp_calls(void) {} static inline void update_total_sw_decomp_calls(void) {} static inline void update_total_decomp_bytes_in(int n) {} static inline void update_completion_einval_errs(void) {} --=20 2.55.0 From nobody Tue Aug 25 08:46:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C4723F1AC5; Wed, 5 Aug 2026 21:20:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964826; cv=none; b=ELl4fRPsSo4AtO2cULpvFL7gX1Ny7amjYhECgpGFY+eY0KJxpUHlq9AurMUpOnzMLPi5d5m0uZx38KEBmiAoVKLqPkXantTmhr2E3WbvmZWJWVboKCEEvUwJnB8UKDI246sND3Yl6nzB2zo1gLig+rOnh/y9UfkoObvtQ548x4Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964826; c=relaxed/simple; bh=2gCd4qkpTC2cn0TrW9fNKJsdBnOMA4DDLXfjvTEGzkU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Ao1o828JMQxuGA4JHn+iVlKtVlY4mjS0Xqa+XHRKxNPM4pzgZ03JUe9e5SHIWPHGZlUKrBabBdYnacsb3c5mrJbq6bxUvS1tcqQnQDduM9GapwHnRA7mGyBqt5YeNlKk/QLPVm61Q1sP1p/gk3BkG81l3P8xdaR5pdoaDFtp1+w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=RZGNIfkY; arc=none smtp.client-ip=198.175.65.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RZGNIfkY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1785964820; x=1817500820; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=2gCd4qkpTC2cn0TrW9fNKJsdBnOMA4DDLXfjvTEGzkU=; b=RZGNIfkYub2pAFrkVcegwSbcAYv23DCrTI6iA0yIx5u41ROdpKqSAzej QNAr+UoobNzdf0ws16fz7ayPoGWuPEbqQv7jc9KyN6pJOkBbcNPJWPgnK g7kznVgLHnOIYoUZzJ1JyUUQW8O7Yp66ZCG6tA9iXWicFuSIoBZtXGeB6 3pXR9tAc+bodjRHQEejCKFs5f7vs/F0CKyaYYGUmB/q4c4kkFN+3zByfv igrgkQrD9H2H7PU0hLyo05gldvudZof9h9CUyXgTuNSMEsTsVRxZwey4G kPY45Ehi7aMR39rJ+FjJTxoqZuvVa9ARqlWMJxxQbgp0uKtAQp24vb3yR g==; X-CSE-ConnectionGUID: ka60IwI5TmCrKnuvUYPTTA== X-CSE-MsgGUID: ea9vuo+5TUK2DXSPdP+tFQ== X-IronPort-AV: E=McAfee;i="6800,10657,11866"; a="109341688" X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="109341688" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 X-CSE-ConnectionGUID: LlRD53ImQg+hymcvEEbnIg== X-CSE-MsgGUID: 6p6cL0YjRpGn29ZItKnJZg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="300138481" Received: from vcostago-desk1.jf.intel.com (HELO [10.88.27.144]) ([10.88.27.144]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 From: Vinicius Costa Gomes Date: Wed, 05 Aug 2026 14:19:24 -0700 Subject: [PATCH v2 3/5] crypto: iaa - avoid counting fallback decompression bytes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260805-iaa-crypto-fixes-zswap-v2-3-55c02694f499@intel.com> References: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> In-Reply-To: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> To: Dave Jiang , Vinod Koul , Frank Li , Kristen Accardi , Herbert Xu , "David S. Miller" , Andrew Morton , Yosry Ahmed , Nhat Pham Cc: dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, Vinicius Costa Gomes , Giovanni Cabiddu X-Mailer: b4 0.16-dev-4217c X-Developer-Signature: v=1; a=ed25519-sha256; t=1785964788; l=2018; i=vinicius.gomes@intel.com; s=20230921; h=from:subject:message-id; bh=xj4K3URZRVeDWjfTpUmV9VHYcH37W3JvnbOHwvU4O+I=; b=BzQqsmRIfA9oZNQ8hrLfEWckHcLXWkjlihWfddUHBI2hGxBPRk7Q4ROOQcO33wrg2J5HmGlki 6Fi+5bs7Nr6AINm6PdcyjuCZ+FProR/W8lOuLjq3dG9ka9TiGTPmBhw X-Developer-Key: i=vinicius.gomes@intel.com; a=ed25519; pk=aJkrtgqgT6TZ8iIHSG8/rTPsmlYnjMrUjCsMYvCzntk= From: Giovanni Cabiddu When decompression falls back to deflate-generic after an analytics error, the request no longer completes through IAA. Move decompression byte accounting into the successful IAA completion path in both the synchronous and asynchronous flows so decomp_bytes only reflects bytes actually processed by IAA. Signed-off-by: Giovanni Cabiddu Signed-off-by: Vinicius Costa Gomes Reviewed-by: Dave Jiang --- drivers/crypto/intel/iaa/iaa_crypto_main.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/drivers/crypto/intel/iaa/iaa_crypto_main.c b/drivers/crypto/in= tel/iaa/iaa_crypto_main.c index 904d9413ba18..9505ca23e6f3 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_main.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_main.c @@ -1071,15 +1071,17 @@ static void iaa_desc_complete(struct idxd_desc *idx= d_desc, } } else { ctx->req->dlen =3D idxd_desc->iax_completion->output_size; + + if (!ctx->compress) { + update_total_decomp_bytes_in(ctx->req->slen); + update_wq_decomp_bytes(iaa_wq->wq, ctx->req->slen); + } } =20 /* Update stats */ if (ctx->compress) { update_total_comp_bytes_out(ctx->req->dlen); update_wq_comp_bytes(iaa_wq->wq, ctx->req->dlen); - } else { - update_total_decomp_bytes_in(ctx->req->slen); - update_wq_decomp_bytes(iaa_wq->wq, ctx->req->slen); } =20 if (ctx->compress && compression_ctx->verify_compress) { @@ -1462,16 +1464,16 @@ static int iaa_decompress(struct crypto_tfm *tfm, s= truct acomp_req *req, } } else { req->dlen =3D idxd_desc->iax_completion->output_size; + + /* Update stats */ + update_total_decomp_bytes_in(slen); + update_wq_decomp_bytes(wq, slen); } =20 *dlen =3D req->dlen; =20 if (!ctx->async_mode) idxd_free_desc(wq, idxd_desc); - - /* Update stats */ - update_total_decomp_bytes_in(slen); - update_wq_decomp_bytes(wq, slen); out: return ret; err: --=20 2.55.0 From nobody Tue Aug 25 08:46:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 219253EFFD7; Wed, 5 Aug 2026 21:20:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964830; cv=none; b=NKQVawTR1a25sUJW8ot90Lju7/2c9g+biyqU9CUyGe92lN11Fz5JMgRxWwxgQbDpd8moI6+VI+YQdk/HNqyy2t9fdvKnX1WZE/Md/7eYei4lfzCiOOXjxamdgd3vslnPUpwOx64MMaYBDxwwHUxz96peIzWR4wh2rwXx/BCny5A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964830; c=relaxed/simple; bh=0mIe74F5PdbWYCSbupTYVjo6B9yeDb1iYFqJ46ZN/dU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=c+4CyGa0zI60vw7HXgWWgDJus0jSGK0+0VEc2kU3V3bMvOrzbfC5i2nPAwRHwaBedYOucPX2tsbhOyFLIb6Fv6ruz2JXKdcZUj11b7hUWSZ3YA+Wyb0ImBzmQKUiBx8/nWiGXSbl777GcOduWM9ww7b6UYI1mzHPxYdclGCj/G8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Ve6KKSFc; arc=none smtp.client-ip=198.175.65.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Ve6KKSFc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1785964823; x=1817500823; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=0mIe74F5PdbWYCSbupTYVjo6B9yeDb1iYFqJ46ZN/dU=; b=Ve6KKSFcxWtDfyAbb1yIxsH2K3CO7AVhBAcw7fRKgVATSIcCXtko+qmS 2FNUk7k2hgJ6DJ82ttyWLJg6e9AIaBqf71dReUsFXWCmgKGbq96ln9yek StlgGgu3uzPntck/uNkQdbIsGR6zJdtMXYlJdPiyQ9Ve9vQn2+MR2YYFX iycl3X3OgRYBbafwIjIc6YLR3nGohb8WRXOmcnrftPtExX1sRd/OlTjRa 1kxVp1tUxVSiKbFWUxsAoOGKRxQgi9i8w5FExMC93XvyvNbepFDRmpCHu uJX63Vfw1RH8nouX6iXWSDQv95PaKHDdhVN9hblt24MOQMk8hyuMS/Cs4 g==; X-CSE-ConnectionGUID: SZzX3QTERGGvwbPLwFno/g== X-CSE-MsgGUID: lxhGOup/RXe/oDo9TQLryw== X-IronPort-AV: E=McAfee;i="6800,10657,11866"; a="109341700" X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="109341700" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 X-CSE-ConnectionGUID: YjmhC5JcQe6Jrr/j4e7cJw== X-CSE-MsgGUID: vvtATsv4SLO0X0w4cPHztg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="300138484" Received: from vcostago-desk1.jf.intel.com (HELO [10.88.27.144]) ([10.88.27.144]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 From: Vinicius Costa Gomes Date: Wed, 05 Aug 2026 14:19:25 -0700 Subject: [PATCH v2 4/5] crypto: iaa - use bounce buffer for multi-sg decompress input Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260805-iaa-crypto-fixes-zswap-v2-4-55c02694f499@intel.com> References: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> In-Reply-To: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> To: Dave Jiang , Vinod Koul , Frank Li , Kristen Accardi , Herbert Xu , "David S. Miller" , Andrew Morton , Yosry Ahmed , Nhat Pham Cc: dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, Vinicius Costa Gomes , Giovanni Cabiddu X-Mailer: b4 0.16-dev-4217c X-Developer-Signature: v=1; a=ed25519-sha256; t=1785964788; l=11356; i=vinicius.gomes@intel.com; s=20230921; h=from:subject:message-id; bh=WaCy7IueYkKmTYUbajrgjaDrujLjnExlWbtCQfhH0U8=; b=OIiwLors/0BHeSlv0NzPWxDPaMgYRMWXBs0d81cC2Pqrsr4sebszTnbeqc1fwjunyXdJEhPTQ bp8PrpsYiLzBBs1svhmcv91ElGIpfPnlJlQ3yAc+YGNtAFAFoIFTveN X-Developer-Key: i=vinicius.gomes@intel.com; a=ed25519; pk=aJkrtgqgT6TZ8iIHSG8/rTPsmlYnjMrUjCsMYvCzntk= From: Giovanni Cabiddu Since commit e2c3b6b21c77 ("mm: zswap: use SG list decompression APIs from zsmalloc"), zswap passes the raw zsmalloc SG list directly to crypto drivers, so a compressed object spanning multiple pages reaches IAA as a multi-entry source. Such requests currently fall back to software decompression. As IAA hardware requires a single DMA source buffer, linearize small multi-entry sources into a pre-allocated bounce page and submit that to the hardware instead of falling back to software. Keep the software fallback only for multi-entry destinations. This recovers most of the performance lost by using the software fallback. Store the bounce-page state in the acomp request context alongside the existing compression CRC, free it through a shared source-unmap helper, and back the pages with a small module-wide mempool so the path remains available in reclaim-driven callers. Signed-off-by: Giovanni Cabiddu Signed-off-by: Vinicius Costa Gomes --- drivers/crypto/intel/iaa/iaa_crypto_main.c | 128 ++++++++++++++++++++++++-= ---- 1 file changed, 106 insertions(+), 22 deletions(-) diff --git a/drivers/crypto/intel/iaa/iaa_crypto_main.c b/drivers/crypto/in= tel/iaa/iaa_crypto_main.c index 9505ca23e6f3..51547c5fcf70 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_main.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_main.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -157,6 +158,16 @@ static bool async_mode; /* Use interrupts */ static bool use_irq; =20 +struct iaa_req_ctx { + u32 compression_crc; + struct page *bounce_src; + dma_addr_t bounce_src_dma; + unsigned int bounce_src_len; +}; + +static mempool_t *iaa_bounce_pool; +#define IAA_BOUNCE_POOL_SIZE 128 + /** * set_iaa_sync_mode - Set IAA sync mode * @name: The name of the sync mode @@ -984,6 +995,23 @@ static inline int check_completion(struct device *dev, return ret; } =20 +static void iaa_unmap_src(struct device *dev, struct acomp_req *req) +{ + struct iaa_req_ctx *req_ctx =3D acomp_request_ctx(req); + + if (req_ctx->bounce_src) { + dma_unmap_page(dev, req_ctx->bounce_src_dma, + req_ctx->bounce_src_len, DMA_TO_DEVICE); + mempool_free(req_ctx->bounce_src, iaa_bounce_pool); + req_ctx->bounce_src =3D NULL; + req_ctx->bounce_src_dma =3D 0; + req_ctx->bounce_src_len =3D 0; + return; + } + + dma_unmap_sg(dev, req->src, 1, DMA_TO_DEVICE); +} + static int deflate_generic_decompress(struct acomp_req *req) { ACOMP_FBREQ_ON_STACK(fbreq, req); @@ -1027,6 +1055,7 @@ static void iaa_desc_complete(struct idxd_desc *idxd_= desc, struct iaa_device_compression_mode *active_compression_mode; struct iaa_compression_ctx *compression_ctx; struct crypto_ctx *ctx =3D __ctx; + struct iaa_req_ctx *req_ctx =3D acomp_request_ctx(ctx->req); struct iaa_device *iaa_device; struct idxd_device *idxd; struct iaa_wq *iaa_wq; @@ -1085,10 +1114,9 @@ static void iaa_desc_complete(struct idxd_desc *idxd= _desc, } =20 if (ctx->compress && compression_ctx->verify_compress) { - u32 *compression_crc =3D acomp_request_ctx(ctx->req); dma_addr_t src_addr, dst_addr; =20 - *compression_crc =3D idxd_desc->iax_completion->crc; + req_ctx->compression_crc =3D idxd_desc->iax_completion->crc; =20 ret =3D iaa_remap_for_verify(dev, iaa_wq, ctx->req, &src_addr, &dst_addr= ); if (ret) { @@ -1111,7 +1139,7 @@ static void iaa_desc_complete(struct idxd_desc *idxd_= desc, } err: dma_unmap_sg(dev, ctx->req->dst, sg_nents(ctx->req->dst), DMA_FROM_DEVICE= ); - dma_unmap_sg(dev, ctx->req->src, sg_nents(ctx->req->src), DMA_TO_DEVICE); + iaa_unmap_src(dev, ctx->req); out: if (ret !=3D 0) dev_dbg(dev, "asynchronous compress failed ret=3D%d\n", ret); @@ -1131,7 +1159,7 @@ static int iaa_compress(struct crypto_tfm *tfm, struc= t acomp_req *req, { struct iaa_device_compression_mode *active_compression_mode; struct iaa_compression_ctx *ctx =3D crypto_tfm_ctx(tfm); - u32 *compression_crc =3D acomp_request_ctx(req); + struct iaa_req_ctx *req_ctx =3D acomp_request_ctx(req); struct iaa_device *iaa_device; struct idxd_desc *idxd_desc; struct iax_hw_desc *desc; @@ -1222,7 +1250,7 @@ static int iaa_compress(struct crypto_tfm *tfm, struc= t acomp_req *req, update_total_comp_bytes_out(*dlen); update_wq_comp_bytes(wq, *dlen); =20 - *compression_crc =3D idxd_desc->iax_completion->crc; + req_ctx->compression_crc =3D idxd_desc->iax_completion->crc; =20 if (!ctx->async_mode) idxd_free_desc(wq, idxd_desc); @@ -1282,7 +1310,7 @@ static int iaa_compress_verify(struct crypto_tfm *tfm= , struct acomp_req *req, { struct iaa_device_compression_mode *active_compression_mode; struct iaa_compression_ctx *ctx =3D crypto_tfm_ctx(tfm); - u32 *compression_crc =3D acomp_request_ctx(req); + struct iaa_req_ctx *req_ctx =3D acomp_request_ctx(req); struct iaa_device *iaa_device; struct idxd_desc *idxd_desc; struct iax_hw_desc *desc; @@ -1342,10 +1370,10 @@ static int iaa_compress_verify(struct crypto_tfm *t= fm, struct acomp_req *req, goto err; } =20 - if (*compression_crc !=3D idxd_desc->iax_completion->crc) { + if (req_ctx->compression_crc !=3D idxd_desc->iax_completion->crc) { ret =3D -EINVAL; - dev_dbg(dev, "(verify) iaa comp/decomp crc mismatch:" - " comp=3D0x%x, decomp=3D0x%x\n", *compression_crc, + dev_dbg(dev, "(verify) iaa comp/decomp crc mismatch: comp=3D0x%x, decomp= =3D0x%x\n", + req_ctx->compression_crc, idxd_desc->iax_completion->crc); print_hex_dump(KERN_INFO, "cmp-rec: ", DUMP_PREFIX_OFFSET, 8, 1, idxd_desc->iax_completion, 64, 0); @@ -1485,6 +1513,7 @@ static int iaa_decompress(struct crypto_tfm *tfm, str= uct acomp_req *req, =20 static int iaa_comp_acompress(struct acomp_req *req) { + struct iaa_req_ctx *req_ctx =3D acomp_request_ctx(req); struct iaa_compression_ctx *compression_ctx; struct crypto_tfm *tfm =3D req->base.tfm; dma_addr_t src_addr, dst_addr; @@ -1493,6 +1522,10 @@ static int iaa_comp_acompress(struct acomp_req *req) struct idxd_wq *wq; struct device *dev; =20 + req_ctx->bounce_src =3D NULL; + req_ctx->bounce_src_dma =3D 0; + req_ctx->bounce_src_len =3D 0; + compression_ctx =3D crypto_tfm_ctx(tfm); =20 if (!iaa_crypto_enabled) { @@ -1584,12 +1617,19 @@ static int iaa_comp_acompress(struct acomp_req *req) =20 static int iaa_comp_adecompress(struct acomp_req *req) { + struct iaa_req_ctx *req_ctx =3D acomp_request_ctx(req); struct crypto_tfm *tfm =3D req->base.tfm; dma_addr_t src_addr, dst_addr; + bool use_bounce_src =3D false; int cpu, ret =3D 0; struct iaa_wq *iaa_wq; struct device *dev; struct idxd_wq *wq; + struct page *page; + + req_ctx->bounce_src =3D NULL; + req_ctx->bounce_src_dma =3D 0; + req_ctx->bounce_src_len =3D 0; =20 if (!iaa_crypto_enabled) { pr_debug("iaa_crypto disabled, not decompressing\n"); @@ -1601,10 +1641,16 @@ static int iaa_comp_adecompress(struct acomp_req *r= eq) return -EINVAL; } =20 - /* Fall back to software if src or dst has multiple sg entries */ - if (sg_nents(req->src) > 1 || sg_nents(req->dst) > 1) + /* Fall back to software if dst has multiple sg entries */ + if (sg_nents(req->dst) > 1) return deflate_generic_decompress(req); =20 + if (sg_nents(req->src) > 1) { + if (req->slen > PAGE_SIZE) + return deflate_generic_decompress(req); + use_bounce_src =3D true; + } + cpu =3D get_cpu(); wq =3D wq_table_next_wq(cpu); put_cpu(); @@ -1623,20 +1669,47 @@ static int iaa_comp_adecompress(struct acomp_req *r= eq) =20 dev =3D &wq->idxd->pdev->dev; =20 - if (!dma_map_sg(dev, req->src, 1, DMA_TO_DEVICE)) { - dev_dbg(dev, "couldn't map src sg for iaa device %d, wq %d\n", - iaa_wq->iaa_device->idxd->id, iaa_wq->wq->id); - iaa_wq_put(wq); - return deflate_generic_decompress(req); + if (unlikely(use_bounce_src)) { + page =3D mempool_alloc(iaa_bounce_pool, GFP_ATOMIC); + if (!page) { + iaa_wq_put(wq); + return deflate_generic_decompress(req); + } + + if (sg_copy_to_buffer(req->src, sg_nents(req->src), + page_address(page), req->slen) !=3D req->slen) { + mempool_free(page, iaa_bounce_pool); + iaa_wq_put(wq); + return deflate_generic_decompress(req); + } + + src_addr =3D dma_map_page(dev, page, 0, req->slen, DMA_TO_DEVICE); + if (dma_mapping_error(dev, src_addr)) { + mempool_free(page, iaa_bounce_pool); + iaa_wq_put(wq); + return deflate_generic_decompress(req); + } + + req_ctx->bounce_src =3D page; + req_ctx->bounce_src_dma =3D src_addr; + req_ctx->bounce_src_len =3D req->slen; + } else { + if (!dma_map_sg(dev, req->src, 1, DMA_TO_DEVICE)) { + dev_dbg(dev, "couldn't map src sg for iaa device %d, wq %d\n", + iaa_wq->iaa_device->idxd->id, iaa_wq->wq->id); + iaa_wq_put(wq); + return deflate_generic_decompress(req); + } + + src_addr =3D sg_dma_address(req->src); + dev_dbg(dev, "map src %llx req->src %p slen %d sg_len %d\n", src_addr, + req->src, req->slen, sg_dma_len(req->src)); } - src_addr =3D sg_dma_address(req->src); - dev_dbg(dev, "map src %llx req->src %p slen %d sg_len %d\n", src_addr, - req->src, req->slen, sg_dma_len(req->src)); =20 if (!dma_map_sg(dev, req->dst, 1, DMA_FROM_DEVICE)) { dev_dbg(dev, "couldn't map dst sg for iaa device %d, wq %d\n", iaa_wq->iaa_device->idxd->id, iaa_wq->wq->id); - dma_unmap_sg(dev, req->src, 1, DMA_TO_DEVICE); + iaa_unmap_src(dev, req); iaa_wq_put(wq); return deflate_generic_decompress(req); } @@ -1653,7 +1726,7 @@ static int iaa_comp_adecompress(struct acomp_req *req) dev_dbg(dev, "asynchronous decompress failed ret=3D%d\n", ret); =20 dma_unmap_sg(dev, req->dst, 1, DMA_FROM_DEVICE); - dma_unmap_sg(dev, req->src, 1, DMA_TO_DEVICE); + iaa_unmap_src(dev, req); iaa_wq_put(wq); =20 return ret; @@ -1687,7 +1760,7 @@ static struct acomp_alg iaa_acomp_fixed_deflate =3D { .cra_driver_name =3D "deflate-iaa", .cra_flags =3D CRYPTO_ALG_ASYNC, .cra_ctxsize =3D sizeof(struct iaa_compression_ctx), - .cra_reqsize =3D sizeof(u32), + .cra_reqsize =3D sizeof(struct iaa_req_ctx), .cra_module =3D THIS_MODULE, .cra_priority =3D IAA_ALG_PRIORITY, } @@ -1886,6 +1959,12 @@ static int __init iaa_crypto_init_module(void) goto err_aecs_init; } =20 + iaa_bounce_pool =3D mempool_create_page_pool(IAA_BOUNCE_POOL_SIZE, 0); + if (!iaa_bounce_pool) { + ret =3D -ENOMEM; + goto err_bounce_pool; + } + ret =3D idxd_driver_register(&iaa_crypto_driver); if (ret) { pr_debug("IAA wq sub-driver registration failed\n"); @@ -1919,6 +1998,9 @@ static int __init iaa_crypto_init_module(void) err_verify_attr_create: idxd_driver_unregister(&iaa_crypto_driver); err_driver_reg: + mempool_destroy(iaa_bounce_pool); + iaa_bounce_pool =3D NULL; +err_bounce_pool: iaa_aecs_cleanup_fixed(); err_aecs_init: =20 @@ -1935,6 +2017,8 @@ static void __exit iaa_crypto_cleanup_module(void) driver_remove_file(&iaa_crypto_driver.drv, &driver_attr_verify_compress); idxd_driver_unregister(&iaa_crypto_driver); + mempool_destroy(iaa_bounce_pool); + iaa_bounce_pool =3D NULL; iaa_aecs_cleanup_fixed(); =20 pr_debug("cleaned up\n"); --=20 2.55.0 From nobody Tue Aug 25 08:46:30 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 495623EE1EE; Wed, 5 Aug 2026 21:20:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964829; cv=none; b=g8oK5vuxD+JSRKlPI5TExs916Dv3zVt0ZWb3u11QIX3RGPeO5otAXh3vdxXrrvpZkx0n4c3/EsieUv6F7WUvmHKG+pJMrcPF9GIRKiCl8hvrJK+K80UxS59hk88Y0n+nfhi9d7yFWEvDtwpaKI3iLqGp4U5bFzVK42/sKJOOD8A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785964829; c=relaxed/simple; bh=fpRoZKbF2e8q5k7sXeU2+xeaef+9Njs626uYt/34cH4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gJ597kcVuclqPd0BYU8ICQ21YalQ7rxD8mxHpo8VJG4yhZRwWziJ7+/GkZ0SzgGxx5UuozAAlccPdVKdhuQyTJvRf4vEgT74O4A5PgeXHakQYvjeC2/qNcRHVDWrKZVMFnTeheZX11Wh7S1O+WKtVahB/MNRcnQ1M2vOW+2H8cY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DJYMWqz2; arc=none smtp.client-ip=198.175.65.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DJYMWqz2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1785964823; x=1817500823; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=fpRoZKbF2e8q5k7sXeU2+xeaef+9Njs626uYt/34cH4=; b=DJYMWqz2wBwOurJjk/eWMay3ESdWP40HS7oH288MdLEdD4IufLuUJGtz s10rU4WRNSwPVpXY1OxodXHA9h1Rp/rCHkMh8vTkoGaIHhRX2DVDC9fjJ MoESPKIBOLAHMmpuv058fCoyhw9J3Hte/VVD7+XmJLvm5g8nXw3PhPTyx 0V5qVnjEOsYRvvtxY8SKIKT+BdIi6UiecUvREkL4TNS6hFdT58ahmJSM+ lWI/E7aqu60RHvhO1kSqdix4wKotIj7zYJ4uFy/RtWioHaFvy3taVWrEi GLe3I0+gVOOGh9Pc63M7HH2JyQbi4JDGtrdaWCGdqqyTM49cS2drDjtqv g==; X-CSE-ConnectionGUID: ASPAX9bFRwaRTTM4syURsA== X-CSE-MsgGUID: +uie/gs0RPu5ylI14gFmuQ== X-IronPort-AV: E=McAfee;i="6800,10657,11866"; a="109341694" X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="109341694" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 X-CSE-ConnectionGUID: Z4I1BAlYQ7iSmKcSmjDDUg== X-CSE-MsgGUID: 6FylbtVgQkikTWR6eBwdYQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,207,1779174000"; d="scan'208";a="300138486" Received: from vcostago-desk1.jf.intel.com (HELO [10.88.27.144]) ([10.88.27.144]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 14:19:49 -0700 From: Vinicius Costa Gomes Date: Wed, 05 Aug 2026 14:19:26 -0700 Subject: [PATCH v2 5/5] crypto: iaa - unmap dst before software fallback on decompress Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260805-iaa-crypto-fixes-zswap-v2-5-55c02694f499@intel.com> References: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> In-Reply-To: <20260805-iaa-crypto-fixes-zswap-v2-0-55c02694f499@intel.com> To: Dave Jiang , Vinod Koul , Frank Li , Kristen Accardi , Herbert Xu , "David S. Miller" , Andrew Morton , Yosry Ahmed , Nhat Pham Cc: dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, Vinicius Costa Gomes , stable@vger.kernel.org X-Mailer: b4 0.16-dev-4217c X-Developer-Signature: v=1; a=ed25519-sha256; t=1785964788; l=3568; i=vinicius.gomes@intel.com; s=20230921; h=from:subject:message-id; bh=fpRoZKbF2e8q5k7sXeU2+xeaef+9Njs626uYt/34cH4=; b=86mR1jS9vLMsfNcxuBjSZQZnzXqC3W5RubH6HCjxOCxSHdw5ZSd+pj6DXrsVVUNayQLHURzkU E3y2/Ba7jgDCQD39ss8oZr2d+AXCwQOykwcw0AfnZ17/19E8mF+aKFP X-Developer-Key: i=vinicius.gomes@intel.com; a=ed25519; pk=aJkrtgqgT6TZ8iIHSG8/rTPsmlYnjMrUjCsMYvCzntk= On a hardware analytics error, decompress retries through the software fallback, which writes req->dst with the CPU while it is still mapped DMA_FROM_DEVICE. With SWIOTLB active the later dma_unmap_sg() copies the stale bounce buffer over req->dst, corrupting the result. Unmap before the fallback runs. The async path unmaps inline; the sync path signals the retry with -EAGAIN so iaa_comp_adecompress() runs the fallback after unmapping. Fixes: 2ec6761df889 ("crypto: iaa - Add support for deflate-iaa compression= algorithm") Cc: stable@vger.kernel.org Signed-off-by: Vinicius Costa Gomes --- drivers/crypto/intel/iaa/iaa_crypto_main.c | 35 +++++++++++++++-----------= ---- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/drivers/crypto/intel/iaa/iaa_crypto_main.c b/drivers/crypto/in= tel/iaa/iaa_crypto_main.c index 51547c5fcf70..c9ab4b83ae02 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_main.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_main.c @@ -995,6 +995,11 @@ static inline int check_completion(struct device *dev, return ret; } =20 +static bool iaa_error_should_retry(struct idxd_desc *idxd_desc) +{ + return idxd_desc->iax_completion->status =3D=3D IAA_ANALYTICS_ERROR; +} + static void iaa_unmap_src(struct device *dev, struct acomp_req *req) { struct iaa_req_ctx *req_ctx =3D acomp_request_ctx(req); @@ -1082,18 +1087,21 @@ static void iaa_desc_complete(struct idxd_desc *idx= d_desc, ctx->compress, false); if (ret) { dev_dbg(dev, "%s: check_completion failed ret=3D%d\n", __func__, ret); - if (!ctx->compress && - idxd_desc->iax_completion->status =3D=3D IAA_ANALYTICS_ERROR) { + if (!ctx->compress && iaa_error_should_retry(idxd_desc)) { pr_warn("%s: falling back to deflate-generic decompress, " "analytics error code %x\n", __func__, idxd_desc->iax_completion->error_code); + dma_unmap_sg(dev, ctx->req->dst, sg_nents(ctx->req->dst), + DMA_FROM_DEVICE); + iaa_unmap_src(dev, ctx->req); + ret =3D deflate_generic_decompress(ctx->req); if (ret) { dev_dbg(dev, "%s: deflate-generic failed ret=3D%d\n", __func__, ret); err =3D -EIO; - goto err; } + goto out; } else { err =3D -EIO; goto err; @@ -1477,19 +1485,9 @@ static int iaa_decompress(struct crypto_tfm *tfm, st= ruct acomp_req *req, ret =3D check_completion(dev, idxd_desc->iax_completion, false, false); if (ret) { dev_dbg(dev, "%s: check_completion failed ret=3D%d\n", __func__, ret); - if (idxd_desc->iax_completion->status =3D=3D IAA_ANALYTICS_ERROR) { - pr_warn("%s: falling back to deflate-generic decompress, " - "analytics error code %x\n", __func__, - idxd_desc->iax_completion->error_code); - ret =3D deflate_generic_decompress(req); - if (ret) { - dev_dbg(dev, "%s: deflate-generic failed ret=3D%d\n", - __func__, ret); - goto err; - } - } else { - goto err; - } + if (iaa_error_should_retry(idxd_desc)) + ret =3D -EAGAIN; + goto err; } else { req->dlen =3D idxd_desc->iax_completion->output_size; =20 @@ -1722,13 +1720,16 @@ static int iaa_comp_adecompress(struct acomp_req *r= eq) if (ret =3D=3D -EINPROGRESS) return ret; =20 - if (ret !=3D 0) + if (ret !=3D 0 && ret !=3D -EAGAIN) dev_dbg(dev, "asynchronous decompress failed ret=3D%d\n", ret); =20 dma_unmap_sg(dev, req->dst, 1, DMA_FROM_DEVICE); iaa_unmap_src(dev, req); iaa_wq_put(wq); =20 + if (ret =3D=3D -EAGAIN) + ret =3D deflate_generic_decompress(req); + return ret; } =20 --=20 2.55.0