From nobody Fri Sep 25 10:04:10 2026 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3288A47143C for ; Mon, 14 Sep 2026 13:37:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789393077; cv=none; b=S9DYyf64VEEkELzxyeSPSmQEHLVSpX6P3UL4KOVx0K2ehuoIGYtLXjcKSvoGTxNMi6HTk/fARHAagNwgIHvZqqbLGNHdPxVcc4gBa5LIHLtuMIfS1Upghs+918Pw/GiUyqVH3qzvJGUGy8Ruf0GUJzptk5q4sBbXIHGeIEX1fOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789393077; c=relaxed/simple; bh=jCoZuWEgSbjtm4wLSbpNx2rVBrFvTgX7s+RrR3f+ndo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HQyFHPX2678zW2IYPTtETGWooDNILIoTo9j8CaNXtd1HROZ1IaIN3pQ/ZePVTHSBfHgCMYcdMTj3yXoG+eq7L5kuYzzlA9GVCrcnMImPWkjo3jtptYzgmlp7dg+B/V+aLXjPBlfl/R6CI8GvN7U1tPrexJjXbPzS4RXCiTz1+1g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=dBkGbM0V; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=XnwImuxF; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="dBkGbM0V"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="XnwImuxF" Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 68ED0hn8135657 for ; Mon, 14 Sep 2026 13:37:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=qcppdkim1; bh=/m1IO6ReHXB nIHxR1Y+am6Pid+itc/UswH9TAnSjnTQ=; b=dBkGbM0Ve6n706ia3z2t5c/CKA0 YA/WNHr0UQqdN0vpuG55FOY+oQd90RfxhuGlBSK9bWfloeXOcBLr4Cc23335Mgze UQ2yKmHjadTyHboxLOvitnBa7QRm6uYoYgWFIpF7tyjZPWFDliZSwocG5d9/ghMh 4TNKkr9wqquUxFm4rpowsrnlRT6EzdhLfki6s3IZStB+NWnog7nWoRhigMYpEKB9 /9O/rocBAzOvKWKv/BOXT4/OvdmrrBWMpzsIzeVevSdnYvXB145OzxBc+fwwW1Qk Q+VVmwDy1w0+saRYuGx/CViOM4e2Gd6l76jgFhQkwvul2xLHPGQgzFseiUg== Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4gpeuq8sss-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Mon, 14 Sep 2026 13:37:53 +0000 (GMT) Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-39533bb224cso4040046a91.3 for ; Mon, 14 Sep 2026 06:37:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1789393068; x=1789997868; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/m1IO6ReHXBnIHxR1Y+am6Pid+itc/UswH9TAnSjnTQ=; b=XnwImuxF8J60BkGL1pyMaS8pap2cQZ8jFaxKa2d6KzAKYf6HAi/IAIrsTylWwrt5V4 K2Zyh2SBGvU/cY/vUtZN+N3NL34s8zhuo14Ja0OFjxRaQne8sMxzWo6c4agEXp12GJ/i GFYziDzCL7W3EivWhkFy96mqYAcnK9YeoaMnftqJ/dzK8xy1ntsCXtmDYADH4U8Aavli T/F6bG0p2nYKQIwDAT3vz9Ql+EYNzjTxRb/EDIYGdldRTsaxtC8VM2P0h6d51Ze30Bud 1Kj2P7/lw/kwq63jsMBrUP3O+EmQA5BXYKb3aEOEONX9uvX13AA3abivTdFAANtfuLTh mzMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789393068; x=1789997868; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/m1IO6ReHXBnIHxR1Y+am6Pid+itc/UswH9TAnSjnTQ=; b=NMBwZKsxsQXPCm93SEugHjQC1sBgFG29s1ZNH9f9nX+w6C/r0Mjv1ANFmqXEETDB+w bHslzwlJb1xByR7rEayIQoMlK9/kUona8EBPeLFo0JNX1xwEntPqa5ncQ0rueqYgSwWl Oh8iP4WgYfb8K08orhkAXkuFsYGD5DlCLixvgTY9rCQeqpPq3Qzk+jQplqKFxiYnfkeH 3KUBeL3jsASMCdnlqhIwfI7pVnpv9hWlv+kdhysxETQF00qafjDY6wGh6Tst+4NYYKmo qlofhqHOA3o18iP8XXlQjHBJ91RzKX3kayuOD+j/k90dGmNWa+153TeHsOxN9D6D23Ct PT1A== X-Forwarded-Encrypted: i=1; AKwUvBwdXEi9UUTxmLfHrpSxzrc/MZTfHIHGFRV85bYIFMbG7r6zBczSjIEvEpUjxZ0GOVC/BNhmwZUabFNLXl8=@vger.kernel.org X-Gm-Message-State: AFuF++lr7IydoemQbpspyLYqReD2QlFkGsDJ3g0mml5jW6HKDb60ou7L 8EUV1DWKNDbv+WA6r86lZgF8pTq2qOE4YbDXeUEKTvEqAvEGx5ZHingnDiT5Dh5EtRYEYWjvMcc taP8g0ZdchTwDj2kPVT8osp8ilo6FyjzRJiufbgGIaBFc1AywAOqgb7/UIHdq4mUrO9k= X-Gm-Gg: AYBFou1f1MYb5WSmoT+FmOBTOgVysI315Bly4GhB023BT1fpmuULpetUuDOKJP3eRi0 FIo+x2/IZodtGUEAJY4tdSYWr/eOMnv59C2BJ5Re+ML3oSl1TpFl940D9mBqQkwsnBrMMll8v+k ev5c5m0ZgoFNwV3Eqo6ibIoLY5sSuNZpJqnYcWlN2jXPxSDA7b1iFCHQaY1fby4HJ8eA63qfNX1 AZiwSNOubwVD3a0LJF20YiNv1uUjHB3UCS5JQ9zjqMKmHIrJ7MJ1rB9zEq746t7YddcCPislg+K GOb3M1iS9vOeHni5qGutbTk/TZNkycDyKBzs9DxAQkm4ye+TxCUwkTUA+1oLOWdYN2bufu4ZhUX 9ZyThkzN7ZvVfQqNr/bRrMBLoBwZjPWQlj6dBLQVKLAqQKzvlYscwPIfgW+o= X-Received: by 2002:a17:902:db12:b0:2d9:6db:9fcb with SMTP id d9443c01a7336-2dd6c6eb73emr53966015ad.11.1789393068074; Mon, 14 Sep 2026 06:37:48 -0700 (PDT) X-Received: by 2002:a17:902:db12:b0:2d9:6db:9fcb with SMTP id d9443c01a7336-2dd6c6eb73emr53965255ad.11.1789393067452; Mon, 14 Sep 2026 06:37:47 -0700 (PDT) Received: from u24-san1p10108.qualcomm.com (i-global254.qualcomm.com. [199.106.103.254]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2dd2cfd8c84sm49132685ad.62.2026.09.14.06.37.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Sep 2026 06:37:47 -0700 (PDT) From: Linlin Zhang To: mst@redhat.com, jasowangio@gmail.com, axboe@kernel.dk, ebiggers@kernel.org, stefanha@redhat.com Cc: pbonzini@redhat.com, eperezma@redhat.com, xuanzhuo@linux.alibaba.com, virtualization@lists.linux.dev, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/2] virtio_blk: Add control virtqueue support Date: Mon, 14 Sep 2026 06:37:20 -0700 Message-ID: <20260914133733.15429-2-linlin.zhang@oss.qualcomm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260914133733.15429-1-linlin.zhang@oss.qualcomm.com> References: <20260914133733.15429-1-linlin.zhang@oss.qualcomm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTE0MDE5NCBTYWx0ZWRfXyPoXndFWgBQw wvVieKDnhzQq6kbSppgBe0fYzvo2qCXDUaylqfYu3hzY4QRogDRRELci/MHYHlq5Hz4u+wvK+Mq PgGyQerQL4CO3zq/YsWkw8BcjJ59h7XuBbp3gDOUrkiJ/ZmJach8OazAWhWmxwVnvfK3AaGpMMI DD2JlYKV1GGKiOLg6HXfL6RroiCGawVxoOegeK5gKik3Y4fFwT07K/F34XH6FUHZGeOYS0duEYN xN0m5FjIeqKtX1qwD3fC6HS8GRiwKJCFm3gmOImbaDC2DmXKpshLQZZwmZBaJZmmxs6wD79ACaV i2NZLySI4tGEnb6rK7i7wcOQyR1i9klWqJo3T0oEwgPvRNTO9goSwe4KZe8kseZ/BjXI/8Si/fQ CFlwybvJf75tD15MKh12E8OchBZykUyXGmPcmnp9byrAPqZsI6iJ3DFoPrYRFtIA0aoBk2vzlbJ WQSy+72sYzStMc2UDNw== X-Authority-Analysis: v=2.4 cv=TrFzFzXh c=1 sm=1 tr=0 ts=6aa7f8b1 cx=c_pps a=UNFcQwm+pnOIJct1K4W+Mw==:117 a=JYp8KDb2vCoCEuGobkYCKw==:17 a=VdqzKS8jKosA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=_K5XuSEh1TEqbUxoQ0s3:22 a=EUspDBNiAAAA:8 a=3eFvK4jGiganB99P3z8A:9 a=uKXjsCUrEbL0IQVhDsJ9:22 X-Proofpoint-ORIG-GUID: _MCGYkufsoW1iBOG4BK25G6vecp23S8V X-Proofpoint-Spam-Info: AW1haW4tMjYwOTE0MDE5NCBTYWx0ZWRfX6sPbwFfxyrFJ 5Xhl+g10MK0Hqi+o4taOwZJY0s1RwAeHVttpGB3oJpOxyLaWEkwuzED2uNo384iaLxTxCTIZbpQ ILUdOeE9sWozVBL6ozPbkm7RXrdUvdQ= X-Proofpoint-GUID: _MCGYkufsoW1iBOG4BK25G6vecp23S8V X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-14_03,2026-09-14_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 priorityscore=1501 lowpriorityscore=0 bulkscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2609040000 definitions=main-2609140194 Content-Type: text/plain; charset="utf-8" From: linlzhan Add support for the optional virtio-blk control virtqueue. If control queue feature bit is negociated, this allows the driver to manage control-queue requests independently from the data path and to safely handle outstanding requests during device removal and suspend. No control command is submitted by this change. The control virtqueue will be used by a subsequent inline encryption implementation. Signed-off-by: linlzhan --- drivers/block/virtio_blk.c | 179 +++++++++++++++++++++++++++++++- include/uapi/linux/virtio_blk.h | 1 + 2 files changed, 176 insertions(+), 4 deletions(-) diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 32bf3ba07a9d..252ec9366153 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -52,6 +53,15 @@ struct virtio_blk_vq { char name[VQ_NAME_LEN]; } ____cacheline_aligned_in_smp; =20 +struct virtio_blk_ctrl_vq { + struct virtqueue *vq; + struct mutex mutex; + spinlock_t lock; + unsigned int inflight; + bool dead; + struct completion drained; +}; + struct virtio_blk { /* * This mutex must be held by anything that may run after @@ -83,6 +93,9 @@ struct virtio_blk { =20 /* For zoned device */ unsigned int zone_sectors; + + /* Control virtqueue state. */ + struct virtio_blk_ctrl_vq ctrl_vq; }; =20 struct virtblk_req { @@ -110,6 +123,12 @@ struct virtblk_req { struct scatterlist sg[]; }; =20 +struct virtblk_ctrl_request { + __virtio32 type; + u8 status; + struct completion compl; +}; + static inline blk_status_t virtblk_result(u8 status) { switch (status) { @@ -863,11 +882,131 @@ static int virtblk_getgeo(struct gendisk *disk, stru= ct hd_geometry *geo) return ret; } =20 +#define VIRTBLK_CTRL_VQ_DRAIN_TIMEOUT (10 * HZ) + +/* Prevent new submissions and wait for in-flight requests to complete. */ +static void virtblk_ctrl_vq_quiesce(struct virtio_blk *vblk) +{ + unsigned long flags; + bool need_wait; + + if (!vblk->ctrl_vq.vq) + return; + + init_completion(&vblk->ctrl_vq.drained); + + spin_lock_irqsave(&vblk->ctrl_vq.lock, flags); + vblk->ctrl_vq.dead =3D true; + need_wait =3D vblk->ctrl_vq.inflight !=3D 0; + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); + + if (need_wait && + !wait_for_completion_timeout(&vblk->ctrl_vq.drained, + VIRTBLK_CTRL_VQ_DRAIN_TIMEOUT)) + dev_warn(&vblk->vdev->dev, + "timed out waiting for control queue requests to complete\n"); +} + +/* Fail requests left in the control queue after reset. */ +static void virtblk_ctrl_vq_drain(struct virtio_blk *vblk) +{ + struct virtblk_ctrl_request *creq; + unsigned long flags; + + if (!vblk->ctrl_vq.vq) + return; + + spin_lock_irqsave(&vblk->ctrl_vq.lock, flags); + while ((creq =3D virtqueue_detach_unused_buf(vblk->ctrl_vq.vq)) !=3D NULL= ) { + if (WARN_ON_ONCE(!vblk->ctrl_vq.inflight)) + ; + else + vblk->ctrl_vq.inflight--; + creq->status =3D VIRTIO_BLK_S_IOERR; + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); + complete(&creq->compl); + spin_lock_irqsave(&vblk->ctrl_vq.lock, flags); + } + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); +} + +static void virtblk_ctrlq_callback(struct virtqueue *vq) +{ + struct virtio_blk *vblk =3D vq->vdev->priv; + struct virtblk_ctrl_request *creq; + unsigned long flags; + unsigned int len; + + spin_lock_irqsave(&vblk->ctrl_vq.lock, flags); + do { + virtqueue_disable_cb(vq); + while ((creq =3D virtqueue_get_buf(vq, &len)) !=3D NULL) { + bool drained =3D false; + + if (WARN_ON_ONCE(!vblk->ctrl_vq.inflight)) { + /* + * Still complete the request. Never leave a + * synchronous caller blocked because the accounting + * state was already inconsistent. + */ + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); + complete(&creq->compl); + spin_lock_irqsave(&vblk->ctrl_vq.lock, flags); + continue; + } + + if (--vblk->ctrl_vq.inflight =3D=3D 0 && vblk->ctrl_vq.dead) + drained =3D true; + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); + if (drained) + complete(&vblk->ctrl_vq.drained); + complete(&creq->compl); + spin_lock_irqsave(&vblk->ctrl_vq.lock, flags); + } + } while (!virtqueue_enable_cb(vq)); + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); +} + +/* Submit a control-queue request and wait for completion. */ +static int virtblk_ctrl_vq_request(struct virtio_blk *vblk, + struct virtblk_ctrl_request *creq, + struct scatterlist *sgs[], + unsigned int out_sgs, unsigned int in_sgs) +{ + unsigned long flags; + int err; + + mutex_lock(&vblk->ctrl_vq.mutex); + init_completion(&creq->compl); + + spin_lock_irqsave(&vblk->ctrl_vq.lock, flags); + if (vblk->ctrl_vq.dead) { + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); + mutex_unlock(&vblk->ctrl_vq.mutex); + return -ENODEV; + } + err =3D virtqueue_add_sgs(vblk->ctrl_vq.vq, sgs, out_sgs, in_sgs, creq, G= FP_ATOMIC); + if (!err) { + vblk->ctrl_vq.inflight++; + virtqueue_kick(vblk->ctrl_vq.vq); + } + spin_unlock_irqrestore(&vblk->ctrl_vq.lock, flags); + if (err) { + mutex_unlock(&vblk->ctrl_vq.mutex); + return err; + } + + wait_for_completion(&creq->compl); + mutex_unlock(&vblk->ctrl_vq.mutex); + return 0; +} + static void virtblk_free_disk(struct gendisk *disk) { struct virtio_blk *vblk =3D disk->private_data; =20 ida_free(&vd_index_ida, vblk->index); + mutex_destroy(&vblk->ctrl_vq.mutex); mutex_destroy(&vblk->vdev_mutex); kfree(vblk); } @@ -965,6 +1104,8 @@ static int init_vq(struct virtio_blk *vblk) struct virtqueue **vqs; unsigned short num_vqs; unsigned short num_poll_vqs; + unsigned short total_vqs; + bool has_ctrl_vq; struct virtio_device *vdev =3D vblk->vdev; struct irq_affinity desc =3D { 0, }; =20 @@ -993,12 +1134,19 @@ static int init_vq(struct virtio_blk *vblk) vblk->io_queues[HCTX_TYPE_READ], vblk->io_queues[HCTX_TYPE_POLL]); =20 + /* + * The control vq is appended after the data vqs whenever + * F_CTRL_VQ is negotiated. + */ + has_ctrl_vq =3D virtio_has_feature(vdev, VIRTIO_BLK_F_CTRL_VQ); + total_vqs =3D num_vqs + (has_ctrl_vq ? 1 : 0); + vblk->vqs =3D kmalloc_objs(*vblk->vqs, num_vqs); if (!vblk->vqs) return -ENOMEM; =20 - vqs_info =3D kzalloc_objs(*vqs_info, num_vqs); - vqs =3D kmalloc_objs(*vqs, num_vqs); + vqs_info =3D kzalloc_objs(*vqs_info, total_vqs); + vqs =3D kmalloc_objs(*vqs, total_vqs); if (!vqs_info || !vqs) { err =3D -ENOMEM; goto out; @@ -1015,8 +1163,13 @@ static int init_vq(struct virtio_blk *vblk) vqs_info[i].name =3D vblk->vqs[i].name; } =20 + if (has_ctrl_vq) { + vqs_info[num_vqs].callback =3D virtblk_ctrlq_callback; + vqs_info[num_vqs].name =3D "control"; + } + /* Discover virtqueues and write information to configuration. */ - err =3D virtio_find_vqs(vdev, num_vqs, vqs, vqs_info, &desc); + err =3D virtio_find_vqs(vdev, total_vqs, vqs, vqs_info, &desc); if (err) goto out; =20 @@ -1025,6 +1178,9 @@ static int init_vq(struct virtio_blk *vblk) vblk->vqs[i].vq =3D vqs[i]; } vblk->num_vqs =3D num_vqs; + vblk->ctrl_vq.vq =3D has_ctrl_vq ? vqs[num_vqs] : NULL; + vblk->ctrl_vq.dead =3D false; + vblk->ctrl_vq.inflight =3D 0; =20 out: kfree(vqs); @@ -1464,14 +1620,18 @@ static int virtblk_probe(struct virtio_device *vdev) } =20 mutex_init(&vblk->vdev_mutex); + mutex_init(&vblk->ctrl_vq.mutex); + spin_lock_init(&vblk->ctrl_vq.lock); =20 vblk->vdev =3D vdev; =20 INIT_WORK(&vblk->config_work, virtblk_config_changed_work); =20 err =3D init_vq(vblk); - if (err) + if (err) { + dev_err(&vdev->dev, "init virt queue failed: err =3D %d\n", err); goto out_free_vblk; + } =20 /* Default queue sizing is to fill the ring. */ if (!virtblk_queue_depth) { @@ -1553,6 +1713,7 @@ static int virtblk_probe(struct virtio_device *vdev) out_free_vq: vdev->config->del_vqs(vdev); kfree(vblk->vqs); + vblk->ctrl_vq.vq =3D NULL; out_free_vblk: kfree(vblk); out_free_index: @@ -1571,16 +1732,21 @@ static void virtblk_remove(struct virtio_device *vd= ev) del_gendisk(vblk->disk); blk_mq_free_tag_set(&vblk->tag_set); =20 + virtblk_ctrl_vq_quiesce(vblk); + mutex_lock(&vblk->vdev_mutex); =20 /* Stop all the virtqueues. */ virtio_reset_device(vdev); + virtblk_ctrl_vq_drain(vblk); =20 /* Virtqueues are stopped, nothing can use vblk->vdev anymore. */ vblk->vdev =3D NULL; =20 vdev->config->del_vqs(vdev); kfree(vblk->vqs); + vblk->vqs =3D NULL; + vblk->ctrl_vq.vq =3D NULL; =20 mutex_unlock(&vblk->vdev_mutex); =20 @@ -1593,6 +1759,8 @@ static int virtblk_freeze_priv(struct virtio_device *= vdev) struct request_queue *q =3D vblk->disk->queue; unsigned int memflags; =20 + virtblk_ctrl_vq_quiesce(vblk); + /* Ensure no requests in virtqueues before deleting vqs. */ memflags =3D blk_mq_freeze_queue(q); blk_mq_quiesce_queue_nowait(q); @@ -1600,6 +1768,7 @@ static int virtblk_freeze_priv(struct virtio_device *= vdev) =20 /* Ensure we don't receive any more interrupts */ virtio_reset_device(vdev); + virtblk_ctrl_vq_drain(vblk); =20 /* Make sure no work handler is accessing the device. */ flush_work(&vblk->config_work); @@ -1612,6 +1781,7 @@ static int virtblk_freeze_priv(struct virtio_device *= vdev) * pointers safely. */ vblk->vqs =3D NULL; + vblk->ctrl_vq.vq =3D NULL; =20 return 0; } @@ -1672,6 +1842,7 @@ static unsigned int features[] =3D { VIRTIO_BLK_F_FLUSH, VIRTIO_BLK_F_TOPOLOGY, VIRTIO_BLK_F_CONFIG_WCE, VIRTIO_BLK_F_MQ, VIRTIO_BLK_F_DISCARD, VIRTIO_BLK_F_WRITE_ZEROES, VIRTIO_BLK_F_SECURE_ERASE, VIRTIO_BLK_F_ZONED, + VIRTIO_BLK_F_CTRL_VQ, }; =20 static struct virtio_driver virtio_blk =3D { diff --git a/include/uapi/linux/virtio_blk.h b/include/uapi/linux/virtio_bl= k.h index 3744e4da1b2a..0a16972a1535 100644 --- a/include/uapi/linux/virtio_blk.h +++ b/include/uapi/linux/virtio_blk.h @@ -42,6 +42,7 @@ #define VIRTIO_BLK_F_WRITE_ZEROES 14 /* WRITE ZEROES is supported */ #define VIRTIO_BLK_F_SECURE_ERASE 16 /* Secure Erase is supported */ #define VIRTIO_BLK_F_ZONED 17 /* Zoned block device */ +#define VIRTIO_BLK_F_CTRL_VQ 22 /* Control queue */ =20 /* Legacy feature bits */ #ifndef VIRTIO_BLK_NO_LEGACY --=20 2.34.1 From nobody Fri Sep 25 10:04:10 2026 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFE8D476CF2 for ; Mon, 14 Sep 2026 13:37:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789393077; cv=none; b=E8QIkLb0L+2nnk6hPQ0wtkWgYrPLoET0KcOfxjfYZ0gnLnp9CyFGKE7m+Bt+lazuJeItK4au30X9Lzso0VQmKsat2H4cnll+eJMXXmMdqS5HXGQbD6lNYjccQ0p12tvrS4uA85JQ0zCNQ92WIuGEJn5usqmTLkvdhzsuyG0s/W4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789393077; c=relaxed/simple; bh=YiKh5V+YEc0HG9Sg0HziwaHjw7xOx3e7esxCZsefzTk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=itwOp1O/oKV0bT+zcBWpsULn2tIYtwKLhJMR/HM/sAYac9hAp3zRyF/61/9+0T3HB6KBscssbIWc7iOgBX2YirV0ZcWEgRJutIcZs8NDOA54hMYLwWuKrGZ5yb4+bbBQLuZXx3dtl1Os+T3ENEHdyAlsp0rsWjqSYRW20hSXFR4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=iQLxKayy; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=Rq8vtfI7; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="iQLxKayy"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="Rq8vtfI7" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 68ED0bYI3588221 for ; Mon, 14 Sep 2026 13:37:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=qcppdkim1; bh=sHzJV7DoUuy lvx+MlonA9cvYxJqTQaH9CVz5nVLjKUA=; b=iQLxKayyrgGr0uSMBCH9YPwJbkH qN+GNB8V3O16BGXxqxoVCMLFpF4xl6b8Pj2Xl1PgmTWcaT7kkHRCUzKmaOpEAmZM csfuQTqLmH4k36T4Yw8knnNdyXLxOzs6ZyCyTmwQUYHn6rHFNy4bnbgTU56La877 bRX5Qgy1Bc7HcMLY+TYUpjOKhLqq/aQzeopXb0rkJtyAuCG9AmaXtfRQ8EJfpEPI uNgbwWHgiSotd4a6e50nYMe9OdyZUotpzBOF1cvZWOLMNsxjJXAN3EH8eIuBeqVZ kfNqFnSrj0o4OEJmf/m1S6P63vsgm35f4m+MY/NJLQmF9BG8YIMZa5RxMqg== Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4gperg8v7v-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Mon, 14 Sep 2026 13:37:52 +0000 (GMT) Received: by mail-pg1-f199.google.com with SMTP id 41be03b00d2f7-cc18ced1a5aso6425693a12.1 for ; Mon, 14 Sep 2026 06:37:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1789393071; x=1789997871; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=sHzJV7DoUuylvx+MlonA9cvYxJqTQaH9CVz5nVLjKUA=; b=Rq8vtfI7wg1FZmVjTVqL4v5WdAHuFLrBWIF6Lt9/SpHMdVj2C8jvfbtG/H+ocfu3qz 69jxfKQzDUppFi+OfgIU5P3cR0okGPJfdDug8/6vBxV0ee2BiTFlEjiJTch9wvPv4bR/ ul5dpZQ+c/Iuwv7PHzoVrTmkdEBzJageIQdNRy0U0WUnBWM4d7r88Jw1iK7VjzzW6KSX umbqoCMSVuC0mBWiW9i21pEuIHRMgx+YHpCE6rqJ//uY591frNz9dLrMWKbfBa08WRKT lZU0Jui1o9hRsUKN+CJcx/QsyMCAVMdCLPsZicCoK8jBSC5Z0JZp42ACLmZYOODx4bcH vD9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789393071; x=1789997871; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=sHzJV7DoUuylvx+MlonA9cvYxJqTQaH9CVz5nVLjKUA=; b=G2qVRCDA8PA9qZen6YqKkbwD6WMlbsD+TwU41crnjmxvrMaNYP+4wfBshVbsoaXoil ubePd8icEgAgmzuX39/Ip39mQPOHYhVpzuDL/jxvFU36/r5rq/BM93FWttqF2fYuu0Iy g28/pWVHdf/2w1x7x74EQEH5SF10fYZnZ0VLxyhpE6vzV1dzo2vdGttOQ9DLhDQjFxKB VoroUMjN4sKEDTPV6tOINp3U5wz2GvQYypxh/PCO54kI1f9V17duSmOGPBDd/+GqLN1E UUNqvVkJs1K8DMYPoZ0BvX3e1qn1ztKsHf0USO3hj5zDI8j9I9VnLS150O/KxliBoupI CORg== X-Forwarded-Encrypted: i=1; AKwUvBwuUSrErYT9AO6L6ttZrPpxECGYpNb4WaGPmHCHIfdpMp/sZtf38ShQ4saD37fbRaLbPKxF1g8kuvEHv8g=@vger.kernel.org X-Gm-Message-State: AFuF++lUZSJM+uHk79lP/nvqfxJY4R0QZ84BBygLozbU9FUI7YwrwCFd INebMeKOy9ocBaUyYEEwsfZlk+/OtCAsRBmUOO9wE1QsRsKGfxmsAqZ4r85JxK04xWYJThat0Nr XkwM6A74/l9GixE8GmSgi8vtP3tR1OOjBLVn4FmAwRs60JTdc4KvHKIS/u5lKKWamLzo= X-Gm-Gg: AYBFou1iupXLxQZ5/1sVnZZOdK52TIH36wizz1Zg6tWz6LtAqleWviWj4+3+1iT8FIl 8GVFpMbwAOwy14haLG53Xy9ZlvqWWtPCoMdcBrB7RHB9EA1bWQdmHygr2IyoGXPernUs06ARhQy FUqa3SL+7M2Kxzhp5ZG+LagwBhNMnlaP+FtC8+ulaermA6dmZczc9VEJwh3RWrSgcWH6YrQG5ni wGN6ZVutUOXuQVFHWq3/jZHFurzoEhubHOjfMk9bpYo7e3hWmZhPGY1eA1YMcCPmS0FFju9fuiw DDoAaJnCF+b6mhucqenAnmJ6AVeut+4/pmU6yusEKiHRuLF2JgegI1qh/nq5n7skcw5vzDmCk3s R7jyze9jKbjDkLnaHOuXZfqQd5rjkPBQlNWlbY2amNEMoOY/XfdZ4vkjDPec= X-Received: by 2002:a17:902:e744:b0:2bf:dd0:c8b1 with SMTP id d9443c01a7336-2dd6c4835d0mr52450625ad.0.1789393070151; Mon, 14 Sep 2026 06:37:50 -0700 (PDT) X-Received: by 2002:a17:902:e744:b0:2bf:dd0:c8b1 with SMTP id d9443c01a7336-2dd6c4835d0mr52449805ad.0.1789393069350; Mon, 14 Sep 2026 06:37:49 -0700 (PDT) Received: from u24-san1p10108.qualcomm.com (i-global254.qualcomm.com. [199.106.103.254]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2dd2cfd8c84sm49132685ad.62.2026.09.14.06.37.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Sep 2026 06:37:48 -0700 (PDT) From: Linlin Zhang To: mst@redhat.com, jasowangio@gmail.com, axboe@kernel.dk, ebiggers@kernel.org, stefanha@redhat.com Cc: pbonzini@redhat.com, eperezma@redhat.com, xuanzhuo@linux.alibaba.com, virtualization@lists.linux.dev, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/2] virtio_blk: add inline encryption support Date: Mon, 14 Sep 2026 06:37:21 -0700 Message-ID: <20260914133733.15429-3-linlin.zhang@oss.qualcomm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260914133733.15429-1-linlin.zhang@oss.qualcomm.com> References: <20260914133733.15429-1-linlin.zhang@oss.qualcomm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Authority-Analysis: v=2.4 cv=QK/91QLL c=1 sm=1 tr=0 ts=6aa7f8b0 cx=c_pps a=Oh5Dbbf/trHjhBongsHeRQ==:117 a=JYp8KDb2vCoCEuGobkYCKw==:17 a=VdqzKS8jKosA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=DJpcGTmdVt4CTyJn9g5Z:22 a=EUspDBNiAAAA:8 a=7cB_SFYodWyvrme8HfkA:9 a=_Vgx9l1VpLgwpw_dHYaR:22 X-Proofpoint-GUID: 6Cg5IhDM3jMGcuptuU0Rw3GWO9Gx61w3 X-Proofpoint-ORIG-GUID: 6Cg5IhDM3jMGcuptuU0Rw3GWO9Gx61w3 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTE0MDE5NCBTYWx0ZWRfXwy7J6g+W3LHG eDO0ELcJ/MhqW2NiHo7sBgY9jNzrt2VEC2yrSR118SM2AHma01ZETd//xDeLt+qXk0zkHQTcRiY IdWYyROpFfXsG08MRd1aDtV+BqVmIABQ1MJmHiR/imfugEHTn6hWVqo12t5BNjb5Q5dvcXR/QZQ GveGMSyGHYG4jIOpWWL4yo1vxy3l2cGHsao+JQ7npKkCBHFL3sHnOkt4GeHmHsjGir6aNc+Wxdo OkoFQFsnwWOTOgNTz7iP976t04LelO3Fc3ESsA9SHbtaRjJVpkRKsCxanFlboWAYCF5SCLWIleZ JHFtTk2GA55Wt8SFq+yPYlh3fHECalWbHnZBF+d8Inzd5O6cHqJ50EdzMFZvqA/uWNk1w5HKO1Z 16OzjWupoFpsgNKnBw1b9SjhKF38no+x+HQy5e+hH3uPMd/kNnk3tqrpaxVu9ccOaDb8SmUZnWS /oZOMb80E01syEX/vKg== X-Proofpoint-Spam-Info: AW1haW4tMjYwOTE0MDE5NCBTYWx0ZWRfX2wgwbsrOVVeO 7jxLuEgnXWGbGO7kIFCyebWhJiI4Z62NIaWx8InGiCgJTq9EoS51a2Uen2SlPIFXFBnpA65691P 0ynoKGBP33uq8x0d8iSXMbQZcmQ9oCs= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-14_03,2026-09-14_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 suspectscore=0 lowpriorityscore=0 priorityscore=1501 spamscore=0 adultscore=0 bulkscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2609040000 definitions=main-2609140194 Content-Type: text/plain; charset="utf-8" From: linlzhan Add support for the virtio-blk inline encryption feature (VIRTIO_BLK_F_INLINE_ENCRYPTION), which lets the guest offload per-I/O encryption to the host's inline crypto engine instead of doing it in software in the guest. Advertise the device's inline encryption characteristics (key types, supported crypto modes, max keyslots, DUN size) and wire them up to a struct blk_crypto_profile so upper layers can attach encryption contexts to bios as usual. Key management (program, evict, generate, import, prepare key, derive software secret) is carried out as control commands over the control virtqueue, and per-request keyslot and data unit number are carried in an extended request header for read/write commands. Inline encryption is only enabled when both the control virtqueue and inline encryption feature bits are negotiated, and is gated behind a new VIRTIO_BLK_INLINE_ENCRYPTION Kconfig option that depends on BLK_INLINE_ENCRYPTION, so kernels that don't select it are unaffected. Signed-off-by: linlzhan --- drivers/block/Kconfig | 12 + drivers/block/virtio_blk.c | 576 +++++++++++++++++++++++++++++++- include/uapi/linux/virtio_blk.h | 115 +++++++ 3 files changed, 690 insertions(+), 13 deletions(-) diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index 858320b6ebb7..58bb050d4617 100644 --- a/drivers/block/Kconfig +++ b/drivers/block/Kconfig @@ -372,4 +372,16 @@ config BLK_DEV_ZONED_LOOP =20 If unsure, say N. =20 +config VIRTIO_BLK_INLINE_ENCRYPTION + tristate "Virtio block inline encryption support" + depends on VIRTIO_BLK && BLK_INLINE_ENCRYPTION + help + Say 'Y or M' here will allow the virtio block driver to route crypto + requests to a different operating system in a virtualized + environment. This is useful to encrypt the data stored in the storage + by using storage inline crypto engine. The control queue feature bit + must be negotiated to enable this functionality. + + If unsure, say N. + endif # BLK_DEV diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 252ec9366153..3fd318e5e1f1 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -17,6 +17,7 @@ #include #include #include +#include =20 #define PART_BITS 4 #define VQ_NAME_LEN 16 @@ -94,13 +95,24 @@ struct virtio_blk { /* For zoned device */ unsigned int zone_sectors; =20 + /* For inline encryption support */ + struct blk_crypto_profile profile; + bool crypto_profile_initialized; + /* Control virtqueue state. */ struct virtio_blk_ctrl_vq ctrl_vq; }; =20 struct virtblk_req { /* Out header */ - struct virtio_blk_outhdr out_hdr; + union { + struct virtio_blk_outhdr base; + struct { + struct virtio_blk_outhdr base; + /* Crypto message (if VIRTIO_BLK_F_INLINE_ENCRYPTION) */ + struct virtio_blk_crypto_msg msg; + } crypto_append; + } out_hdr; =20 /* In header */ union { @@ -124,8 +136,23 @@ struct virtblk_req { }; =20 struct virtblk_ctrl_request { + /* Type byte, always its own out-sg for every command. */ __virtio32 type; + /* Out request, sent as a second, separate out-sg if any. */ + union { + struct virtio_blk_crypto_key_desc key_desc; + struct virtio_blk_crypto_key_blob blob; + } out_req; + + /* In response */ + union { + struct virtio_blk_crypto_key_blob blob; + struct virtio_blk_crypto_sw_secret secret; + struct virtio_blk_crypto_modes modes; + } in_resp; + /* Status byte, always its own in-sg for every command. */ u8 status; + struct completion compl; }; =20 @@ -159,12 +186,17 @@ static int virtblk_add_req(struct virtqueue *vq, stru= ct virtblk_req *vbr) { struct scatterlist out_hdr, in_hdr, *sgs[3]; unsigned int num_out =3D 0, num_in =3D 0; + size_t out_hdr_len =3D sizeof(vbr->out_hdr.base); =20 - sg_init_one(&out_hdr, &vbr->out_hdr, sizeof(vbr->out_hdr)); + if (vbr->out_hdr.base.type =3D=3D cpu_to_virtio32(vq->vdev, VIRTIO_BLK_T_= CRYPTO_IN) || + vbr->out_hdr.base.type =3D=3D cpu_to_virtio32(vq->vdev, VIRTIO_BLK_T_= CRYPTO_OUT)) + out_hdr_len =3D sizeof(vbr->out_hdr.crypto_append); + + sg_init_one(&out_hdr, &vbr->out_hdr, out_hdr_len); sgs[num_out++] =3D &out_hdr; =20 if (vbr->sg_table.nents) { - if (vbr->out_hdr.type & cpu_to_virtio32(vq->vdev, VIRTIO_BLK_T_OUT)) + if (vbr->out_hdr.base.type & cpu_to_virtio32(vq->vdev, VIRTIO_BLK_T_OUT)) sgs[num_out++] =3D vbr->sg_table.sgl; else sgs[num_out + num_in++] =3D vbr->sg_table.sgl; @@ -254,6 +286,22 @@ static void virtblk_cleanup_cmd(struct request *req) kfree(bvec_virt(&req->special_vec)); } =20 +#if IS_ENABLED(CONFIG_VIRTIO_BLK_INLINE_ENCRYPTION) +static bool is_crypto_request(struct request *req) +{ + struct request_queue *q =3D req->q; + + return q->crypto_profile && + req->crypt_ctx && + req->crypt_keyslot; +} +#else +static inline bool is_crypto_request(struct request *req) +{ + return false; +} +#endif + static blk_status_t virtblk_setup_cmd(struct virtio_device *vdev, struct request *req, struct virtblk_req *vbr) @@ -262,20 +310,27 @@ static blk_status_t virtblk_setup_cmd(struct virtio_d= evice *vdev, bool unmap =3D false; u32 type; u64 sector =3D 0; + int i; =20 if (!IS_ENABLED(CONFIG_BLK_DEV_ZONED) && op_is_zone_mgmt(req_op(req))) return BLK_STS_NOTSUPP; =20 /* Set fields for all request types */ - vbr->out_hdr.ioprio =3D cpu_to_virtio32(vdev, req_get_ioprio(req)); + vbr->out_hdr.base.ioprio =3D cpu_to_virtio32(vdev, req_get_ioprio(req)); =20 switch (req_op(req)) { case REQ_OP_READ: - type =3D VIRTIO_BLK_T_IN; + if (is_crypto_request(req)) + type =3D VIRTIO_BLK_T_CRYPTO_IN; + else + type =3D VIRTIO_BLK_T_IN; sector =3D blk_rq_pos(req); break; case REQ_OP_WRITE: - type =3D VIRTIO_BLK_T_OUT; + if (is_crypto_request(req)) + type =3D VIRTIO_BLK_T_CRYPTO_OUT; + else + type =3D VIRTIO_BLK_T_OUT; sector =3D blk_rq_pos(req); break; case REQ_OP_FLUSH: @@ -328,8 +383,8 @@ static blk_status_t virtblk_setup_cmd(struct virtio_dev= ice *vdev, =20 /* Set fields for non-REQ_OP_DRV_IN request types */ vbr->in_hdr_len =3D in_hdr_len; - vbr->out_hdr.type =3D cpu_to_virtio32(vdev, type); - vbr->out_hdr.sector =3D cpu_to_virtio64(vdev, sector); + vbr->out_hdr.base.type =3D cpu_to_virtio32(vdev, type); + vbr->out_hdr.base.sector =3D cpu_to_virtio64(vdev, sector); =20 if (type =3D=3D VIRTIO_BLK_T_DISCARD || type =3D=3D VIRTIO_BLK_T_WRITE_ZE= ROES || type =3D=3D VIRTIO_BLK_T_SECURE_ERASE) { @@ -337,6 +392,18 @@ static blk_status_t virtblk_setup_cmd(struct virtio_de= vice *vdev, return BLK_STS_RESOURCE; } =20 + if (type =3D=3D VIRTIO_BLK_T_CRYPTO_IN || type =3D=3D VIRTIO_BLK_T_CRYPTO= _OUT) { + memset(&vbr->out_hdr.crypto_append.msg, 0, + sizeof(vbr->out_hdr.crypto_append.msg)); + vbr->out_hdr.crypto_append.msg.slot =3D + cpu_to_virtio32(vdev, + blk_crypto_keyslot_index(req->crypt_keyslot)); + for (i =3D 0; i < ARRAY_SIZE(vbr->out_hdr.crypto_append.msg.dun); i++) { + vbr->out_hdr.crypto_append.msg.dun[i] =3D + cpu_to_virtio64(vdev, req->crypt_ctx->bc_dun[i]); + } + } + return 0; } =20 @@ -587,8 +654,8 @@ static int virtblk_submit_zone_report(struct virtio_blk= *vblk, =20 vbr =3D blk_mq_rq_to_pdu(req); vbr->in_hdr_len =3D sizeof(vbr->in_hdr.status); - vbr->out_hdr.type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_ZONE_REPOR= T); - vbr->out_hdr.sector =3D cpu_to_virtio64(vblk->vdev, sector); + vbr->out_hdr.base.type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_ZONE_= REPORT); + vbr->out_hdr.base.sector =3D cpu_to_virtio64(vblk->vdev, sector); =20 err =3D blk_rq_map_kern(req, report_buf, report_len, GFP_KERNEL); if (err) @@ -836,8 +903,8 @@ static int virtblk_get_id(struct gendisk *disk, char *i= d_str) =20 vbr =3D blk_mq_rq_to_pdu(req); vbr->in_hdr_len =3D sizeof(vbr->in_hdr.status); - vbr->out_hdr.type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_GET_ID); - vbr->out_hdr.sector =3D 0; + vbr->out_hdr.base.type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_GET_I= D); + vbr->out_hdr.base.sector =3D 0; =20 err =3D blk_rq_map_kern(req, id_str, VIRTIO_BLK_ID_BYTES, GFP_KERNEL); if (err) @@ -1001,11 +1068,478 @@ static int virtblk_ctrl_vq_request(struct virtio_b= lk *vblk, return 0; } =20 +#if IS_ENABLED(CONFIG_VIRTIO_BLK_INLINE_ENCRYPTION) +// Maps VIRTIO_BLK_CRYPTO_MODE_* values to the kernel's internal enum. +static const enum blk_crypto_mode_num + virtio_blk_crypto_mode_map[VIRTIO_BLK_CRYPTO_MODE_MAX + 1] =3D { + [VIRTIO_BLK_CRYPTO_MODE_INVALID] =3D BLK_ENCRYPTION_MODE_INVALID, + [VIRTIO_BLK_CRYPTO_MODE_AES_256_XTS] =3D BLK_ENCRYPTION_MODE_AES_256_XTS, +}; + +static int virtblk_get_crypto_modes(struct virtio_blk *vblk, + unsigned int *crypto_modes_supported) +{ + unsigned int nr_modes =3D VIRTIO_BLK_CRYPTO_MODE_MAX + 1; + struct scatterlist type_sg, resp_sg, status_sg, *sgs[3]; + struct virtblk_ctrl_request *creq; + unsigned int i; + int err; + + creq =3D kzalloc_obj(*creq, GFP_KERNEL); + if (!creq) + return -ENOMEM; + + creq->type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_GET_CRYPTO_MODES); + + sg_init_one(&type_sg, &creq->type, sizeof(creq->type)); + sg_init_one(&resp_sg, &creq->in_resp.modes, sizeof(creq->in_resp.modes)); + sg_init_one(&status_sg, &creq->status, sizeof(creq->status)); + sgs[0] =3D &type_sg; + sgs[1] =3D &resp_sg; + sgs[2] =3D &status_sg; + + err =3D virtblk_ctrl_vq_request(vblk, creq, sgs, 1, 2); + if (err) + goto out_free; + + err =3D blk_status_to_errno(virtblk_result(creq->status)); + if (err) + goto out_free; + + for (i =3D 1; i < nr_modes; i++) { + u32 mode_mask =3D virtio32_to_cpu(vblk->vdev, + creq->in_resp.modes.modes[i]); + enum blk_crypto_mode_num mode =3D virtio_blk_crypto_mode_map[i]; + + if (!mode_mask) + continue; + if (!mode) { + dev_warn(&vblk->vdev->dev, + "ignoring unknown crypto mode %u\n", i); + continue; + } + crypto_modes_supported[mode] =3D mode_mask; + } + +out_free: + kfree(creq); + return err; +} + +static int set_virtblk_crypto_key_desc(struct virtio_device *vdev, + struct virtblk_ctrl_request *creq, + const struct blk_crypto_key *key, + unsigned int slot) +{ + struct virtio_blk_crypto_key_desc *desc =3D &creq->out_req.key_desc; + enum blk_crypto_key_type key_type =3D key->crypto_cfg.key_type; + + if (sizeof(desc->bytes) < key->size) + return -EOVERFLOW; + + memset(desc, 0, sizeof(*desc)); + desc->slot =3D cpu_to_virtio32(vdev, slot); + memcpy(desc->bytes, key->bytes, key->size); + desc->key_size =3D cpu_to_virtio32(vdev, key->size); + desc->crypto_mode =3D cpu_to_virtio32(vdev, key->crypto_cfg.crypto_mode); + switch (key->crypto_cfg.key_type) { + case BLK_CRYPTO_KEY_TYPE_RAW: + desc->key_type =3D cpu_to_virtio32(vdev, + VIRTIO_BLK_CRYPTO_KEY_TYPE_RAW); + break; + case BLK_CRYPTO_KEY_TYPE_HW_WRAPPED: + desc->key_type =3D cpu_to_virtio32(vdev, + VIRTIO_BLK_CRYPTO_KEY_TYPE_HW_WRAPPED); + break; + default: + return -EOPNOTSUPP; + } + desc->data_unit_size_bits =3D cpu_to_virtio32(vdev, key->data_unit_size_b= its); + desc->dun_bytes =3D cpu_to_virtio32(vdev, key->crypto_cfg.dun_bytes); + + return 0; +} + +static inline struct virtio_blk *virtblk_from_profile(struct blk_crypto_pr= ofile *profile) +{ + return container_of(profile, struct virtio_blk, profile); +} + +static int virtblk_crypto_keyslot_program(struct blk_crypto_profile *profi= le, + const struct blk_crypto_key *key, + unsigned int slot) +{ + struct virtio_blk *vblk =3D virtblk_from_profile(profile); + struct scatterlist type_sg, out_req_sg, status_sg, *sgs[3]; + struct virtblk_ctrl_request *creq; + int err; + + creq =3D kzalloc_obj(*creq, GFP_KERNEL); + if (!creq) + return -ENOMEM; + + creq->type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_CRYPTO_KEYSLOT_PR= OGRAM); + + err =3D set_virtblk_crypto_key_desc(vblk->vdev, creq, key, slot); + if (err) + goto out_free; + + sg_init_one(&type_sg, &creq->type, sizeof(creq->type)); + sg_init_one(&out_req_sg, &creq->out_req.key_desc, sizeof(creq->out_req.ke= y_desc)); + sg_init_one(&status_sg, &creq->status, sizeof(creq->status)); + sgs[0] =3D &type_sg; + sgs[1] =3D &out_req_sg; + sgs[2] =3D &status_sg; + + err =3D virtblk_ctrl_vq_request(vblk, creq, sgs, 2, 1); + if (err) + goto out_free; + + err =3D blk_status_to_errno(virtblk_result(creq->status)); +out_free: + kfree(creq); + return err; +} + +static int virtblk_crypto_keyslot_evict(struct blk_crypto_profile *profile, + const struct blk_crypto_key *key, + unsigned int slot) +{ + struct virtio_blk *vblk =3D virtblk_from_profile(profile); + struct scatterlist type_sg, out_req_sg, status_sg, *sgs[3]; + struct virtblk_ctrl_request *creq; + int err; + + creq =3D kzalloc_obj(*creq, GFP_KERNEL); + if (!creq) + return -ENOMEM; + + creq->type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_CRYPTO_KEYSLOT_EV= ICT); + + err =3D set_virtblk_crypto_key_desc(vblk->vdev, creq, key, slot); + if (err) + goto out_free; + + sg_init_one(&type_sg, &creq->type, sizeof(creq->type)); + sg_init_one(&out_req_sg, &creq->out_req.key_desc, sizeof(creq->out_req.ke= y_desc)); + sg_init_one(&status_sg, &creq->status, sizeof(creq->status)); + sgs[0] =3D &type_sg; + sgs[1] =3D &out_req_sg; + sgs[2] =3D &status_sg; + + err =3D virtblk_ctrl_vq_request(vblk, creq, sgs, 2, 1); + if (err) + goto out_free; + + err =3D blk_status_to_errno(virtblk_result(creq->status)); +out_free: + kfree(creq); + return err; +} + +static int virtblk_crypto_derive_sw_secret(struct blk_crypto_profile *prof= ile, + const u8 *eph_key, size_t eph_key_size, + u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]) +{ + struct virtio_blk *vblk =3D virtblk_from_profile(profile); + struct scatterlist type_sg, out_req_sg, resp_sg, status_sg, *sgs[4]; + struct virtblk_ctrl_request *creq; + int err; + + if (eph_key_size > VIRTIO_BLK_CRYPTO_MAX_KEY_SIZE + || sizeof(creq->in_resp.secret.secret) < BLK_CRYPTO_SW_SECRET_SIZE) + return -EOVERFLOW; + + creq =3D kzalloc_obj(*creq, GFP_KERNEL); + if (!creq) + return -ENOMEM; + + creq->type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_CRYPTO_DERIVE_SW_= SECRET); + memcpy(creq->out_req.blob.key, eph_key, eph_key_size); + creq->out_req.blob.key_size =3D cpu_to_virtio32(vblk->vdev, eph_key_size); + + sg_init_one(&type_sg, &creq->type, sizeof(creq->type)); + sg_init_one(&out_req_sg, &creq->out_req.blob, sizeof(creq->out_req.blob)); + sg_init_one(&resp_sg, &creq->in_resp.secret, sizeof(creq->in_resp.secret)= ); + sg_init_one(&status_sg, &creq->status, sizeof(creq->status)); + sgs[0] =3D &type_sg; + sgs[1] =3D &out_req_sg; + sgs[2] =3D &resp_sg; + sgs[3] =3D &status_sg; + + err =3D virtblk_ctrl_vq_request(vblk, creq, sgs, 2, 2); + if (err) + goto out_free; + + err =3D blk_status_to_errno(virtblk_result(creq->status)); + if (err) + goto out_free; + + memcpy(sw_secret, creq->in_resp.secret.secret, BLK_CRYPTO_SW_SECRET_SIZE); +out_free: + kfree(creq); + return err; +} + +static int virtblk_crypto_generate_key(struct blk_crypto_profile *profile, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + struct virtio_blk *vblk =3D virtblk_from_profile(profile); + struct scatterlist type_sg, resp_sg, status_sg, *sgs[3]; + struct virtblk_ctrl_request *creq; + unsigned int key_size; + int err; + + creq =3D kzalloc_obj(*creq, GFP_KERNEL); + if (!creq) + return -ENOMEM; + + creq->type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_CRYPTO_GENERATE_K= EY); + + sg_init_one(&type_sg, &creq->type, sizeof(creq->type)); + sg_init_one(&resp_sg, &creq->in_resp.blob, sizeof(creq->in_resp.blob)); + sg_init_one(&status_sg, &creq->status, sizeof(creq->status)); + sgs[0] =3D &type_sg; + sgs[1] =3D &resp_sg; + sgs[2] =3D &status_sg; + + err =3D virtblk_ctrl_vq_request(vblk, creq, sgs, 1, 2); + if (err) + goto out_free; + + err =3D blk_status_to_errno(virtblk_result(creq->status)); + if (err) + goto out_free; + + key_size =3D virtio32_to_cpu(vblk->vdev, creq->in_resp.blob.key_size); + if (!key_size || + key_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE) { + dev_err(&vblk->vdev->dev, + "backend returned oversized generated key: %u\n", key_size); + err =3D -EOVERFLOW; + goto out_free; + } + memcpy(lt_key, creq->in_resp.blob.key, key_size); + err =3D key_size; +out_free: + kfree(creq); + return err; +} + +static int virtblk_crypto_prepare_key(struct blk_crypto_profile *profile, + const u8 *lt_key, size_t lt_key_size, + u8 eph_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + struct virtio_blk *vblk =3D virtblk_from_profile(profile); + struct scatterlist type_sg, out_req_sg, resp_sg, status_sg, *sgs[4]; + struct virtblk_ctrl_request *creq; + unsigned int key_size; + int err; + + if (lt_key_size > VIRTIO_BLK_CRYPTO_MAX_KEY_SIZE) + return -EOVERFLOW; + + creq =3D kzalloc_obj(*creq, GFP_KERNEL); + if (!creq) + return -ENOMEM; + + creq->type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_CRYPTO_PREPARE_KE= Y); + memcpy(creq->out_req.blob.key, lt_key, lt_key_size); + creq->out_req.blob.key_size =3D cpu_to_virtio32(vblk->vdev, lt_key_size); + + sg_init_one(&type_sg, &creq->type, sizeof(creq->type)); + sg_init_one(&out_req_sg, &creq->out_req.blob, sizeof(creq->out_req.blob)); + sg_init_one(&resp_sg, &creq->in_resp.blob, sizeof(creq->in_resp.blob)); + sg_init_one(&status_sg, &creq->status, sizeof(creq->status)); + sgs[0] =3D &type_sg; + sgs[1] =3D &out_req_sg; + sgs[2] =3D &resp_sg; + sgs[3] =3D &status_sg; + + err =3D virtblk_ctrl_vq_request(vblk, creq, sgs, 2, 2); + if (err) + goto out_free; + + err =3D blk_status_to_errno(virtblk_result(creq->status)); + if (err) + goto out_free; + + key_size =3D virtio32_to_cpu(vblk->vdev, creq->in_resp.blob.key_size); + if (!key_size || + key_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE) { + dev_err(&vblk->vdev->dev, + "backend returned oversized prepared key: %u\n", key_size); + err =3D -EOVERFLOW; + goto out_free; + } + memcpy(eph_key, creq->in_resp.blob.key, key_size); + err =3D key_size; +out_free: + kfree(creq); + return err; +} + +static int virtblk_crypto_import_key(struct blk_crypto_profile *profile, + const u8 *raw_key, size_t raw_key_size, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + struct virtio_blk *vblk =3D virtblk_from_profile(profile); + struct scatterlist type_sg, out_req_sg, resp_sg, status_sg, *sgs[4]; + struct virtblk_ctrl_request *creq; + unsigned int key_size; + int err; + + if (raw_key_size > VIRTIO_BLK_CRYPTO_MAX_KEY_SIZE) + return -EOVERFLOW; + + creq =3D kzalloc_obj(*creq, GFP_KERNEL); + if (!creq) + return -ENOMEM; + + creq->type =3D cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_CRYPTO_IMPORT_KEY= ); + memcpy(creq->out_req.blob.key, raw_key, raw_key_size); + creq->out_req.blob.key_size =3D cpu_to_virtio32(vblk->vdev, raw_key_size); + + sg_init_one(&type_sg, &creq->type, sizeof(creq->type)); + sg_init_one(&out_req_sg, &creq->out_req.blob, sizeof(creq->out_req.blob)); + sg_init_one(&resp_sg, &creq->in_resp.blob, sizeof(creq->in_resp.blob)); + sg_init_one(&status_sg, &creq->status, sizeof(creq->status)); + sgs[0] =3D &type_sg; + sgs[1] =3D &out_req_sg; + sgs[2] =3D &resp_sg; + sgs[3] =3D &status_sg; + + err =3D virtblk_ctrl_vq_request(vblk, creq, sgs, 2, 2); + if (err) + goto out_free; + + err =3D blk_status_to_errno(virtblk_result(creq->status)); + if (err) + goto out_free; + + key_size =3D virtio32_to_cpu(vblk->vdev, creq->in_resp.blob.key_size); + if (!key_size || + key_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE) { + dev_err(&vblk->vdev->dev, + "backend returned oversized imported key: %u\n", key_size); + err =3D -EOVERFLOW; + goto out_free; + } + memcpy(lt_key, creq->in_resp.blob.key, key_size); + err =3D key_size; +out_free: + kfree(creq); + return err; +} + +static const struct blk_crypto_ll_ops virtblk_crypto_ops =3D { + .keyslot_program =3D virtblk_crypto_keyslot_program, + .keyslot_evict =3D virtblk_crypto_keyslot_evict, + .derive_sw_secret =3D virtblk_crypto_derive_sw_secret, + .generate_key =3D virtblk_crypto_generate_key, + .prepare_key =3D virtblk_crypto_prepare_key, + .import_key =3D virtblk_crypto_import_key, +}; + +static int virtblk_init_crypto(struct virtio_blk *vblk) +{ + struct virtio_device *vdev =3D vblk->vdev; + unsigned int crypto_modes_supported[BLK_ENCRYPTION_MODE_MAX] =3D { 0 }; + unsigned int key_type_supported =3D 0; + u16 max_slots =3D 0; + /* virtio_cread() requires the variable size to match the config field ex= actly */ + u8 max_dun_bytes =3D 0, key_types =3D 0; + int err; + + virtio_cread(vdev, struct virtio_blk_config, + enc_characteristics.max_slots, &max_slots); + virtio_cread(vdev, struct virtio_blk_config, + enc_characteristics.max_dun_bytes, &max_dun_bytes); + virtio_cread(vdev, struct virtio_blk_config, + enc_characteristics.key_types, &key_types); + + dev_info_once(&vdev->dev, + "max_slots =3D %u, max_dun_bytes =3D %u, key_types =3D 0x%x\n", + max_slots, max_dun_bytes, key_types); + + if (!max_slots) + return -EINVAL; + + /* + * struct virtio_blk_crypto_msg.dun is a fixed array of four __virtio64 + * values (32 bytes total), matching the size of + * blk_crypto_ctx::bc_dun[4]. Refuse to advertise more than that as + * supported, or blk-crypto could negotiate a larger dun_bytes with the + * filesystem and have the high-order bytes of req->crypt_ctx->bc_dun + * silently dropped in virtblk_setup_cmd(). + */ + if (max_dun_bytes > sizeof_field(struct virtio_blk_crypto_msg, dun)) + return -EINVAL; + + if (key_types & VIRTIO_BLK_CRYPTO_KEY_TYPE_RAW) + key_type_supported |=3D BLK_CRYPTO_KEY_TYPE_RAW; + if (key_types & VIRTIO_BLK_CRYPTO_KEY_TYPE_HW_WRAPPED) + key_type_supported |=3D BLK_CRYPTO_KEY_TYPE_HW_WRAPPED; + if (!key_type_supported) + return -EINVAL; + + err =3D virtblk_get_crypto_modes(vblk, crypto_modes_supported); + if (err) { + dev_err(&vdev->dev, "get crypto modes failed: %d\n", err); + return err; + } + + /* + * Use the plain (non-devm) initializer: vblk->profile is embedded in + * struct virtio_blk, whose lifetime is tied to the gendisk, not to + * &vdev->dev. Tying destruction to the vdev via devm would run the + * destroy callback after virtblk_remove() has already freed vblk. + * virtblk_free_disk() calls blk_crypto_profile_destroy() explicitly + * instead, guarded by crypto_profile_initialized below. + */ + err =3D blk_crypto_profile_init(&vblk->profile, max_slots); + if (err) { + dev_err(&vdev->dev, "crypto profile initialization failed: %d\n", err); + return err; + } + + vblk->profile.ll_ops =3D virtblk_crypto_ops; + vblk->profile.max_dun_bytes_supported =3D max_dun_bytes; + vblk->profile.key_types_supported =3D key_type_supported; + vblk->profile.dev =3D &vdev->dev; + memcpy(vblk->profile.modes_supported, crypto_modes_supported, + BLK_ENCRYPTION_MODE_MAX * sizeof(unsigned int)); + + vblk->crypto_profile_initialized =3D true; + + dev_info(&vdev->dev, "inline crypto profile initialized\n"); + + return 0; +} + +static void virtblk_destroy_crypto(struct virtio_blk *vblk) +{ + if (vblk->crypto_profile_initialized) + blk_crypto_profile_destroy(&vblk->profile); +} +#else + +static inline int virtblk_init_crypto(struct virtio_blk *vblk) +{ + return -EOPNOTSUPP; +} + +static inline void virtblk_destroy_crypto(struct virtio_blk *vblk) +{ +} +#endif /* CONFIG_VIRTIO_BLK_INLINE_ENCRYPTION */ + static void virtblk_free_disk(struct gendisk *disk) { struct virtio_blk *vblk =3D disk->private_data; =20 ida_free(&vd_index_ida, vblk->index); + virtblk_destroy_crypto(vblk); mutex_destroy(&vblk->ctrl_vq.mutex); mutex_destroy(&vblk->vdev_mutex); kfree(vblk); @@ -1698,6 +2232,19 @@ static int virtblk_probe(struct virtio_device *vdev) err =3D blk_revalidate_disk_zones(vblk->disk); if (err) goto out_cleanup_disk; + } else if (IS_ENABLED(CONFIG_VIRTIO_BLK_INLINE_ENCRYPTION) && + virtio_has_feature(vdev, VIRTIO_BLK_F_INLINE_ENCRYPTION) && + virtio_has_feature(vdev, VIRTIO_BLK_F_CTRL_VQ)) { + err =3D virtblk_init_crypto(vblk); + if (!err) { + if (!blk_crypto_register(&vblk->profile, vblk->disk->queue)) + dev_warn(&vdev->dev, + "failed to register inline crypto profile\n"); + } else { + dev_warn(&vdev->dev, + "inline crypto init failed: %d, continuing without inline crypto supp= ort\n", + err); + } } =20 err =3D device_add_disk(&vdev->dev, vblk->disk, virtblk_attr_groups); @@ -1798,6 +2345,9 @@ static int virtblk_restore_priv(struct virtio_device = *vdev) virtio_device_ready(vdev); blk_mq_unquiesce_queue(vblk->disk->queue); =20 + if (vblk->profile.slots) + blk_crypto_reprogram_all_keys(&vblk->profile); + return 0; } =20 @@ -1842,7 +2392,7 @@ static unsigned int features[] =3D { VIRTIO_BLK_F_FLUSH, VIRTIO_BLK_F_TOPOLOGY, VIRTIO_BLK_F_CONFIG_WCE, VIRTIO_BLK_F_MQ, VIRTIO_BLK_F_DISCARD, VIRTIO_BLK_F_WRITE_ZEROES, VIRTIO_BLK_F_SECURE_ERASE, VIRTIO_BLK_F_ZONED, - VIRTIO_BLK_F_CTRL_VQ, + VIRTIO_BLK_F_CTRL_VQ, VIRTIO_BLK_F_INLINE_ENCRYPTION, }; =20 static struct virtio_driver virtio_blk =3D { diff --git a/include/uapi/linux/virtio_blk.h b/include/uapi/linux/virtio_bl= k.h index 0a16972a1535..952534e2b489 100644 --- a/include/uapi/linux/virtio_blk.h +++ b/include/uapi/linux/virtio_blk.h @@ -43,6 +43,7 @@ #define VIRTIO_BLK_F_SECURE_ERASE 16 /* Secure Erase is supported */ #define VIRTIO_BLK_F_ZONED 17 /* Zoned block device */ #define VIRTIO_BLK_F_CTRL_VQ 22 /* Control queue */ +#define VIRTIO_BLK_F_INLINE_ENCRYPTION 23 /* Inline encryption */ =20 /* Legacy feature bits */ #ifndef VIRTIO_BLK_NO_LEGACY @@ -58,6 +59,10 @@ =20 #define VIRTIO_BLK_ID_BYTES 20 /* ID string length */ =20 +/* Key type bitmask for VIRTIO_BLK_F_INLINE_ENCRYPTION */ +#define VIRTIO_BLK_CRYPTO_KEY_TYPE_RAW (1 << 0) +#define VIRTIO_BLK_CRYPTO_KEY_TYPE_HW_WRAPPED (1 << 1) + struct virtio_blk_config { /* The capacity (in 512-byte sectors). */ __virtio64 capacity; @@ -149,6 +154,15 @@ struct virtio_blk_config { __u8 model; __u8 unused2[3]; } zoned; + + /* Inline Encryption device characteristics (if VIRTIO_BLK_F_INLINE_ENCRY= PTION) */ + struct virtio_blk_enc_characteristics { + __virtio16 max_slots; + __u8 max_dun_bytes; + /* Bitmask of supported key types: VIRTIO_BLK_CRYPTO_KEY_TYPE_* */ + __u8 key_types; + __virtio32 unused3; + } enc_characteristics; } __attribute__((packed)); =20 /* @@ -207,6 +221,33 @@ struct virtio_blk_config { /* Reset All zones command */ #define VIRTIO_BLK_T_ZONE_RESET_ALL 26 =20 +/* Inline-encrypted write: crypto_msg set in outhdr */ +#define VIRTIO_BLK_T_CRYPTO_OUT 27 + +/* Inline-encrypted read: crypto_msg set in outhdr */ +#define VIRTIO_BLK_T_CRYPTO_IN 28 + +/* Get inline crypto modes */ +#define VIRTIO_BLK_T_GET_CRYPTO_MODES 29 + +/* Program a key into the keyslot */ +#define VIRTIO_BLK_T_CRYPTO_KEYSLOT_PROGRAM 30 + +/* Evict a key */ +#define VIRTIO_BLK_T_CRYPTO_KEYSLOT_EVICT 31 + +/* Derive the software secret from a hardware-wrapped key */ +#define VIRTIO_BLK_T_CRYPTO_DERIVE_SW_SECRET 32 + +/* Generate a new hardware-wrapped key */ +#define VIRTIO_BLK_T_CRYPTO_GENERATE_KEY 33 + +/* Import a raw key as a hardware-wrapped key */ +#define VIRTIO_BLK_T_CRYPTO_IMPORT_KEY 34 + +/* Convert a long-term wrapped key to its ephemerally-wrapped form */ +#define VIRTIO_BLK_T_CRYPTO_PREPARE_KEY 35 + #ifndef VIRTIO_BLK_NO_LEGACY /* Barrier before this op. */ #define VIRTIO_BLK_T_BARRIER 0x80000000 @@ -226,6 +267,80 @@ struct virtio_blk_outhdr { __virtio64 sector; }; =20 +/* + * Crypto message descriptor, appended to the outhdr of a + * VIRTIO_BLK_T_CRYPTO_OUT or VIRTIO_BLK_T_CRYPTO_IN request. + */ +struct virtio_blk_crypto_msg { + /* virtual key slot index */ + __virtio32 slot; + __u8 unused[4]; + /* data unit number (DUN / IV) for this request */ + __virtio64 dun[4]; +}; + +/* + * Inline crypto key descriptor. Request part for + * VIRTIO_BLK_T_CRYPTO_KEYSLOT_PROGRAM/KEYSLOT_EVICT. + */ +/* Must be >=3D BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE in include/linux/blk-cr= ypto.h */ +#define VIRTIO_BLK_CRYPTO_MAX_KEY_SIZE 128 + +struct virtio_blk_crypto_key_desc { + __virtio32 slot; + __u8 bytes[VIRTIO_BLK_CRYPTO_MAX_KEY_SIZE]; + __virtio32 key_size; + __virtio32 crypto_mode; + __virtio32 key_type; + __virtio32 data_unit_size_bits; + __virtio32 dun_bytes; +}; + +/* + * A raw or hardware-wrapped key blob, used as the request and/or reply pa= rt + * of the VIRTIO_BLK_T_CRYPTO_GENERATE_KEY/IMPORT_KEY/PREPARE_KEY/ + * DERIVE_SW_SECRET commands. + */ +struct virtio_blk_crypto_key_blob { + __virtio32 key_size; + __u8 key[VIRTIO_BLK_CRYPTO_MAX_KEY_SIZE]; +}; + +/* Must match BLK_CRYPTO_SW_SECRET_SIZE in include/linux/blk-crypto.h */ +#define VIRTIO_BLK_CRYPTO_SW_SECRET_SIZE 32 + +/* Reply to a VIRTIO_BLK_T_CRYPTO_DERIVE_SW_SECRET request. */ +struct virtio_blk_crypto_sw_secret { + __u8 secret[VIRTIO_BLK_CRYPTO_SW_SECRET_SIZE]; +}; + +/* + * Crypto mode numbers used in VIRTIO_BLK_T_GET_CRYPTO_MODES replies and in + * indexing struct virtio_blk_crypto_modes.modes[] below. These numbers are + * assigned by the virtio spec and are stable: a number is never reused for + * a different crypto mode, and additional crypto modes are assigned new, + * higher numbers. + */ +enum { + VIRTIO_BLK_CRYPTO_MODE_INVALID, + VIRTIO_BLK_CRYPTO_MODE_AES_256_XTS, + __VIRTIO_BLK_CRYPTO_MODE_MAX, /* sentinel: always one past the last real = mode */ +}; + +/* Highest crypto mode number defined by this version of the header. */ +#define VIRTIO_BLK_CRYPTO_MODE_MAX (__VIRTIO_BLK_CRYPTO_MODE_MAX - 1) + +/* Reply to a VIRTIO_BLK_T_GET_CRYPTO_MODES request. */ +struct virtio_blk_crypto_modes { + /* + * modes[N], for crypto mode number N <=3D VIRTIO_BLK_CRYPTO_MODE_MAX, is + * a bitmask of the data unit sizes with which crypto mode N can be + * used: bit i is set if a data unit size of (1 << i) bytes is + * supported. modes[0] is reserved and always 0. + */ + __virtio32 modes[__VIRTIO_BLK_CRYPTO_MODE_MAX]; +}; + /* * Supported zoned device models. */ --=20 2.34.1