From: linlzhan <linlin.zhang@oss.qualcomm.com>

Current virtio-blk does not provide a mechanism for a guest to
program hardware keys or submit encrypted I/O using pre-programmed
keyslots.  It drops the crypto context when issuing a bio request
to the virtio-blk queue, preventing inline-encryption-based FBE
on virtio block devices.

This series enables File-Based Encryption in guest VMs on Qualcomm
GVM platforms where the ICE inline encryption hardware is shared
between the host and guests.  In this environment the guest kernel
has no access to the ICE hardware directly; it supplies a virtual
keyslot index and data unit number with each encrypted I/O request
via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
virtual slot to a physical ICE keyslot and submit the bio — without
transferring raw key material across the VM boundary.

             +----------------------------------------------------------------+   
             |                                                                |   
             |               LA GVM                                           |   
             |                                                                |   
             |                                                       |        |   
             |                                                       |        |   
             |                                                       |        |   
             |                                                +------v------+ |   
             |                               +----y-----------+   blk-mq    | |   
             |                               |                +-------+-----+ |   
             |                               |                        |       |   
             |                               |                        |       |   
             |                    +----------v-----------+            |       |   
             |                    |blk-crypto-profile    +------------v       |   
             |                    +----------------------+            |       |   
             |                                                        |       |   
             |                                                        |       |   
             |            Control path:                               |       |   
             |            generate/import/prepare key                 |       |   
             |            program/evict/derive_sw_secret key          |       |   
+----------+ |                                                        |       |   
|          | |+-------------+     +-----------------------+     +-----v-----+ |   
|Trust Zone| || crypto-virt <-----+ virtio-blk-crypto-ext <-----+virtio-blk | |   
|          | |+------+------+     +-----------------------+     +-----+-----+ |   
|          | |       |                                         Data path: I/O |   
+-------^--+ +-------+------------------------------------------------+-------+   
        |         SMC|call                                (virt_slot, DUN, DUSize)
        |            |                                    appended in virtblk_req 
    +---+------------v------------------------------------------------v----------+
    |  +--------------+                             +--------------------------+ |
    |  | SMC trap     |                  Hypervisor | MMIO trap/VIRQ injection | |
    +--+--------------+-----------------------------+--------------------------+ +

    +--------------------------------------------------------------------------------------------+
    |                                                                    PVM                     |
    |                                                                                            |
    | Crypto IO    +--------------+                                                              |
  +-+-------------->  QEMU/Crosvm +-------+                                                      |
  | |              +--------------+  IO data                                                     |
  | |                             virt_slot, DUN, DUSize                                         |
  | |                                     |                                                      |
  | |+----------------+       +--------- -v-----+                                                |
  | ||  blk-crypto    <-------+ blk-crypto-proxy+-------------+----------------------+           |
  | |+----------------+       +-----------+-----+             |                      |           |
  | |                             slot path based             |                      |           |
  | |                               bio_crypt_ctx             |                      |           |
  | |+-----------------------+       +--- v----+              |                      |           |
  | || blk-crypto-profile    <---x---+  blk-mq |      bcp_hypervisor_ops             |           |
  | |+-----------------------+       +----+----+              |                      |           |
  | |                                     |                   |             bcp_slot_virt_ops    |
  | |                         +-----------v-----+             |                      |           |
  | |                         |         SCSI    |             |                      |           |
  | |                         +-----------+-----+             |                      |           |
  | |                                     |                   |                      |           |
  | |                 crypto msg in UTRD  |                   |                      |           |
  | |+----------------------+        +----v-----+  +----------v-- ---------+ +-------v--- ------+|
  | ||    ufs crypto        <--------+ ufs-core |  |blk-crypto-hyp-backend | | keyslot-partition||
  | |+----------------------+        +-----+----+  +-----------------------+ +------------------+|
  | +--------------------------------------+-----------------------------------------------------+
  |                                        |                                                      
  |                                        |                                                      
+-----------------------------------+   +--v---+-----------+                                      
| +------------------------+        |   |  ICE |    UFS    |                                      
||MMIO trap/VIRQ injection |  HYP   |   +------+-----------+                                      
+ +------------------------+--------+                                                             

Patches 1-3 land in the guest kernel.  Patch 1 negotiates
VIRTIO_BLK_F_INLINE_ENCRYPTION and wires it into blk-crypto.
Patches 2-3 add the Qualcomm GVM-side crypto backend, which routes
key programming and software-secret derivation through SCM calls to
TrustZone.  These patches are sent for review; the virtio-blk inline
encryption protocol is also under review (see
https://lore.kernel.org/all/20260814142306.3934029-1-linlin.zhang@oss.qualcomm.com/).
These patches and the virtio-spec depend on each other.  They must be
kept consistent for upstream merging.

Patch 4 adds the dt-binding for the Qualcomm crypto-virt node used by
the guest-side backend.

Patch 5 introduces a "slot path" in blk-crypto that lets a bio carry
a pre-programmed physical ICE keyslot index in bc_slot rather than a
blk_crypto_key pointer (bc_key == NULL).  This is needed on the host
side where the hypervisor has already programmed the keyslot; the host
kernel has no access to the raw key.  Patch 6 extends
ufshcd_prepare_lrbp_crypto() to handle this path.  Patch 7 moves
bio_crypt_dun_increment() to the public header so it can be called
from drivers/block/.

Patch 8 adds /dev/blk-crypto-proxy, a misc character device for
userspace virtio-blk backends.  The interface is three ioctls:
BCP_BIND_CONTEXT binds a host block device fd and a hypervisor VM fd;
BCP_GET_CRYPTO_CAPS queries the device's inline-crypto capabilities
and the VM's ICE keyslot allocation; BCP_SUBMIT_IO_BY_VSLOT resolves
a guest virtual slot to a physical ICE keyslot and submits the
inline-encrypted bio synchronously.  The driver is
hypervisor-agnostic and storage-vendor-agnostic, using two pluggable
op-sets registered by platform drivers at runtime.

Patch 9 implements bcp_slot_virt_ops for Qualcomm platforms: it parses
a qcom,ice-keyslot-map device-tree node that maps each guest_id to a
contiguous physical keyslot range.  Patch 10 adds slot_offset to struct
blk_crypto_profile so that blk_crypto_keyslot_index() returns the
correct physical slot number when the host's ICE range does not start
at slot 0.  Patch 11 extends ufs-qcom to read the host's own slot
reservation from the same DT node and initialize the
blk_crypto_profile accordingly.

Patches 1-4 are technically ready for review.  However, since they
depend on the proposed virtio-blk inline encryption protocol, progress
on these patches is expected to follow consensus on the protocol design.
Feedback on the overall architecture is therefore particularly valuable,
as it will also help advance the associated virtio-spec work.

Patches 5-8 implement the core host-side infrastructure and are
believed ready for review.

Patches 9-11 do not depend on any hypervisor-specific code.  Of them,
patches 9 and 11 provide the Qualcomm platform implementation based
on a static device-tree keyslot mapping; this may be revised in a future
version to use a TZ SCM query interface.

The kernel-internal header declares bcp_hypervisor_ops, which
translates a hypervisor VM fd to an opaque guest_id.  No upstream
implementation is included in this series because the series was
validated on a downstream Qualcomm GVM platform using the Gunyah
hypervisor, which provides a stable per-VM identifier but is not
yet upstream.  KVM does not currently expose an externally-visible
per-VM identifier that a kernel module could use for this purpose.
Input from KVM maintainers on whether and how such an interface
could be added, or whether an alternative identity mechanism is
preferred, would be welcome.

Known limitations
-------
  - Only AES-256-XTS has been tested.
  - virtio_blk_crypto_msg.dun is a fixed __virtio64; the driver refuses
    to enable inline crypto if the device advertises max_dun_bytes > 8 to
    prevent silent IV truncation and reuse.
  - Inline encryption is mutually exclusive with VIRTIO_BLK_F_ZONED.
  - The qcom_ice_slots driver uses a
    global singleton and ignores the blk_crypto_profile argument to its
    callbacks, so multiple storage controllers sharing a single slot table
    are not yet supported.
  - BCP_SUBMIT_IO_BY_VSLOT submits each bio synchronously with
    submit_bio_wait(); concurrent in-flight bios from multiple threads
    sharing one fd are not supported.

Testing
-------
Compilation pass on Linux-next.
End-to-end FBE virtualization with wrapped key enabled was validated
on top of gunyah hypervisor.  wrapped_key_test is a local utility to
get wrapped key and ephemeral wrapped key via storage ioctl interfaces.
  - /data/wrapped_key_test /dev/block/userdata generate
  - /data/wrapped_key_test /dev/block/userdata prepare /data/lt_key.bin
  - /data/fscryptctl insert_wrapped_key < /data/eph_key.bin
  - /data/fscryptctl set_policy --identifier=20f553802e64e36b43469211266a5f1c /data/testing
  - echo "data" > /data/testing/file.txt
  - sync and reboot
  - /data/wrapped_key_test /dev/block/userdata prepare /data/lt_key.bin
  - /data/fscryptctl insert_wrapped_key < /data/eph_key_2.bin
  - /data/fscryptctl set_policy --identifier=d8ca51d6d2094b73b2dae5ee7e3a10b6 /data/testing
  - cat /data/testing/file.txt


linlzhan (11):
  virtio_blk: add inline encryption support
  soc: qcom: add crypto_virt backend for virtio-blk inline crypto
  soc: qcom: crypto_virt: add support for create, prepare and import
    keys
  dt-bindings: soc: qcom: add binding for qcom,crypto-virt
  blk-crypto: add slot-based inline encryption path
  scsi: ufs: core: add slot path to ufshcd_prepare_lrbp_crypto
  blk-crypto: move bio_crypt_dun_increment() to the public header
  block: add /dev/blk-crypto-proxy for host-side virtio-blk inline
    encryption
  soc: qcom: add ICE keyslot partitioning driver for guest VMs
  blk-crypto: add slot_offset to blk_crypto_profile
  scsi: ufs: ufs-qcom: support ICE keyslot partitioning for guest VMs

 .../bindings/soc/qcom/qcom,crypto-virt.yaml   |  39 +
 block/blk-crypto-internal.h                   |   5 +-
 block/blk-crypto-profile.c                    |   7 +-
 block/blk-crypto.c                            |  57 +-
 drivers/block/Kconfig                         |  28 +
 drivers/block/Makefile                        |   3 +
 drivers/block/blk-crypto-proxy.c              | 667 ++++++++++++++++++
 drivers/block/virtio_blk.c                    | 199 +++++-
 drivers/block/virtio_blk_crypto_ext.c         | 283 ++++++++
 drivers/soc/qcom/Kconfig                      |  30 +
 drivers/soc/qcom/Makefile                     |   2 +
 drivers/soc/qcom/crypto_virt.c                | 197 ++++++
 drivers/soc/qcom/qcom_ice_slots.c             | 232 ++++++
 drivers/ufs/core/ufshcd-crypto.h              |  14 +-
 drivers/ufs/host/ufs-qcom.c                   |  91 ++-
 include/linux/blk-crypto-profile.h            |   9 +
 include/linux/blk-crypto-proxy.h              | 100 +++
 include/linux/blk-crypto.h                    |  28 +
 include/linux/virtio_blk_crypto_ext.h         |  78 ++
 include/uapi/linux/blk-crypto-proxy.h         | 122 ++++
 include/uapi/linux/virtio_blk.h               |  62 ++
 21 files changed, 2224 insertions(+), 29 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/soc/qcom/qcom,crypto-virt.yaml
 create mode 100644 drivers/block/blk-crypto-proxy.c
 create mode 100644 drivers/block/virtio_blk_crypto_ext.c
 create mode 100644 drivers/soc/qcom/crypto_virt.c
 create mode 100644 drivers/soc/qcom/qcom_ice_slots.c
 create mode 100644 include/linux/blk-crypto-proxy.h
 create mode 100644 include/linux/virtio_blk_crypto_ext.h
 create mode 100644 include/uapi/linux/blk-crypto-proxy.h

-- 
2.34.1

On Thu, Aug 27, 2026 at 09:07:09AM -0700, Linlin Zhang wrote:
> From: linlzhan <linlin.zhang@oss.qualcomm.com>
> 
> Current virtio-blk does not provide a mechanism for a guest to
> program hardware keys or submit encrypted I/O using pre-programmed
> keyslots.  It drops the crypto context when issuing a bio request
> to the virtio-blk queue, preventing inline-encryption-based FBE
> on virtio block devices.
> 
> This series enables File-Based Encryption in guest VMs on Qualcomm
> GVM platforms where the ICE inline encryption hardware is shared
> between the host and guests.  In this environment the guest kernel
> has no access to the ICE hardware directly; it supplies a virtual
> keyslot index and data unit number with each encrypted I/O request
> via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
> virtual slot to a physical ICE keyslot and submit the bio — without
> transferring raw key material across the VM boundary.

This seems to be designed incorrectly by not making virtio-blk itself
support key programming and eviction.  That complicates things
significantly by then having to handle the key programming and eviction
out-of-band using Qualcomm-specific SCM calls.  It also means that
adding other implementations of this would be very difficult.

There are some claims that not transmitting keys across the VM boundary
is desirable.  But that doesn't seem meaningful, given that all the I/O
is transmitted across that boundary in plaintext anyway, and also it
seems that hardware-wrapped keys will be supported too.

Please make virtio-blk support the key programming, eviction, and
HW-wrapped key management operations that are needed for this to work.

- Eric

On 8/28/2026 2:42 AM, Eric Biggers wrote:
> On Thu, Aug 27, 2026 at 09:07:09AM -0700, Linlin Zhang wrote:
>> From: linlzhan <linlin.zhang@oss.qualcomm.com>
>>
>> Current virtio-blk does not provide a mechanism for a guest to
>> program hardware keys or submit encrypted I/O using pre-programmed
>> keyslots.  It drops the crypto context when issuing a bio request
>> to the virtio-blk queue, preventing inline-encryption-based FBE
>> on virtio block devices.
>>
>> This series enables File-Based Encryption in guest VMs on Qualcomm
>> GVM platforms where the ICE inline encryption hardware is shared
>> between the host and guests.  In this environment the guest kernel
>> has no access to the ICE hardware directly; it supplies a virtual
>> keyslot index and data unit number with each encrypted I/O request
>> via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
>> virtual slot to a physical ICE keyslot and submit the bio — without
>> transferring raw key material across the VM boundary.
> 
> This seems to be designed incorrectly by not making virtio-blk itself
> support key programming and eviction.  That complicates things
> significantly by then having to handle the key programming and eviction
> out-of-band using Qualcomm-specific SCM calls.  It also means that
> adding other implementations of this would be very difficult.
> 
> There are some claims that not transmitting keys across the VM boundary
> is desirable.  But that doesn't seem meaningful, given that all the I/O
> is transmitted across that boundary in plaintext anyway, and also it
> seems that hardware-wrapped keys will be supported too.
> 
> Please make virtio-blk support the key programming, eviction, and
> HW-wrapped key management operations that are needed for this to work.
> 
> - Eric

Thanks for your comments!

Not making virtio-blk itself support key programming and eviction is
something done deliberately. Based on that HW-wrapped key management
operations are also handled in the out-of-band path.

There are bellow 2 approaches I investigated to let virtio-blk programming
the key.

  1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
     key and evict key interfaces.(Same to 'the virtio-blk interface standardized
     blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)

     The guest's block crypto profile manages the keyslot in virtual slot
     format in this scenario.

      - block crypto key and virt_slot index it passed to the hypervisor's
       (EL2) device emulation (QEMU, using QEMU in the following) which
       runs in userspace of the host. Besides of transferring the virtual
       slot to the physical slot, a programming block crypto key UAPI need
       be added. Follow current blk-crypto design, it may be like
       BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
       allows userspace process a key into a key slot.

     - For key eviction, it's similar to above key programming handling, also
       need a key eviction in blk IOCTLs, but leads to the security risk
       that allow userspace client to evict a key in a key slot.

    virt_slot, DUN and DUSize is appended to virtblk request during crypto
    I/O.

  2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
     key and evict key interfaces.

     The guest's block crypto profile doesn't manage keyslot for the guest,
     the host's block crypto profile manages keyslot for both the guest and
     the host. The trigger of key programming operation is moved from the
     guest to the host.

       - The whole block crypto key (key size, key bytes, blk_crypto_config)
         and DUN are appended to the virtblk request during IO, a little
         high payload.

         The backend parses the crypto message in the virtio queue and
         construct a block crypto key and DUN for the bio_crypto_ctx
         set to the BIO. So that the IO flow in the host can program
         the key. 

         The question is that the blk-crypto-profile distinguishs the
         block crypto key via the key's address. But the host has
         different key addresses for the programming and eviction key
         operations of the same block crypto key from GVM, because the
         key is re-constructed in the host for the key program and
         eviction operations. 

         To fix it, the approach I thought is maintaining a new key
         hash table in the backend, and comparing the block crypto key
         content and DUN parsed from virtio queue with that in the key
         hash table. 
         My major concern is that this need keep the keys synchronization
         b/w this new hash table and the blk-crypto-profile's hash table
         carefully, avoiding that key is still present in the
         blk-crypto-profile's hash table, but removed in backend hash
         table. Another point is that the whole block crypto key and
         DUN are appended into virtio block request per crypto I/O.

       - For key eviction, adding a key eviction in blk IOCTLs allows
         userspace client to evict a key in a key slot. I thought this
         is a security concern.


     This option doesn't need map virt_slot to physical one.


Taking all the above into account, I made a compromise to implement
blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
blk only need focus on the data path. I agree that it is complex than
the second option mentioned in the above, but small payload (only
virt_slot, DUN, DUSize) in the virtio block request and no security
risk of key eviction from userspace.


I would like to hear your thoughts about the above and am appreciated
if you could share your insights about the design of inline
encryption in virtio block.

Regards,
Linlin

On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
> Thanks for your comments!
> 
> Not making virtio-blk itself support key programming and eviction is
> something done deliberately. Based on that HW-wrapped key management
> operations are also handled in the out-of-band path.
> 
> There are bellow 2 approaches I investigated to let virtio-blk programming
> the key.
> 
>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
> 
>      The guest's block crypto profile manages the keyslot in virtual slot
>      format in this scenario.
> 
>       - block crypto key and virt_slot index it passed to the hypervisor's
>        (EL2) device emulation (QEMU, using QEMU in the following) which
>        runs in userspace of the host. Besides of transferring the virtual
>        slot to the physical slot, a programming block crypto key UAPI need
>        be added. Follow current blk-crypto design, it may be like
>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>        allows userspace process a key into a key slot.
> 
>      - For key eviction, it's similar to above key programming handling, also
>        need a key eviction in blk IOCTLs, but leads to the security risk
>        that allow userspace client to evict a key in a key slot.
> 
>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>     I/O.
> 
>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>      key and evict key interfaces.
> 
>      The guest's block crypto profile doesn't manage keyslot for the guest,
>      the host's block crypto profile manages keyslot for both the guest and
>      the host. The trigger of key programming operation is moved from the
>      guest to the host.
> 
>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>          and DUN are appended to the virtblk request during IO, a little
>          high payload.
> 
>          The backend parses the crypto message in the virtio queue and
>          construct a block crypto key and DUN for the bio_crypto_ctx
>          set to the BIO. So that the IO flow in the host can program
>          the key. 
> 
>          The question is that the blk-crypto-profile distinguishs the
>          block crypto key via the key's address. But the host has
>          different key addresses for the programming and eviction key
>          operations of the same block crypto key from GVM, because the
>          key is re-constructed in the host for the key program and
>          eviction operations. 
> 
>          To fix it, the approach I thought is maintaining a new key
>          hash table in the backend, and comparing the block crypto key
>          content and DUN parsed from virtio queue with that in the key
>          hash table. 
>          My major concern is that this need keep the keys synchronization
>          b/w this new hash table and the blk-crypto-profile's hash table
>          carefully, avoiding that key is still present in the
>          blk-crypto-profile's hash table, but removed in backend hash
>          table. Another point is that the whole block crypto key and
>          DUN are appended into virtio block request per crypto I/O.
> 
>        - For key eviction, adding a key eviction in blk IOCTLs allows
>          userspace client to evict a key in a key slot. I thought this
>          is a security concern.
> 
> 
>      This option doesn't need map virt_slot to physical one.
> 
> 
> Taking all the above into account, I made a compromise to implement
> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
> blk only need focus on the data path. I agree that it is complex than
> the second option mentioned in the above, but small payload (only
> virt_slot, DUN, DUSize) in the virtio block request and no security
> risk of key eviction from userspace.
> 
> 
> I would like to hear your thoughts about the above and am appreciated
> if you could share your insights about the design of inline
> encryption in virtio block.

Well, the way it should work is that each virtio-blk device should have
its own set of *virtual* keyslots on the host side.  From the guest's
perspective it would act very similarly to UFS / eMMC inline encryption,
and it would be easy to integrate into the existing stack.

Then to process encrypted I/O, the host would use the keyslot number in
the I/O to look up the blk_crypto_key it previously saved, and issue I/O
using that key (using bio_crypt_set_ctx()).  The existing keyslot
management logic in the block layer would allocate or wait for a
physical keyslot as needed, so it should just work.

Eviction would similarly be passed through to blk_crypto_evict_key().

Note that with this design, there would be no static partitioning of the
physical keyslots.  The host would just allocate and release them as
needed, similar to memory allocation.  The total number of virtual
keyslots could be greater than the number of physical keyslots.

This design would also work with hardware-wrapped keys.

The hardest part is still the UAPIs for the VMM to do what it needs to
do (assuming that it even needs to support physical inline encryption
hardware at all, and not simply use the AES acceleration on the CPU),
but that is the case with any of the proposals.

- Eric

On 9/1/2026 5:07 AM, Eric Biggers wrote:
> On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
>> Thanks for your comments!
>>
>> Not making virtio-blk itself support key programming and eviction is
>> something done deliberately. Based on that HW-wrapped key management
>> operations are also handled in the out-of-band path.
>>
>> There are bellow 2 approaches I investigated to let virtio-blk programming
>> the key.
>>
>>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
>>
>>      The guest's block crypto profile manages the keyslot in virtual slot
>>      format in this scenario.
>>
>>       - block crypto key and virt_slot index it passed to the hypervisor's
>>        (EL2) device emulation (QEMU, using QEMU in the following) which
>>        runs in userspace of the host. Besides of transferring the virtual
>>        slot to the physical slot, a programming block crypto key UAPI need
>>        be added. Follow current blk-crypto design, it may be like
>>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>>        allows userspace process a key into a key slot.
>>
>>      - For key eviction, it's similar to above key programming handling, also
>>        need a key eviction in blk IOCTLs, but leads to the security risk
>>        that allow userspace client to evict a key in a key slot.
>>
>>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>>     I/O.
>>
>>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>>      key and evict key interfaces.
>>
>>      The guest's block crypto profile doesn't manage keyslot for the guest,
>>      the host's block crypto profile manages keyslot for both the guest and
>>      the host. The trigger of key programming operation is moved from the
>>      guest to the host.
>>
>>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>>          and DUN are appended to the virtblk request during IO, a little
>>          high payload.
>>
>>          The backend parses the crypto message in the virtio queue and
>>          construct a block crypto key and DUN for the bio_crypto_ctx
>>          set to the BIO. So that the IO flow in the host can program
>>          the key. 
>>
>>          The question is that the blk-crypto-profile distinguishs the
>>          block crypto key via the key's address. But the host has
>>          different key addresses for the programming and eviction key
>>          operations of the same block crypto key from GVM, because the
>>          key is re-constructed in the host for the key program and
>>          eviction operations. 
>>
>>          To fix it, the approach I thought is maintaining a new key
>>          hash table in the backend, and comparing the block crypto key
>>          content and DUN parsed from virtio queue with that in the key
>>          hash table. 
>>          My major concern is that this need keep the keys synchronization
>>          b/w this new hash table and the blk-crypto-profile's hash table
>>          carefully, avoiding that key is still present in the
>>          blk-crypto-profile's hash table, but removed in backend hash
>>          table. Another point is that the whole block crypto key and
>>          DUN are appended into virtio block request per crypto I/O.
>>
>>        - For key eviction, adding a key eviction in blk IOCTLs allows
>>          userspace client to evict a key in a key slot. I thought this
>>          is a security concern.
>>
>>
>>      This option doesn't need map virt_slot to physical one.
>>
>>
>> Taking all the above into account, I made a compromise to implement
>> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
>> blk only need focus on the data path. I agree that it is complex than
>> the second option mentioned in the above, but small payload (only
>> virt_slot, DUN, DUSize) in the virtio block request and no security
>> risk of key eviction from userspace.
>>
>>
>> I would like to hear your thoughts about the above and am appreciated
>> if you could share your insights about the design of inline
>> encryption in virtio block.
> 
> Well, the way it should work is that each virtio-blk device should have
> its own set of *virtual* keyslots on the host side.  From the guest's
> perspective it would act very similarly to UFS / eMMC inline encryption,
> and it would be easy to integrate into the existing stack.
> 
> Then to process encrypted I/O, the host would use the keyslot number in
> the I/O to look up the blk_crypto_key it previously saved, and issue I/O
> using that key (using bio_crypt_set_ctx()).  The existing keyslot
> management logic in the block layer would allocate or wait for a
> physical keyslot as needed, so it should just work.
> 
> Eviction would similarly be passed through to blk_crypto_evict_key().
> 
> Note that with this design, there would be no static partitioning of the
> physical keyslots.  The host would just allocate and release them as
> needed, similar to memory allocation.  The total number of virtual
> keyslots could be greater than the number of physical keyslots.
> 
> This design would also work with hardware-wrapped keys.
> 
> The hardest part is still the UAPIs for the VMM to do what it needs to
> do (assuming that it even needs to support physical inline encryption
> hardware at all, and not simply use the AES acceleration on the CPU),
> but that is the case with any of the proposals.
> 
> - Eric

Thanks for your insights and clarification.

This is a very clear architecture for virtio-blk inline encryption support
and is quite similar to what I referred to as approach 1, with a few notable
differences:

  - The host maintains a set of virtual keyslots per virtio-blk device.
  - Physical keyslots are not statically partitioned.
  - The virtual-to-physical slot mapping is managed as part of the virtio-blk
    device implementation rather than being tied to a VM-wide slot table.

With this design, I believe there would be two VM exits associated with
encrypted I/O:
  1. Key programming
     Before encrypted I/O can be submitted, the guest needs to program a key
     into a virtual keyslot:
     
     Guest virtio-blk driver
         -> VM exit
         -> virtio-blk backend (e.g. QEMU)
         -> ioctl
         -> host virtio-blk proxy driver
            (stores the blk_crypto_key in a virtual keyslot)

  2. Encrypted I/O submission
     The I/O request carries the virtual keyslot number and DUN:

     Guest virtio-blk driver
        -> VM exit
        -> virtio-blk backend (e.g. QEMU)
        -> ioctl
        -> host virtio-blk proxy driver
           (looks up the blk_crypto_key associated with the virtual keyslot
            and submits I/O using bio_crypt_set_ctx())

I have been wondering whether a model similar to the passthrough
blk-crypto-profile used by certain dm targets could be applicable here.

In such a design, ownership of keyslot management would effectively and totally
move to the host. The guest would no longer manage virtual keyslots, and the
host block layer would continue using its existing keyslot manager to allocate,
reuse, and evict physical keyslots as needed.

This corresponds to what I previously described as approach 2.
With this approach, encrypted I/O would require only a single VM exit:
  1. Encrypted I/O submission
     The I/O request carries the blk_crypto_key and DUN:

     Guest virtio-blk driver
        -> VM exit
        -> virtio-blk backend (e.g. QEMU)
        -> ioctl
        -> host virtio-blk proxy driver
           (receives the blk_crypto_key and DUN, then submits I/O using
            bio_crypt_set_ctx())

Regardless of which approach is used, the host would still need to maintain a
key lookup table so that the host blk-crypto layer consistently sees the same
blk_crypto_key object for a guest key until that key is evicted.

Note that, the key table format is
  - approach 1: <virtual_slot, blk_crypto_key>
  - approach 2: <hash_id of blk_crypto_key, blk_crypto_key>

Therefore, if transferring blk_crypto_key material from the guest to the host
(and from userspace to kernel space) is considered acceptable, I wonder whether
approach 2 might be preferable because it avoids virtual keyslots entirely and
reduces the encrypted I/O path to a single VM exit.

Do you see any major drawbacks with such an approach? In particular, do you
think reducing the number of VM exits by eliminating virtual keyslots is a
worthwhile direction for virtio-blk inline encryption support?

- Linlin

On Tue, Sep 01, 2026 at 04:22:33PM +0800, Linlin Zhang wrote:
> With this design, I believe there would be two VM exits associated with
> encrypted I/O:
>   1. Key programming
>      Before encrypted I/O can be submitted, the guest needs to program a key
>      into a virtual keyslot:
>      
>      Guest virtio-blk driver
>          -> VM exit
>          -> virtio-blk backend (e.g. QEMU)
>          -> ioctl
>          -> host virtio-blk proxy driver
>             (stores the blk_crypto_key in a virtual keyslot)
> 
>   2. Encrypted I/O submission
>      The I/O request carries the virtual keyslot number and DUN:
> 
>      Guest virtio-blk driver
>         -> VM exit
>         -> virtio-blk backend (e.g. QEMU)
>         -> ioctl
>         -> host virtio-blk proxy driver
>            (looks up the blk_crypto_key associated with the virtual keyslot
>             and submits I/O using bio_crypt_set_ctx())

Most I/O requests only need step (2), since they reuse step (1) from a
previous I/O request.  The kernel evicts a keyslot only when it is the
least recently used among all the keyslots and another one is needed.
(Or when eviction is explicitly requested.)

This is especially relevant when only a small number of keys is used,
like is the case when the per-file key support in fscrypt is disabled.
Programming keys on Qualcomm ICE has always been very slow even on
physical hardware, and the kernel was already designed to mitigate that.

> I have been wondering whether a model similar to the passthrough
> blk-crypto-profile used by certain dm targets could be applicable here.
> 
> In such a design, ownership of keyslot management would effectively and totally
> move to the host. The guest would no longer manage virtual keyslots, and the
> host block layer would continue using its existing keyslot manager to allocate,
> reuse, and evict physical keyslots as needed.

I think the virtual keyslots are still useful so that the key bytes, key
size, key type, algorithm, and data unit size don't have to be
transmitted in every I/O request, then all revalidated and processed
again.  There is a reason that inline encryption hardware uses keyslots,
and I think the same largely applies to virtio-blk.

- Eric

On 9/2/2026 5:28 AM, Eric Biggers wrote:
> On Tue, Sep 01, 2026 at 04:22:33PM +0800, Linlin Zhang wrote:
>> With this design, I believe there would be two VM exits associated with
>> encrypted I/O:
>>   1. Key programming
>>      Before encrypted I/O can be submitted, the guest needs to program a key
>>      into a virtual keyslot:
>>      
>>      Guest virtio-blk driver
>>          -> VM exit
>>          -> virtio-blk backend (e.g. QEMU)
>>          -> ioctl
>>          -> host virtio-blk proxy driver
>>             (stores the blk_crypto_key in a virtual keyslot)
>>
>>   2. Encrypted I/O submission
>>      The I/O request carries the virtual keyslot number and DUN:
>>
>>      Guest virtio-blk driver
>>         -> VM exit
>>         -> virtio-blk backend (e.g. QEMU)
>>         -> ioctl
>>         -> host virtio-blk proxy driver
>>            (looks up the blk_crypto_key associated with the virtual keyslot
>>             and submits I/O using bio_crypt_set_ctx())
> 
> Most I/O requests only need step (2), since they reuse step (1) from a
> previous I/O request.  The kernel evicts a keyslot only when it is the
> least recently used among all the keyslots and another one is needed.
> (Or when eviction is explicitly requested.)
> 
> This is especially relevant when only a small number of keys is used,
> like is the case when the per-file key support in fscrypt is disabled.
> Programming keys on Qualcomm ICE has always been very slow even on
> physical hardware, and the kernel was already designed to mitigate that.
> 

ACK

>> I have been wondering whether a model similar to the passthrough
>> blk-crypto-profile used by certain dm targets could be applicable here.
>>
>> In such a design, ownership of keyslot management would effectively and totally
>> move to the host. The guest would no longer manage virtual keyslots, and the
>> host block layer would continue using its existing keyslot manager to allocate,
>> reuse, and evict physical keyslots as needed.
> 
> I think the virtual keyslots are still useful so that the key bytes, key
> size, key type, algorithm, and data unit size don't have to be
> transmitted in every I/O request, then all revalidated and processed
> again.  There is a reason that inline encryption hardware uses keyslots,
> and I think the same largely applies to virtio-blk.
> 

Thanks for your clarification!

I agree that virtual keyslots still provide an important benefit by
avoiding revalidation of the crypto context on every I/O request and
reducing payload for most encrypted I/O request.

One concern I had is around ownership and lifetime management. If 
key programming and I/O submission are exposed through separate interfaces,
the implementation needs to ensure that a programmed virtual keyslot
cannot be modified or replaced unexpectedly before the associated I/O
is submitted. Otherwise, I/O could end up being issued with a different
key than the one originally intended.

As Stefan pointed out, this sounds more like an object lifetime and
permission model problem than a keyslot model problem. My current
thinking is that blk-crypto-proxy could own both virtual keyslot management
and the corresponding access control. For example, key programming and
key eviction could be exposed through blk-crypto-proxy ioctls, while
the block device and its virtual keyslot namespace are instantiated when
the blk-crypto-proxy device is opened. That would allow ownership and
lifetime of programmed keys to be tied to a specific file descriptor context.

Separately, regarding the I/O submission path itself (patch 08 in this
series), I'd also appreciate your thoughts and block maintainers'
opinions on the direction that would be preferable from a block-layer
perspective.

My initial prototype introduced a dedicated ioctl for submitting I/O
carrying blk-crypto metadata because of the additional validation
requirements around DUN handling and data-unit alignment. However, I
understand the concern about introducing a separate I/O submission
interface that bypasses existing optimized paths such as io_uring.

Do you think these blk-crypto-specific requirements should instead be
integrated into an existing interface, such as io_uring, or is there any
precedent for introducing a dedicated interface when additional crypto
metadata needs to accompany I/O requests?


> - Eric

On 9/1/2026 4:22 PM, Linlin Zhang wrote:
> 
> 
> On 9/1/2026 5:07 AM, Eric Biggers wrote:
>> On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
>>> Thanks for your comments!
>>>
>>> Not making virtio-blk itself support key programming and eviction is
>>> something done deliberately. Based on that HW-wrapped key management
>>> operations are also handled in the out-of-band path.
>>>
>>> There are bellow 2 approaches I investigated to let virtio-blk programming
>>> the key.
>>>
>>>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>>>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>>>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
>>>
>>>      The guest's block crypto profile manages the keyslot in virtual slot
>>>      format in this scenario.
>>>
>>>       - block crypto key and virt_slot index it passed to the hypervisor's
>>>        (EL2) device emulation (QEMU, using QEMU in the following) which
>>>        runs in userspace of the host. Besides of transferring the virtual
>>>        slot to the physical slot, a programming block crypto key UAPI need
>>>        be added. Follow current blk-crypto design, it may be like
>>>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>>>        allows userspace process a key into a key slot.
>>>
>>>      - For key eviction, it's similar to above key programming handling, also
>>>        need a key eviction in blk IOCTLs, but leads to the security risk
>>>        that allow userspace client to evict a key in a key slot.
>>>
>>>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>>>     I/O.
>>>
>>>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>>>      key and evict key interfaces.
>>>
>>>      The guest's block crypto profile doesn't manage keyslot for the guest,
>>>      the host's block crypto profile manages keyslot for both the guest and
>>>      the host. The trigger of key programming operation is moved from the
>>>      guest to the host.
>>>
>>>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>>>          and DUN are appended to the virtblk request during IO, a little
>>>          high payload.
>>>
>>>          The backend parses the crypto message in the virtio queue and
>>>          construct a block crypto key and DUN for the bio_crypto_ctx
>>>          set to the BIO. So that the IO flow in the host can program
>>>          the key. 
>>>
>>>          The question is that the blk-crypto-profile distinguishs the
>>>          block crypto key via the key's address. But the host has
>>>          different key addresses for the programming and eviction key
>>>          operations of the same block crypto key from GVM, because the
>>>          key is re-constructed in the host for the key program and
>>>          eviction operations. 
>>>
>>>          To fix it, the approach I thought is maintaining a new key
>>>          hash table in the backend, and comparing the block crypto key
>>>          content and DUN parsed from virtio queue with that in the key
>>>          hash table. 
>>>          My major concern is that this need keep the keys synchronization
>>>          b/w this new hash table and the blk-crypto-profile's hash table
>>>          carefully, avoiding that key is still present in the
>>>          blk-crypto-profile's hash table, but removed in backend hash
>>>          table. Another point is that the whole block crypto key and
>>>          DUN are appended into virtio block request per crypto I/O.
>>>
>>>        - For key eviction, adding a key eviction in blk IOCTLs allows
>>>          userspace client to evict a key in a key slot. I thought this
>>>          is a security concern.
>>>
>>>
>>>      This option doesn't need map virt_slot to physical one.
>>>
>>>
>>> Taking all the above into account, I made a compromise to implement
>>> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
>>> blk only need focus on the data path. I agree that it is complex than
>>> the second option mentioned in the above, but small payload (only
>>> virt_slot, DUN, DUSize) in the virtio block request and no security
>>> risk of key eviction from userspace.
>>>
>>>
>>> I would like to hear your thoughts about the above and am appreciated
>>> if you could share your insights about the design of inline
>>> encryption in virtio block.
>>
>> Well, the way it should work is that each virtio-blk device should have
>> its own set of *virtual* keyslots on the host side.  From the guest's
>> perspective it would act very similarly to UFS / eMMC inline encryption,
>> and it would be easy to integrate into the existing stack.
>>
>> Then to process encrypted I/O, the host would use the keyslot number in
>> the I/O to look up the blk_crypto_key it previously saved, and issue I/O
>> using that key (using bio_crypt_set_ctx()).  The existing keyslot
>> management logic in the block layer would allocate or wait for a
>> physical keyslot as needed, so it should just work.
>>
>> Eviction would similarly be passed through to blk_crypto_evict_key().
>>
>> Note that with this design, there would be no static partitioning of the
>> physical keyslots.  The host would just allocate and release them as
>> needed, similar to memory allocation.  The total number of virtual
>> keyslots could be greater than the number of physical keyslots.
>>
>> This design would also work with hardware-wrapped keys.
>>
>> The hardest part is still the UAPIs for the VMM to do what it needs to
>> do (assuming that it even needs to support physical inline encryption
>> hardware at all, and not simply use the AES acceleration on the CPU),
>> but that is the case with any of the proposals.
>>
>> - Eric
> 
> Thanks for your insights and clarification.
> 
> This is a very clear architecture for virtio-blk inline encryption support
> and is quite similar to what I referred to as approach 1, with a few notable
> differences:
> 
>   - The host maintains a set of virtual keyslots per virtio-blk device.
>   - Physical keyslots are not statically partitioned.
>   - The virtual-to-physical slot mapping is managed as part of the virtio-blk
>     device implementation rather than being tied to a VM-wide slot table.
> 
> With this design, I believe there would be two VM exits associated with
> encrypted I/O:
>   1. Key programming
>      Before encrypted I/O can be submitted, the guest needs to program a key
>      into a virtual keyslot:
>      
>      Guest virtio-blk driver
>          -> VM exit
>          -> virtio-blk backend (e.g. QEMU)
>          -> ioctl
>          -> host virtio-blk proxy driver
>             (stores the blk_crypto_key in a virtual keyslot)
> 
>   2. Encrypted I/O submission
>      The I/O request carries the virtual keyslot number and DUN:
> 
>      Guest virtio-blk driver
>         -> VM exit
>         -> virtio-blk backend (e.g. QEMU)
>         -> ioctl
>         -> host virtio-blk proxy driver
>            (looks up the blk_crypto_key associated with the virtual keyslot
>             and submits I/O using bio_crypt_set_ctx())

Supplementation.
This may be a potential security concern. The host virtio-blk proxy driver exposes
API for key programming, the input parameters are blk_crypto key and virtual
slot. If a malicious program replace the key in a specific virtual slot between
key program call and I/O submission via this API, the data would be encrypted
by the unintentional key.

> 
> I have been wondering whether a model similar to the passthrough
> blk-crypto-profile used by certain dm targets could be applicable here.
> 
> In such a design, ownership of keyslot management would effectively and totally
> move to the host. The guest would no longer manage virtual keyslots, and the
> host block layer would continue using its existing keyslot manager to allocate,
> reuse, and evict physical keyslots as needed.
> 
> This corresponds to what I previously described as approach 2.
> With this approach, encrypted I/O would require only a single VM exit:
>   1. Encrypted I/O submission
>      The I/O request carries the blk_crypto_key and DUN:
> 
>      Guest virtio-blk driver
>         -> VM exit
>         -> virtio-blk backend (e.g. QEMU)
>         -> ioctl
>         -> host virtio-blk proxy driver
>            (receives the blk_crypto_key and DUN, then submits I/O using
>             bio_crypt_set_ctx())
> 
> Regardless of which approach is used, the host would still need to maintain a
> key lookup table so that the host blk-crypto layer consistently sees the same
> blk_crypto_key object for a guest key until that key is evicted.
> 
> Note that, the key table format is
>   - approach 1: <virtual_slot, blk_crypto_key>
>   - approach 2: <hash_id of blk_crypto_key, blk_crypto_key>
> 
> Therefore, if transferring blk_crypto_key material from the guest to the host
> (and from userspace to kernel space) is considered acceptable, I wonder whether
> approach 2 might be preferable because it avoids virtual keyslots entirely and
> reduces the encrypted I/O path to a single VM exit.
> 
> Do you see any major drawbacks with such an approach? In particular, do you
> think reducing the number of VM exits by eliminating virtual keyslots is a
> worthwhile direction for virtio-blk inline encryption support?
> 
> - Linlin

On Tue, Sep 01, 2026 at 04:45:19PM +0800, Linlin Zhang wrote:
> 
> 
> On 9/1/2026 4:22 PM, Linlin Zhang wrote:
> > 
> > 
> > On 9/1/2026 5:07 AM, Eric Biggers wrote:
> >> On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
> >>> Thanks for your comments!
> >>>
> >>> Not making virtio-blk itself support key programming and eviction is
> >>> something done deliberately. Based on that HW-wrapped key management
> >>> operations are also handled in the out-of-band path.
> >>>
> >>> There are bellow 2 approaches I investigated to let virtio-blk programming
> >>> the key.
> >>>
> >>>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
> >>>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
> >>>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
> >>>
> >>>      The guest's block crypto profile manages the keyslot in virtual slot
> >>>      format in this scenario.
> >>>
> >>>       - block crypto key and virt_slot index it passed to the hypervisor's
> >>>        (EL2) device emulation (QEMU, using QEMU in the following) which
> >>>        runs in userspace of the host. Besides of transferring the virtual
> >>>        slot to the physical slot, a programming block crypto key UAPI need
> >>>        be added. Follow current blk-crypto design, it may be like
> >>>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
> >>>        allows userspace process a key into a key slot.
> >>>
> >>>      - For key eviction, it's similar to above key programming handling, also
> >>>        need a key eviction in blk IOCTLs, but leads to the security risk
> >>>        that allow userspace client to evict a key in a key slot.
> >>>
> >>>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
> >>>     I/O.
> >>>
> >>>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
> >>>      key and evict key interfaces.
> >>>
> >>>      The guest's block crypto profile doesn't manage keyslot for the guest,
> >>>      the host's block crypto profile manages keyslot for both the guest and
> >>>      the host. The trigger of key programming operation is moved from the
> >>>      guest to the host.
> >>>
> >>>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
> >>>          and DUN are appended to the virtblk request during IO, a little
> >>>          high payload.
> >>>
> >>>          The backend parses the crypto message in the virtio queue and
> >>>          construct a block crypto key and DUN for the bio_crypto_ctx
> >>>          set to the BIO. So that the IO flow in the host can program
> >>>          the key. 
> >>>
> >>>          The question is that the blk-crypto-profile distinguishs the
> >>>          block crypto key via the key's address. But the host has
> >>>          different key addresses for the programming and eviction key
> >>>          operations of the same block crypto key from GVM, because the
> >>>          key is re-constructed in the host for the key program and
> >>>          eviction operations. 
> >>>
> >>>          To fix it, the approach I thought is maintaining a new key
> >>>          hash table in the backend, and comparing the block crypto key
> >>>          content and DUN parsed from virtio queue with that in the key
> >>>          hash table. 
> >>>          My major concern is that this need keep the keys synchronization
> >>>          b/w this new hash table and the blk-crypto-profile's hash table
> >>>          carefully, avoiding that key is still present in the
> >>>          blk-crypto-profile's hash table, but removed in backend hash
> >>>          table. Another point is that the whole block crypto key and
> >>>          DUN are appended into virtio block request per crypto I/O.
> >>>
> >>>        - For key eviction, adding a key eviction in blk IOCTLs allows
> >>>          userspace client to evict a key in a key slot. I thought this
> >>>          is a security concern.
> >>>
> >>>
> >>>      This option doesn't need map virt_slot to physical one.
> >>>
> >>>
> >>> Taking all the above into account, I made a compromise to implement
> >>> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
> >>> blk only need focus on the data path. I agree that it is complex than
> >>> the second option mentioned in the above, but small payload (only
> >>> virt_slot, DUN, DUSize) in the virtio block request and no security
> >>> risk of key eviction from userspace.
> >>>
> >>>
> >>> I would like to hear your thoughts about the above and am appreciated
> >>> if you could share your insights about the design of inline
> >>> encryption in virtio block.
> >>
> >> Well, the way it should work is that each virtio-blk device should have
> >> its own set of *virtual* keyslots on the host side.  From the guest's
> >> perspective it would act very similarly to UFS / eMMC inline encryption,
> >> and it would be easy to integrate into the existing stack.
> >>
> >> Then to process encrypted I/O, the host would use the keyslot number in
> >> the I/O to look up the blk_crypto_key it previously saved, and issue I/O
> >> using that key (using bio_crypt_set_ctx()).  The existing keyslot
> >> management logic in the block layer would allocate or wait for a
> >> physical keyslot as needed, so it should just work.
> >>
> >> Eviction would similarly be passed through to blk_crypto_evict_key().
> >>
> >> Note that with this design, there would be no static partitioning of the
> >> physical keyslots.  The host would just allocate and release them as
> >> needed, similar to memory allocation.  The total number of virtual
> >> keyslots could be greater than the number of physical keyslots.
> >>
> >> This design would also work with hardware-wrapped keys.
> >>
> >> The hardest part is still the UAPIs for the VMM to do what it needs to
> >> do (assuming that it even needs to support physical inline encryption
> >> hardware at all, and not simply use the AES acceleration on the CPU),
> >> but that is the case with any of the proposals.
> >>
> >> - Eric
> > 
> > Thanks for your insights and clarification.
> > 
> > This is a very clear architecture for virtio-blk inline encryption support
> > and is quite similar to what I referred to as approach 1, with a few notable
> > differences:
> > 
> >   - The host maintains a set of virtual keyslots per virtio-blk device.
> >   - Physical keyslots are not statically partitioned.
> >   - The virtual-to-physical slot mapping is managed as part of the virtio-blk
> >     device implementation rather than being tied to a VM-wide slot table.
> > 
> > With this design, I believe there would be two VM exits associated with
> > encrypted I/O:
> >   1. Key programming
> >      Before encrypted I/O can be submitted, the guest needs to program a key
> >      into a virtual keyslot:
> >      
> >      Guest virtio-blk driver
> >          -> VM exit
> >          -> virtio-blk backend (e.g. QEMU)
> >          -> ioctl
> >          -> host virtio-blk proxy driver
> >             (stores the blk_crypto_key in a virtual keyslot)
> > 
> >   2. Encrypted I/O submission
> >      The I/O request carries the virtual keyslot number and DUN:
> > 
> >      Guest virtio-blk driver
> >         -> VM exit
> >         -> virtio-blk backend (e.g. QEMU)
> >         -> ioctl
> >         -> host virtio-blk proxy driver
> >            (looks up the blk_crypto_key associated with the virtual keyslot
> >             and submits I/O using bio_crypt_set_ctx())
> 
> Supplementation.
> This may be a potential security concern. The host virtio-blk proxy driver exposes
> API for key programming, the input parameters are blk_crypto key and virtual
> slot. If a malicious program replace the key in a specific virtual slot between
> key program call and I/O submission via this API, the data would be encrypted
> by the unintentional key.

If the virtual key slots in the uapi are per file descriptor rather than
per inode, then other programs cannot interfere with each other's
virtual key slots. Each program gets its own virtual key space when it
opens a file descriptor. Sharing is only possible by inheriting or
passing a file descriptor to another process and that's good for
security.

Stefan

On 9/2/2026 3:44 AM, Stefan Hajnoczi wrote:
> On Tue, Sep 01, 2026 at 04:45:19PM +0800, Linlin Zhang wrote:
>>
>>
>> On 9/1/2026 4:22 PM, Linlin Zhang wrote:
>>>
>>>
>>> On 9/1/2026 5:07 AM, Eric Biggers wrote:
>>>> On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
>>>>> Thanks for your comments!
>>>>>
>>>>> Not making virtio-blk itself support key programming and eviction is
>>>>> something done deliberately. Based on that HW-wrapped key management
>>>>> operations are also handled in the out-of-band path.
>>>>>
>>>>> There are bellow 2 approaches I investigated to let virtio-blk programming
>>>>> the key.
>>>>>
>>>>>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>>>>>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>>>>>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
>>>>>
>>>>>      The guest's block crypto profile manages the keyslot in virtual slot
>>>>>      format in this scenario.
>>>>>
>>>>>       - block crypto key and virt_slot index it passed to the hypervisor's
>>>>>        (EL2) device emulation (QEMU, using QEMU in the following) which
>>>>>        runs in userspace of the host. Besides of transferring the virtual
>>>>>        slot to the physical slot, a programming block crypto key UAPI need
>>>>>        be added. Follow current blk-crypto design, it may be like
>>>>>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>>>>>        allows userspace process a key into a key slot.
>>>>>
>>>>>      - For key eviction, it's similar to above key programming handling, also
>>>>>        need a key eviction in blk IOCTLs, but leads to the security risk
>>>>>        that allow userspace client to evict a key in a key slot.
>>>>>
>>>>>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>>>>>     I/O.
>>>>>
>>>>>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>>>>>      key and evict key interfaces.
>>>>>
>>>>>      The guest's block crypto profile doesn't manage keyslot for the guest,
>>>>>      the host's block crypto profile manages keyslot for both the guest and
>>>>>      the host. The trigger of key programming operation is moved from the
>>>>>      guest to the host.
>>>>>
>>>>>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>>>>>          and DUN are appended to the virtblk request during IO, a little
>>>>>          high payload.
>>>>>
>>>>>          The backend parses the crypto message in the virtio queue and
>>>>>          construct a block crypto key and DUN for the bio_crypto_ctx
>>>>>          set to the BIO. So that the IO flow in the host can program
>>>>>          the key. 
>>>>>
>>>>>          The question is that the blk-crypto-profile distinguishs the
>>>>>          block crypto key via the key's address. But the host has
>>>>>          different key addresses for the programming and eviction key
>>>>>          operations of the same block crypto key from GVM, because the
>>>>>          key is re-constructed in the host for the key program and
>>>>>          eviction operations. 
>>>>>
>>>>>          To fix it, the approach I thought is maintaining a new key
>>>>>          hash table in the backend, and comparing the block crypto key
>>>>>          content and DUN parsed from virtio queue with that in the key
>>>>>          hash table. 
>>>>>          My major concern is that this need keep the keys synchronization
>>>>>          b/w this new hash table and the blk-crypto-profile's hash table
>>>>>          carefully, avoiding that key is still present in the
>>>>>          blk-crypto-profile's hash table, but removed in backend hash
>>>>>          table. Another point is that the whole block crypto key and
>>>>>          DUN are appended into virtio block request per crypto I/O.
>>>>>
>>>>>        - For key eviction, adding a key eviction in blk IOCTLs allows
>>>>>          userspace client to evict a key in a key slot. I thought this
>>>>>          is a security concern.
>>>>>
>>>>>
>>>>>      This option doesn't need map virt_slot to physical one.
>>>>>
>>>>>
>>>>> Taking all the above into account, I made a compromise to implement
>>>>> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
>>>>> blk only need focus on the data path. I agree that it is complex than
>>>>> the second option mentioned in the above, but small payload (only
>>>>> virt_slot, DUN, DUSize) in the virtio block request and no security
>>>>> risk of key eviction from userspace.
>>>>>
>>>>>
>>>>> I would like to hear your thoughts about the above and am appreciated
>>>>> if you could share your insights about the design of inline
>>>>> encryption in virtio block.
>>>>
>>>> Well, the way it should work is that each virtio-blk device should have
>>>> its own set of *virtual* keyslots on the host side.  From the guest's
>>>> perspective it would act very similarly to UFS / eMMC inline encryption,
>>>> and it would be easy to integrate into the existing stack.
>>>>
>>>> Then to process encrypted I/O, the host would use the keyslot number in
>>>> the I/O to look up the blk_crypto_key it previously saved, and issue I/O
>>>> using that key (using bio_crypt_set_ctx()).  The existing keyslot
>>>> management logic in the block layer would allocate or wait for a
>>>> physical keyslot as needed, so it should just work.
>>>>
>>>> Eviction would similarly be passed through to blk_crypto_evict_key().
>>>>
>>>> Note that with this design, there would be no static partitioning of the
>>>> physical keyslots.  The host would just allocate and release them as
>>>> needed, similar to memory allocation.  The total number of virtual
>>>> keyslots could be greater than the number of physical keyslots.
>>>>
>>>> This design would also work with hardware-wrapped keys.
>>>>
>>>> The hardest part is still the UAPIs for the VMM to do what it needs to
>>>> do (assuming that it even needs to support physical inline encryption
>>>> hardware at all, and not simply use the AES acceleration on the CPU),
>>>> but that is the case with any of the proposals.
>>>>
>>>> - Eric
>>>
>>> Thanks for your insights and clarification.
>>>
>>> This is a very clear architecture for virtio-blk inline encryption support
>>> and is quite similar to what I referred to as approach 1, with a few notable
>>> differences:
>>>
>>>   - The host maintains a set of virtual keyslots per virtio-blk device.
>>>   - Physical keyslots are not statically partitioned.
>>>   - The virtual-to-physical slot mapping is managed as part of the virtio-blk
>>>     device implementation rather than being tied to a VM-wide slot table.
>>>
>>> With this design, I believe there would be two VM exits associated with
>>> encrypted I/O:
>>>   1. Key programming
>>>      Before encrypted I/O can be submitted, the guest needs to program a key
>>>      into a virtual keyslot:
>>>      
>>>      Guest virtio-blk driver
>>>          -> VM exit
>>>          -> virtio-blk backend (e.g. QEMU)
>>>          -> ioctl
>>>          -> host virtio-blk proxy driver
>>>             (stores the blk_crypto_key in a virtual keyslot)
>>>
>>>   2. Encrypted I/O submission
>>>      The I/O request carries the virtual keyslot number and DUN:
>>>
>>>      Guest virtio-blk driver
>>>         -> VM exit
>>>         -> virtio-blk backend (e.g. QEMU)
>>>         -> ioctl
>>>         -> host virtio-blk proxy driver
>>>            (looks up the blk_crypto_key associated with the virtual keyslot
>>>             and submits I/O using bio_crypt_set_ctx())
>>
>> Supplementation.
>> This may be a potential security concern. The host virtio-blk proxy driver exposes
>> API for key programming, the input parameters are blk_crypto key and virtual
>> slot. If a malicious program replace the key in a specific virtual slot between
>> key program call and I/O submission via this API, the data would be encrypted
>> by the unintentional key.
> 
> If the virtual key slots in the uapi are per file descriptor rather than
> per inode, then other programs cannot interfere with each other's
> virtual key slots. Each program gets its own virtual key space when it
> opens a file descriptor. Sharing is only possible by inheriting or
> passing a file descriptor to another process and that's good for
> security.
> 

Thanks for your advice!
That sounds a good way. I'll check if this mechanism can be used in the above
design properly.

> Stefan

On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
> 
> 
> On 8/28/2026 2:42 AM, Eric Biggers wrote:
> > On Thu, Aug 27, 2026 at 09:07:09AM -0700, Linlin Zhang wrote:
> >> From: linlzhan <linlin.zhang@oss.qualcomm.com>
> >>
> >> Current virtio-blk does not provide a mechanism for a guest to
> >> program hardware keys or submit encrypted I/O using pre-programmed
> >> keyslots.  It drops the crypto context when issuing a bio request
> >> to the virtio-blk queue, preventing inline-encryption-based FBE
> >> on virtio block devices.
> >>
> >> This series enables File-Based Encryption in guest VMs on Qualcomm
> >> GVM platforms where the ICE inline encryption hardware is shared
> >> between the host and guests.  In this environment the guest kernel
> >> has no access to the ICE hardware directly; it supplies a virtual
> >> keyslot index and data unit number with each encrypted I/O request
> >> via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
> >> virtual slot to a physical ICE keyslot and submit the bio — without
> >> transferring raw key material across the VM boundary.
> > 
> > This seems to be designed incorrectly by not making virtio-blk itself
> > support key programming and eviction.  That complicates things
> > significantly by then having to handle the key programming and eviction
> > out-of-band using Qualcomm-specific SCM calls.  It also means that
> > adding other implementations of this would be very difficult.
> > 
> > There are some claims that not transmitting keys across the VM boundary
> > is desirable.  But that doesn't seem meaningful, given that all the I/O
> > is transmitted across that boundary in plaintext anyway, and also it
> > seems that hardware-wrapped keys will be supported too.
> > 
> > Please make virtio-blk support the key programming, eviction, and
> > HW-wrapped key management operations that are needed for this to work.
> > 
> > - Eric
> 
> Thanks for your comments!
> 
> Not making virtio-blk itself support key programming and eviction is
> something done deliberately. Based on that HW-wrapped key management
> operations are also handled in the out-of-band path.
> 
> There are bellow 2 approaches I investigated to let virtio-blk programming
> the key.
> 
>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
> 
>      The guest's block crypto profile manages the keyslot in virtual slot
>      format in this scenario.
> 
>       - block crypto key and virt_slot index it passed to the hypervisor's
>        (EL2) device emulation (QEMU, using QEMU in the following) which
>        runs in userspace of the host. Besides of transferring the virtual
>        slot to the physical slot, a programming block crypto key UAPI need
>        be added. Follow current blk-crypto design, it may be like
>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>        allows userspace process a key into a key slot.
> 
>      - For key eviction, it's similar to above key programming handling, also
>        need a key eviction in blk IOCTLs, but leads to the security risk
>        that allow userspace client to evict a key in a key slot.

Yes, userspace shouldn't have access to the entire physical key slot
range. The host kernel or other VMs may need key slots and an untrusted
QEMU process must not be able to modify those key slots or use them for
I/O.

The uapi design should include a solution for this. For example, there
could be an ioctl like BLKCRYPTOSEALKEYS that permanently restricts the
key range on this block device file descriptor and cannot be undone.
Libvirt or other management tooling would call this ioctl with
CAP_SYS_ADMIN before passing the file descriptor when launching QEMU
without CAP_SYS_ADMIN. This prevents QEMU from ever having access to the
full physical key range.

Just an idea. Those familiar with blk-crypto may have a better one. It
seems likely that we can find a design that matches the level of
security of the out-of-band approach.

> 
>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>     I/O.
> 
>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>      key and evict key interfaces.
> 
>      The guest's block crypto profile doesn't manage keyslot for the guest,
>      the host's block crypto profile manages keyslot for both the guest and
>      the host. The trigger of key programming operation is moved from the
>      guest to the host.
> 
>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>          and DUN are appended to the virtblk request during IO, a little
>          high payload.
> 
>          The backend parses the crypto message in the virtio queue and
>          construct a block crypto key and DUN for the bio_crypto_ctx
>          set to the BIO. So that the IO flow in the host can program
>          the key. 
> 
>          The question is that the blk-crypto-profile distinguishs the
>          block crypto key via the key's address. But the host has
>          different key addresses for the programming and eviction key
>          operations of the same block crypto key from GVM, because the
>          key is re-constructed in the host for the key program and
>          eviction operations. 
> 
>          To fix it, the approach I thought is maintaining a new key
>          hash table in the backend, and comparing the block crypto key
>          content and DUN parsed from virtio queue with that in the key
>          hash table. 
>          My major concern is that this need keep the keys synchronization
>          b/w this new hash table and the blk-crypto-profile's hash table
>          carefully, avoiding that key is still present in the
>          blk-crypto-profile's hash table, but removed in backend hash
>          table. Another point is that the whole block crypto key and
>          DUN are appended into virtio block request per crypto I/O.

This sounds like an approach that skips key programming and instead
sends the keys along with each I/O request. The device implementation is
responsible for managing key slots on the physical ICE. My main concern
with this would be whether the blk_crypto_ll_ops semantics can be
faithfully replicated (e.g. error reporting) without explicit key
programming operations.

Stefan

> 
>        - For key eviction, adding a key eviction in blk IOCTLs allows
>          userspace client to evict a key in a key slot. I thought this
>          is a security concern.
> 
> 
>      This option doesn't need map virt_slot to physical one.
> 
> 
> Taking all the above into account, I made a compromise to implement
> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
> blk only need focus on the data path. I agree that it is complex than
> the second option mentioned in the above, but small payload (only
> virt_slot, DUN, DUSize) in the virtio block request and no security
> risk of key eviction from userspace.
> 
> 
> I would like to hear your thoughts about the above and am appreciated
> if you could share your insights about the design of inline
> encryption in virtio block.
> 
> Regards,
> Linlin
> 
> 
> 

On 9/1/2026 4:41 AM, Stefan Hajnoczi wrote:
> On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
>>
>>
>> On 8/28/2026 2:42 AM, Eric Biggers wrote:
>>> On Thu, Aug 27, 2026 at 09:07:09AM -0700, Linlin Zhang wrote:
>>>> From: linlzhan <linlin.zhang@oss.qualcomm.com>
>>>>
>>>> Current virtio-blk does not provide a mechanism for a guest to
>>>> program hardware keys or submit encrypted I/O using pre-programmed
>>>> keyslots.  It drops the crypto context when issuing a bio request
>>>> to the virtio-blk queue, preventing inline-encryption-based FBE
>>>> on virtio block devices.
>>>>
>>>> This series enables File-Based Encryption in guest VMs on Qualcomm
>>>> GVM platforms where the ICE inline encryption hardware is shared
>>>> between the host and guests.  In this environment the guest kernel
>>>> has no access to the ICE hardware directly; it supplies a virtual
>>>> keyslot index and data unit number with each encrypted I/O request
>>>> via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
>>>> virtual slot to a physical ICE keyslot and submit the bio — without
>>>> transferring raw key material across the VM boundary.
>>>
>>> This seems to be designed incorrectly by not making virtio-blk itself
>>> support key programming and eviction.  That complicates things
>>> significantly by then having to handle the key programming and eviction
>>> out-of-band using Qualcomm-specific SCM calls.  It also means that
>>> adding other implementations of this would be very difficult.
>>>
>>> There are some claims that not transmitting keys across the VM boundary
>>> is desirable.  But that doesn't seem meaningful, given that all the I/O
>>> is transmitted across that boundary in plaintext anyway, and also it
>>> seems that hardware-wrapped keys will be supported too.
>>>
>>> Please make virtio-blk support the key programming, eviction, and
>>> HW-wrapped key management operations that are needed for this to work.
>>>
>>> - Eric
>>
>> Thanks for your comments!
>>
>> Not making virtio-blk itself support key programming and eviction is
>> something done deliberately. Based on that HW-wrapped key management
>> operations are also handled in the out-of-band path.
>>
>> There are bellow 2 approaches I investigated to let virtio-blk programming
>> the key.
>>
>>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
>>
>>      The guest's block crypto profile manages the keyslot in virtual slot
>>      format in this scenario.
>>
>>       - block crypto key and virt_slot index it passed to the hypervisor's
>>        (EL2) device emulation (QEMU, using QEMU in the following) which
>>        runs in userspace of the host. Besides of transferring the virtual
>>        slot to the physical slot, a programming block crypto key UAPI need
>>        be added. Follow current blk-crypto design, it may be like
>>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>>        allows userspace process a key into a key slot.
>>
>>      - For key eviction, it's similar to above key programming handling, also
>>        need a key eviction in blk IOCTLs, but leads to the security risk
>>        that allow userspace client to evict a key in a key slot.
> 
> Yes, userspace shouldn't have access to the entire physical key slot
> range. The host kernel or other VMs may need key slots and an untrusted
> QEMU process must not be able to modify those key slots or use them for
> I/O.
> 
> The uapi design should include a solution for this. For example, there
> could be an ioctl like BLKCRYPTOSEALKEYS that permanently restricts the
> key range on this block device file descriptor and cannot be undone.
> Libvirt or other management tooling would call this ioctl with
> CAP_SYS_ADMIN before passing the file descriptor when launching QEMU
> without CAP_SYS_ADMIN. This prevents QEMU from ever having access to the
> full physical key range.
> 
> Just an idea. Those familiar with blk-crypto may have a better one. It
> seems likely that we can find a design that matches the level of
> security of the out-of-band approach.

Thanks for your comment!

Eric mentioned a proposal that each virtio block device has its own *virtual*
keyslots in the host, which is not static partitioning of the physical keyslots.
I have concerns about the new key programing UAPI and 2 VM exits per encrypted
I/O. I'll confirm with Eric if that needs be considered and if we need pass
through the crypto to the host blk-crypto-profile (which means the max slots
of blk-crypto-profile in the guest is 0, and virtio block driver only doesn't
implement the key program interface in blk_crypto_ll_ops).

> 
>>
>>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>>     I/O.
>>
>>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>>      key and evict key interfaces.
>>
>>      The guest's block crypto profile doesn't manage keyslot for the guest,
>>      the host's block crypto profile manages keyslot for both the guest and
>>      the host. The trigger of key programming operation is moved from the
>>      guest to the host.
>>
>>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>>          and DUN are appended to the virtblk request during IO, a little
>>          high payload.
>>
>>          The backend parses the crypto message in the virtio queue and
>>          construct a block crypto key and DUN for the bio_crypto_ctx
>>          set to the BIO. So that the IO flow in the host can program
>>          the key. 
>>
>>          The question is that the blk-crypto-profile distinguishs the
>>          block crypto key via the key's address. But the host has
>>          different key addresses for the programming and eviction key
>>          operations of the same block crypto key from GVM, because the
>>          key is re-constructed in the host for the key program and
>>          eviction operations. 
>>
>>          To fix it, the approach I thought is maintaining a new key
>>          hash table in the backend, and comparing the block crypto key
>>          content and DUN parsed from virtio queue with that in the key
>>          hash table. 
>>          My major concern is that this need keep the keys synchronization
>>          b/w this new hash table and the blk-crypto-profile's hash table
>>          carefully, avoiding that key is still present in the
>>          blk-crypto-profile's hash table, but removed in backend hash
>>          table. Another point is that the whole block crypto key and
>>          DUN are appended into virtio block request per crypto I/O.
> 
> This sounds like an approach that skips key programming and instead
> sends the keys along with each I/O request. The device implementation is
> responsible for managing key slots on the physical ICE. My main concern
> with this would be whether the blk_crypto_ll_ops semantics can be
> faithfully replicated (e.g. error reporting) without explicit key
> programming operations.
> 
> Stefan

You're right. When the virtio-blk device processes encrypted I/O, it
would be responsible for extracting the blk_crypto_key and DUN from the
virtqueue and attaching them to the bio (via bio_crypt_set_ctx()). The
existing keyslot management logic in the block layer would then handle
physical keyslot allocation and waiting as needed, so the rest of the
blk-crypto infrastructure should continue to work unchanged.

Regarding your concern, there is already a somewhat similar
implementation in the device-mapper layer (
dm_table_construct_crypto_profile() implements a passthrough
blk_crypto_ll_ops profile. See
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/md/dm-table.c?h=v7.3-rc1
).

Based on my current understanding, I don't see any obvious
architectural issues with this approach. However, I may be missing
something, so I'd appreciate any feedback if you see flaws in the
design or potential problems that should be taken into account.
> 
>>
>>        - For key eviction, adding a key eviction in blk IOCTLs allows
>>          userspace client to evict a key in a key slot. I thought this
>>          is a security concern.
>>
>>
>>      This option doesn't need map virt_slot to physical one.
>>
>>
>> Taking all the above into account, I made a compromise to implement
>> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
>> blk only need focus on the data path. I agree that it is complex than
>> the second option mentioned in the above, but small payload (only
>> virt_slot, DUN, DUSize) in the virtio block request and no security
>> risk of key eviction from userspace.
>>
>>
>> I would like to hear your thoughts about the above and am appreciated
>> if you could share your insights about the design of inline
>> encryption in virtio block.
>>
>> Regards,
>> Linlin
>>
>>
>>

On Tue, Sep 01, 2026 at 05:21:53PM +0800, Linlin Zhang wrote:
> 
> 
> On 9/1/2026 4:41 AM, Stefan Hajnoczi wrote:
> > On Fri, Aug 28, 2026 at 11:37:57PM +0800, Linlin Zhang wrote:
> >>
> >>
> >> On 8/28/2026 2:42 AM, Eric Biggers wrote:
> >>> On Thu, Aug 27, 2026 at 09:07:09AM -0700, Linlin Zhang wrote:
> >>>> From: linlzhan <linlin.zhang@oss.qualcomm.com>
> >>>>
> >>>> Current virtio-blk does not provide a mechanism for a guest to
> >>>> program hardware keys or submit encrypted I/O using pre-programmed
> >>>> keyslots.  It drops the crypto context when issuing a bio request
> >>>> to the virtio-blk queue, preventing inline-encryption-based FBE
> >>>> on virtio block devices.
> >>>>
> >>>> This series enables File-Based Encryption in guest VMs on Qualcomm
> >>>> GVM platforms where the ICE inline encryption hardware is shared
> >>>> between the host and guests.  In this environment the guest kernel
> >>>> has no access to the ICE hardware directly; it supplies a virtual
> >>>> keyslot index and data unit number with each encrypted I/O request
> >>>> via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
> >>>> virtual slot to a physical ICE keyslot and submit the bio — without
> >>>> transferring raw key material across the VM boundary.
> >>>
> >>> This seems to be designed incorrectly by not making virtio-blk itself
> >>> support key programming and eviction.  That complicates things
> >>> significantly by then having to handle the key programming and eviction
> >>> out-of-band using Qualcomm-specific SCM calls.  It also means that
> >>> adding other implementations of this would be very difficult.
> >>>
> >>> There are some claims that not transmitting keys across the VM boundary
> >>> is desirable.  But that doesn't seem meaningful, given that all the I/O
> >>> is transmitted across that boundary in plaintext anyway, and also it
> >>> seems that hardware-wrapped keys will be supported too.
> >>>
> >>> Please make virtio-blk support the key programming, eviction, and
> >>> HW-wrapped key management operations that are needed for this to work.
> >>>
> >>> - Eric
> >>
> >> Thanks for your comments!
> >>
> >> Not making virtio-blk itself support key programming and eviction is
> >> something done deliberately. Based on that HW-wrapped key management
> >> operations are also handled in the out-of-band path.
> >>
> >> There are bellow 2 approaches I investigated to let virtio-blk programming
> >> the key.
> >>
> >>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
> >>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
> >>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
> >>
> >>      The guest's block crypto profile manages the keyslot in virtual slot
> >>      format in this scenario.
> >>
> >>       - block crypto key and virt_slot index it passed to the hypervisor's
> >>        (EL2) device emulation (QEMU, using QEMU in the following) which
> >>        runs in userspace of the host. Besides of transferring the virtual
> >>        slot to the physical slot, a programming block crypto key UAPI need
> >>        be added. Follow current blk-crypto design, it may be like
> >>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
> >>        allows userspace process a key into a key slot.
> >>
> >>      - For key eviction, it's similar to above key programming handling, also
> >>        need a key eviction in blk IOCTLs, but leads to the security risk
> >>        that allow userspace client to evict a key in a key slot.
> > 
> > Yes, userspace shouldn't have access to the entire physical key slot
> > range. The host kernel or other VMs may need key slots and an untrusted
> > QEMU process must not be able to modify those key slots or use them for
> > I/O.
> > 
> > The uapi design should include a solution for this. For example, there
> > could be an ioctl like BLKCRYPTOSEALKEYS that permanently restricts the
> > key range on this block device file descriptor and cannot be undone.
> > Libvirt or other management tooling would call this ioctl with
> > CAP_SYS_ADMIN before passing the file descriptor when launching QEMU
> > without CAP_SYS_ADMIN. This prevents QEMU from ever having access to the
> > full physical key range.
> > 
> > Just an idea. Those familiar with blk-crypto may have a better one. It
> > seems likely that we can find a design that matches the level of
> > security of the out-of-band approach.
> 
> Thanks for your comment!
> 
> Eric mentioned a proposal that each virtio block device has its own *virtual*
> keyslots in the host, which is not static partitioning of the physical keyslots.
> I have concerns about the new key programing UAPI and 2 VM exits per encrypted
> I/O. I'll confirm with Eric if that needs be considered and if we need pass
> through the crypto to the host blk-crypto-profile (which means the max slots
> of blk-crypto-profile in the guest is 0, and virtio block driver only doesn't
> implement the key program interface in blk_crypto_ll_ops).
> 
> > 
> >>
> >>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
> >>     I/O.
> >>
> >>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
> >>      key and evict key interfaces.
> >>
> >>      The guest's block crypto profile doesn't manage keyslot for the guest,
> >>      the host's block crypto profile manages keyslot for both the guest and
> >>      the host. The trigger of key programming operation is moved from the
> >>      guest to the host.
> >>
> >>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
> >>          and DUN are appended to the virtblk request during IO, a little
> >>          high payload.
> >>
> >>          The backend parses the crypto message in the virtio queue and
> >>          construct a block crypto key and DUN for the bio_crypto_ctx
> >>          set to the BIO. So that the IO flow in the host can program
> >>          the key. 
> >>
> >>          The question is that the blk-crypto-profile distinguishs the
> >>          block crypto key via the key's address. But the host has
> >>          different key addresses for the programming and eviction key
> >>          operations of the same block crypto key from GVM, because the
> >>          key is re-constructed in the host for the key program and
> >>          eviction operations. 
> >>
> >>          To fix it, the approach I thought is maintaining a new key
> >>          hash table in the backend, and comparing the block crypto key
> >>          content and DUN parsed from virtio queue with that in the key
> >>          hash table. 
> >>          My major concern is that this need keep the keys synchronization
> >>          b/w this new hash table and the blk-crypto-profile's hash table
> >>          carefully, avoiding that key is still present in the
> >>          blk-crypto-profile's hash table, but removed in backend hash
> >>          table. Another point is that the whole block crypto key and
> >>          DUN are appended into virtio block request per crypto I/O.
> > 
> > This sounds like an approach that skips key programming and instead
> > sends the keys along with each I/O request. The device implementation is
> > responsible for managing key slots on the physical ICE. My main concern
> > with this would be whether the blk_crypto_ll_ops semantics can be
> > faithfully replicated (e.g. error reporting) without explicit key
> > programming operations.
> > 
> > Stefan
> 
> You're right. When the virtio-blk device processes encrypted I/O, it
> would be responsible for extracting the blk_crypto_key and DUN from the
> virtqueue and attaching them to the bio (via bio_crypt_set_ctx()). The
> existing keyslot management logic in the block layer would then handle
> physical keyslot allocation and waiting as needed, so the rest of the
> blk-crypto infrastructure should continue to work unchanged.
> 
> Regarding your concern, there is already a somewhat similar
> implementation in the device-mapper layer (
> dm_table_construct_crypto_profile() implements a passthrough
> blk_crypto_ll_ops profile. See
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/md/dm-table.c?h=v7.3-rc1
> ).
> 
> Based on my current understanding, I don't see any obvious
> architectural issues with this approach. However, I may be missing
> something, so I'd appreciate any feedback if you see flaws in the
> design or potential problems that should be taken into account.

I don't have enough blk-crypto knowledge to give good feedback on the
details of this approach, but Eric and others could help with that.

Stefan

On 8/28/2026 11:37 PM, Linlin Zhang wrote:
> 
> 
> On 8/28/2026 2:42 AM, Eric Biggers wrote:
>> On Thu, Aug 27, 2026 at 09:07:09AM -0700, Linlin Zhang wrote:
>>> From: linlzhan <linlin.zhang@oss.qualcomm.com>
>>>
>>> Current virtio-blk does not provide a mechanism for a guest to
>>> program hardware keys or submit encrypted I/O using pre-programmed
>>> keyslots.  It drops the crypto context when issuing a bio request
>>> to the virtio-blk queue, preventing inline-encryption-based FBE
>>> on virtio block devices.
>>>
>>> This series enables File-Based Encryption in guest VMs on Qualcomm
>>> GVM platforms where the ICE inline encryption hardware is shared
>>> between the host and guests.  In this environment the guest kernel
>>> has no access to the ICE hardware directly; it supplies a virtual
>>> keyslot index and data unit number with each encrypted I/O request
>>> via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
>>> virtual slot to a physical ICE keyslot and submit the bio — without
>>> transferring raw key material across the VM boundary.
>>
>> This seems to be designed incorrectly by not making virtio-blk itself
>> support key programming and eviction.  That complicates things
>> significantly by then having to handle the key programming and eviction
>> out-of-band using Qualcomm-specific SCM calls.  It also means that
>> adding other implementations of this would be very difficult.
>>
>> There are some claims that not transmitting keys across the VM boundary
>> is desirable.  But that doesn't seem meaningful, given that all the I/O
>> is transmitted across that boundary in plaintext anyway, and also it
>> seems that hardware-wrapped keys will be supported too.
>>
>> Please make virtio-blk support the key programming, eviction, and
>> HW-wrapped key management operations that are needed for this to work.
>>
>> - Eric
> 
> Thanks for your comments!
> 
> Not making virtio-blk itself support key programming and eviction is
> something done deliberately. Based on that HW-wrapped key management
> operations are also handled in the out-of-band path.
> 
> There are bellow 2 approaches I investigated to let virtio-blk programming
> the key.
> 
>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
> 
>      The guest's block crypto profile manages the keyslot in virtual slot
>      format in this scenario.
> 
>       - block crypto key and virt_slot index it passed to the hypervisor's
>        (EL2) device emulation (QEMU, using QEMU in the following) which
>        runs in userspace of the host. Besides of transferring the virtual
>        slot to the physical slot, a programming block crypto key UAPI need
>        be added. Follow current blk-crypto design, it may be like
>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>        allows userspace process a key into a key slot.
> 
>      - For key eviction, it's similar to above key programming handling, also
>        need a key eviction in blk IOCTLs, but leads to the security risk
>        that allow userspace client to evict a key in a key slot.
> 
>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>     I/O.
> 
>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>      key and evict key interfaces.
> 
>      The guest's block crypto profile doesn't manage keyslot for the guest,
>      the host's block crypto profile manages keyslot for both the guest and
>      the host. The trigger of key programming operation is moved from the
>      guest to the host.
> 
>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>          and DUN are appended to the virtblk request during IO, a little
>          high payload.
> 
>          The backend parses the crypto message in the virtio queue and
>          construct a block crypto key and DUN for the bio_crypto_ctx
>          set to the BIO. So that the IO flow in the host can program
>          the key. 
> 
>          The question is that the blk-crypto-profile distinguishs the
>          block crypto key via the key's address. But the host has
>          different key addresses for the programming and eviction key
>          operations of the same block crypto key from GVM, because the
>          key is re-constructed in the host for the key program and
>          eviction operations. 
> 
>          To fix it, the approach I thought is maintaining a new key
>          hash table in the backend, and comparing the block crypto key
>          content and DUN parsed from virtio queue with that in the key
>          hash table. 
>          My major concern is that this need keep the keys synchronization
>          b/w this new hash table and the blk-crypto-profile's hash table
>          carefully, avoiding that key is still present in the
>          blk-crypto-profile's hash table, but removed in backend hash
>          table. Another point is that the whole block crypto key and
>          DUN are appended into virtio block request per crypto I/O.
> 
>        - For key eviction, adding a key eviction in blk IOCTLs allows
>          userspace client to evict a key in a key slot. I thought this
>          is a security concern.

Correction.

The key eviction operation is also moved from the Guest's blk_crypto_profile
to the Host's blk_crypto_profile. As a result, the entire block crypto key
(including the key size, key material, and blk_crypto_config) is appended to
the virtblk request. The backend parses the supplied blk_crypto_key, looks
it up in its key hash table, and can then invoke blk_crypto_evict_key() to
evict the corresponding key from the ICE keyslot. Unlike Approach 1, Approach
2 does not allow the guest to specify the keyslot to be evicted through the
block ioctl interface.

I'm appreciated if you can share your insights and thoughts about these 3
approaches.

> 
> 
>      This option doesn't need map virt_slot to physical one.
> 
> 
> Taking all the above into account, I made a compromise to implement
> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
> blk only need focus on the data path. I agree that it is complex than
> the second option mentioned in the above, but small payload (only
> virt_slot, DUN, DUSize) in the virtio block request and no security
> risk of key eviction from userspace.
> 
> 
> I would like to hear your thoughts about the above and am appreciated
> if you could share your insights about the design of inline
> encryption in virtio block.
> 
> Regards,
> Linlin
> 
> 
> 

On 8/28/2026 11:37 PM, Linlin Zhang wrote:
> 
> 
> On 8/28/2026 2:42 AM, Eric Biggers wrote:
>> On Thu, Aug 27, 2026 at 09:07:09AM -0700, Linlin Zhang wrote:
>>> From: linlzhan <linlin.zhang@oss.qualcomm.com>
>>>
>>> Current virtio-blk does not provide a mechanism for a guest to
>>> program hardware keys or submit encrypted I/O using pre-programmed
>>> keyslots.  It drops the crypto context when issuing a bio request
>>> to the virtio-blk queue, preventing inline-encryption-based FBE
>>> on virtio block devices.
>>>
>>> This series enables File-Based Encryption in guest VMs on Qualcomm
>>> GVM platforms where the ICE inline encryption hardware is shared
>>> between the host and guests.  In this environment the guest kernel
>>> has no access to the ICE hardware directly; it supplies a virtual
>>> keyslot index and data unit number with each encrypted I/O request
>>> via VIRTIO_BLK_F_INLINE_ENCRYPTION, and the host must translate the
>>> virtual slot to a physical ICE keyslot and submit the bio — without
>>> transferring raw key material across the VM boundary.
>>
>> This seems to be designed incorrectly by not making virtio-blk itself
>> support key programming and eviction.  That complicates things
>> significantly by then having to handle the key programming and eviction
>> out-of-band using Qualcomm-specific SCM calls.  It also means that
>> adding other implementations of this would be very difficult.
>>
>> There are some claims that not transmitting keys across the VM boundary
>> is desirable.  But that doesn't seem meaningful, given that all the I/O
>> is transmitted across that boundary in plaintext anyway, and also it
>> seems that hardware-wrapped keys will be supported too.
>>
>> Please make virtio-blk support the key programming, eviction, and
>> HW-wrapped key management operations that are needed for this to work.
>>
>> - Eric
> 
> Thanks for your comments!
> 
> Not making virtio-blk itself support key programming and eviction is
> something done deliberately. Based on that HW-wrapped key management
> operations are also handled in the out-of-band path.
> 
> There are bellow 2 approaches I investigated to let virtio-blk programming
> the key.
> 
>   1. virtio_blk implements blk_crypto_ll_ops interfaces, including program
>      key and evict key interfaces.(Same to 'the virtio-blk interface standardized
>      blk_crypto_ll_ops requests' mentioned by Stefan in the virtio SPEC thread)
> 
>      The guest's block crypto profile manages the keyslot in virtual slot
>      format in this scenario.
> 
>       - block crypto key and virt_slot index it passed to the hypervisor's
>        (EL2) device emulation (QEMU, using QEMU in the following) which
>        runs in userspace of the host. Besides of transferring the virtual
>        slot to the physical slot, a programming block crypto key UAPI need
>        be added. Follow current blk-crypto design, it may be like
>        BLKCRYPTOGENERATEKEY. I thought this results in a security risk that
>        allows userspace process a key into a key slot.
> 
>      - For key eviction, it's similar to above key programming handling, also
>        need a key eviction in blk IOCTLs, but leads to the security risk
>        that allow userspace client to evict a key in a key slot.
> 
>     virt_slot, DUN and DUSize is appended to virtblk request during crypto
>     I/O.
> 
>   2. virtio_blk implements blk_crypto_ll_ops interfaces, excluding program
>      key and evict key interfaces.
> 
>      The guest's block crypto profile doesn't manage keyslot for the guest,
>      the host's block crypto profile manages keyslot for both the guest and
>      the host. The trigger of key programming operation is moved from the
>      guest to the host.
> 
>        - The whole block crypto key (key size, key bytes, blk_crypto_config)
>          and DUN are appended to the virtblk request during IO, a little
>          high payload.
> 
>          The backend parses the crypto message in the virtio queue and
>          construct a block crypto key and DUN for the bio_crypto_ctx
>          set to the BIO. So that the IO flow in the host can program
>          the key. 
> 
>          The question is that the blk-crypto-profile distinguishs the
>          block crypto key via the key's address. But the host has
>          different key addresses for the programming and eviction key
>          operations of the same block crypto key from GVM, because the
>          key is re-constructed in the host for the key program and
>          eviction operations. 
> 
>          To fix it, the approach I thought is maintaining a new key
>          hash table in the backend, and comparing the block crypto key
>          content and DUN parsed from virtio queue with that in the key
>          hash table. 
>          My major concern is that this need keep the keys synchronization
>          b/w this new hash table and the blk-crypto-profile's hash table
>          carefully, avoiding that key is still present in the
>          blk-crypto-profile's hash table, but removed in backend hash
>          table. Another point is that the whole block crypto key and
>          DUN are appended into virtio block request per crypto I/O.
> 
>        - For key eviction, adding a key eviction in blk IOCTLs allows
>          userspace client to evict a key in a key slot. I thought this
>          is a security concern.
> 

Do you have any suggestion about the key eviction operation? Could this
key eviction request triggered in QEMU be accepted? Would you please
help share insights about how to handle it?

Thanks a lot!

> 
>      This option doesn't need map virt_slot to physical one.
> 
> 
> Taking all the above into account, I made a compromise to implement
> blk_crypto_ll_ops interfaces in a out-of-band path, which lets the virtio
> blk only need focus on the data path. I agree that it is complex than
> the second option mentioned in the above, but small payload (only
> virt_slot, DUN, DUSize) in the virtio block request and no security
> risk of key eviction from userspace.
> 
> 
> I would like to hear your thoughts about the above and am appreciated
> if you could share your insights about the design of inline
> encryption in virtio block.
> 
> Regards,
> Linlin
> 
> 
>