Document feature UBLK_F_BATCH_IO.

Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 Documentation/block/ublk.rst | 60 +++++++++++++++++++++++++++++++++---
 1 file changed, 56 insertions(+), 4 deletions(-)

diff --git a/Documentation/block/ublk.rst b/Documentation/block/ublk.rst
index 8c4030bcabb6..09a5604f8e10 100644
--- a/Documentation/block/ublk.rst
+++ b/Documentation/block/ublk.rst
@@ -260,9 +260,12 @@ The following IO commands are communicated via io_uring passthrough command,
 and each command is only for forwarding the IO and committing the result
 with specified IO tag in the command data:
 
-- ``UBLK_IO_FETCH_REQ``
+Traditional Per-I/O Commands
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-  Sent from the server IO pthread for fetching future incoming IO requests
+- ``UBLK_U_IO_FETCH_REQ``
+
+  Sent from the server I/O pthread for fetching future incoming I/O requests
   destined to ``/dev/ublkb*``. This command is sent only once from the server
   IO pthread for ublk driver to setup IO forward environment.
 
@@ -278,7 +281,7 @@ with specified IO tag in the command data:
   supported by the driver, daemons must be per-queue instead - i.e. all I/Os
   associated to a single qid must be handled by the same task.
 
-- ``UBLK_IO_COMMIT_AND_FETCH_REQ``
+- ``UBLK_U_IO_COMMIT_AND_FETCH_REQ``
 
   When an IO request is destined to ``/dev/ublkb*``, the driver stores
   the IO's ``ublksrv_io_desc`` to the specified mapped area; then the
@@ -293,7 +296,7 @@ with specified IO tag in the command data:
   requests with the same IO tag. That is, ``UBLK_IO_COMMIT_AND_FETCH_REQ``
   is reused for both fetching request and committing back IO result.
 
-- ``UBLK_IO_NEED_GET_DATA``
+- ``UBLK_U_IO_NEED_GET_DATA``
 
   With ``UBLK_F_NEED_GET_DATA`` enabled, the WRITE request will be firstly
   issued to ublk server without data copy. Then, IO backend of ublk server
@@ -322,6 +325,55 @@ with specified IO tag in the command data:
   ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy
   the server buffer (pages) read to the IO request pages.
 
+Batch I/O Commands (UBLK_F_BATCH_IO)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``UBLK_F_BATCH_IO`` feature provides an alternative high-performance
+I/O handling model that replaces the traditional per-I/O commands with
+per-queue batch commands. This significantly reduces communication overhead
+and enables better load balancing across multiple server tasks.
+
+Key differences from traditional mode:
+
+- **Per-queue vs Per-I/O**: Commands operate on queues rather than individual I/Os
+- **Batch processing**: Multiple I/Os are handled in single operations
+- **Multishot commands**: Use io_uring multishot for reduced submission overhead
+- **Flexible task assignment**: Any task can handle any I/O (no per-I/O daemons)
+- **Better load balancing**: Tasks can adjust their workload dynamically
+
+Batch I/O Commands:
+
+- ``UBLK_U_IO_PREP_IO_CMDS``
+
+  Prepares multiple I/O commands in batch. The server provides a buffer
+  containing multiple I/O descriptors that will be processed together.
+  This reduces the number of individual command submissions required.
+
+- ``UBLK_U_IO_COMMIT_IO_CMDS``
+
+  Commits results for multiple I/O operations in batch. The server provides
+  a buffer containing the results of multiple completed I/Os, allowing
+  efficient bulk completion of requests.
+
+- ``UBLK_U_IO_FETCH_IO_CMDS``
+
+  **Multishot command** for fetching I/O commands in batch. This is the key
+  command that enables high-performance batch processing:
+
+  * Uses io_uring multishot capability for reduced submission overhead
+  * Single command can fetch multiple I/O requests over time
+  * Buffer size determines maximum batch size per operation
+  * Multiple fetch commands can be submitted for load balancing
+  * Only one fetch command is active at any time per queue
+  * Supports dynamic load balancing across multiple server tasks
+
+  Each task can submit ``UBLK_U_IO_FETCH_IO_CMDS`` with different buffer
+  sizes to control how much work it handles. This enables sophisticated
+  load balancing strategies in multi-threaded servers.
+
+Migration: Applications using traditional commands (``UBLK_U_IO_FETCH_REQ``,
+``UBLK_U_IO_COMMIT_AND_FETCH_REQ``) cannot use batch mode simultaneously.
+
 Zero copy
 ---------
 
-- 
2.47.0

On Thu, Nov 20, 2025 at 6:00 PM Ming Lei <ming.lei@redhat.com> wrote:
>
> Document feature UBLK_F_BATCH_IO.
>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
>  Documentation/block/ublk.rst | 60 +++++++++++++++++++++++++++++++++---
>  1 file changed, 56 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/block/ublk.rst b/Documentation/block/ublk.rst
> index 8c4030bcabb6..09a5604f8e10 100644
> --- a/Documentation/block/ublk.rst
> +++ b/Documentation/block/ublk.rst
> @@ -260,9 +260,12 @@ The following IO commands are communicated via io_uring passthrough command,
>  and each command is only for forwarding the IO and committing the result
>  with specified IO tag in the command data:
>
> -- ``UBLK_IO_FETCH_REQ``
> +Traditional Per-I/O Commands
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> -  Sent from the server IO pthread for fetching future incoming IO requests
> +- ``UBLK_U_IO_FETCH_REQ``
> +
> +  Sent from the server I/O pthread for fetching future incoming I/O requests
>    destined to ``/dev/ublkb*``. This command is sent only once from the server
>    IO pthread for ublk driver to setup IO forward environment.
>
> @@ -278,7 +281,7 @@ with specified IO tag in the command data:
>    supported by the driver, daemons must be per-queue instead - i.e. all I/Os
>    associated to a single qid must be handled by the same task.
>
> -- ``UBLK_IO_COMMIT_AND_FETCH_REQ``
> +- ``UBLK_U_IO_COMMIT_AND_FETCH_REQ``
>
>    When an IO request is destined to ``/dev/ublkb*``, the driver stores
>    the IO's ``ublksrv_io_desc`` to the specified mapped area; then the
> @@ -293,7 +296,7 @@ with specified IO tag in the command data:
>    requests with the same IO tag. That is, ``UBLK_IO_COMMIT_AND_FETCH_REQ``
>    is reused for both fetching request and committing back IO result.
>
> -- ``UBLK_IO_NEED_GET_DATA``
> +- ``UBLK_U_IO_NEED_GET_DATA``
>
>    With ``UBLK_F_NEED_GET_DATA`` enabled, the WRITE request will be firstly
>    issued to ublk server without data copy. Then, IO backend of ublk server
> @@ -322,6 +325,55 @@ with specified IO tag in the command data:
>    ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy
>    the server buffer (pages) read to the IO request pages.
>
> +Batch I/O Commands (UBLK_F_BATCH_IO)
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``UBLK_F_BATCH_IO`` feature provides an alternative high-performance
> +I/O handling model that replaces the traditional per-I/O commands with
> +per-queue batch commands. This significantly reduces communication overhead
> +and enables better load balancing across multiple server tasks.
> +
> +Key differences from traditional mode:
> +
> +- **Per-queue vs Per-I/O**: Commands operate on queues rather than individual I/Os
> +- **Batch processing**: Multiple I/Os are handled in single operations
> +- **Multishot commands**: Use io_uring multishot for reduced submission overhead
> +- **Flexible task assignment**: Any task can handle any I/O (no per-I/O daemons)
> +- **Better load balancing**: Tasks can adjust their workload dynamically
> +
> +Batch I/O Commands:
> +
> +- ``UBLK_U_IO_PREP_IO_CMDS``
> +
> +  Prepares multiple I/O commands in batch. The server provides a buffer
> +  containing multiple I/O descriptors that will be processed together.
> +  This reduces the number of individual command submissions required.
> +
> +- ``UBLK_U_IO_COMMIT_IO_CMDS``
> +
> +  Commits results for multiple I/O operations in batch. The server provides

And prepares the I/O descriptors to accept new requests?

> +  a buffer containing the results of multiple completed I/Os, allowing
> +  efficient bulk completion of requests.
> +
> +- ``UBLK_U_IO_FETCH_IO_CMDS``
> +
> +  **Multishot command** for fetching I/O commands in batch. This is the key
> +  command that enables high-performance batch processing:
> +
> +  * Uses io_uring multishot capability for reduced submission overhead
> +  * Single command can fetch multiple I/O requests over time
> +  * Buffer size determines maximum batch size per operation
> +  * Multiple fetch commands can be submitted for load balancing
> +  * Only one fetch command is active at any time per queue

Can you clarify what the lifetime of the fetch command is? It looks
like as long as the buffer selection and posting of the multishot CQE
succeeds, the same UBLK_U_IO_FETCH_IO_CMDS will continue to be used.
If additional UBLK_U_IO_FETCH_IO_CMDS commands are issued to the queue
(e.g. by other threads), they won't be used until the first one fails
to select a buffer or post the CQE? Seems like this would make it
difficult to load-balance incoming requests on a single ublk queue
between multiple threads.

Best,
Caleb

> +  * Supports dynamic load balancing across multiple server tasks
> +
> +  Each task can submit ``UBLK_U_IO_FETCH_IO_CMDS`` with different buffer
> +  sizes to control how much work it handles. This enables sophisticated
> +  load balancing strategies in multi-threaded servers.
> +
> +Migration: Applications using traditional commands (``UBLK_U_IO_FETCH_REQ``,
> +``UBLK_U_IO_COMMIT_AND_FETCH_REQ``) cannot use batch mode simultaneously.
> +
>  Zero copy
>  ---------
>
> --
> 2.47.0
>

On Mon, Dec 01, 2025 at 01:46:19PM -0800, Caleb Sander Mateos wrote:
> On Thu, Nov 20, 2025 at 6:00 PM Ming Lei <ming.lei@redhat.com> wrote:
> >
> > Document feature UBLK_F_BATCH_IO.
> >
> > Signed-off-by: Ming Lei <ming.lei@redhat.com>
> > ---
> >  Documentation/block/ublk.rst | 60 +++++++++++++++++++++++++++++++++---
> >  1 file changed, 56 insertions(+), 4 deletions(-)
> >
> > diff --git a/Documentation/block/ublk.rst b/Documentation/block/ublk.rst
> > index 8c4030bcabb6..09a5604f8e10 100644
> > --- a/Documentation/block/ublk.rst
> > +++ b/Documentation/block/ublk.rst
> > @@ -260,9 +260,12 @@ The following IO commands are communicated via io_uring passthrough command,
> >  and each command is only for forwarding the IO and committing the result
> >  with specified IO tag in the command data:
> >
> > -- ``UBLK_IO_FETCH_REQ``
> > +Traditional Per-I/O Commands
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >
> > -  Sent from the server IO pthread for fetching future incoming IO requests
> > +- ``UBLK_U_IO_FETCH_REQ``
> > +
> > +  Sent from the server I/O pthread for fetching future incoming I/O requests
> >    destined to ``/dev/ublkb*``. This command is sent only once from the server
> >    IO pthread for ublk driver to setup IO forward environment.
> >
> > @@ -278,7 +281,7 @@ with specified IO tag in the command data:
> >    supported by the driver, daemons must be per-queue instead - i.e. all I/Os
> >    associated to a single qid must be handled by the same task.
> >
> > -- ``UBLK_IO_COMMIT_AND_FETCH_REQ``
> > +- ``UBLK_U_IO_COMMIT_AND_FETCH_REQ``
> >
> >    When an IO request is destined to ``/dev/ublkb*``, the driver stores
> >    the IO's ``ublksrv_io_desc`` to the specified mapped area; then the
> > @@ -293,7 +296,7 @@ with specified IO tag in the command data:
> >    requests with the same IO tag. That is, ``UBLK_IO_COMMIT_AND_FETCH_REQ``
> >    is reused for both fetching request and committing back IO result.
> >
> > -- ``UBLK_IO_NEED_GET_DATA``
> > +- ``UBLK_U_IO_NEED_GET_DATA``
> >
> >    With ``UBLK_F_NEED_GET_DATA`` enabled, the WRITE request will be firstly
> >    issued to ublk server without data copy. Then, IO backend of ublk server
> > @@ -322,6 +325,55 @@ with specified IO tag in the command data:
> >    ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy
> >    the server buffer (pages) read to the IO request pages.
> >
> > +Batch I/O Commands (UBLK_F_BATCH_IO)
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +The ``UBLK_F_BATCH_IO`` feature provides an alternative high-performance
> > +I/O handling model that replaces the traditional per-I/O commands with
> > +per-queue batch commands. This significantly reduces communication overhead
> > +and enables better load balancing across multiple server tasks.
> > +
> > +Key differences from traditional mode:
> > +
> > +- **Per-queue vs Per-I/O**: Commands operate on queues rather than individual I/Os
> > +- **Batch processing**: Multiple I/Os are handled in single operations
> > +- **Multishot commands**: Use io_uring multishot for reduced submission overhead
> > +- **Flexible task assignment**: Any task can handle any I/O (no per-I/O daemons)
> > +- **Better load balancing**: Tasks can adjust their workload dynamically
> > +
> > +Batch I/O Commands:
> > +
> > +- ``UBLK_U_IO_PREP_IO_CMDS``
> > +
> > +  Prepares multiple I/O commands in batch. The server provides a buffer
> > +  containing multiple I/O descriptors that will be processed together.
> > +  This reduces the number of individual command submissions required.
> > +
> > +- ``UBLK_U_IO_COMMIT_IO_CMDS``
> > +
> > +  Commits results for multiple I/O operations in batch. The server provides
> 
> And prepares the I/O descriptors to accept new requests?
> 
> > +  a buffer containing the results of multiple completed I/Os, allowing
> > +  efficient bulk completion of requests.
> > +
> > +- ``UBLK_U_IO_FETCH_IO_CMDS``
> > +
> > +  **Multishot command** for fetching I/O commands in batch. This is the key
> > +  command that enables high-performance batch processing:
> > +
> > +  * Uses io_uring multishot capability for reduced submission overhead
> > +  * Single command can fetch multiple I/O requests over time
> > +  * Buffer size determines maximum batch size per operation
> > +  * Multiple fetch commands can be submitted for load balancing
> > +  * Only one fetch command is active at any time per queue
> 
> Can you clarify what the lifetime of the fetch command is? It looks

The fetch command is live if the provided buffer isn't full, which aligns
with typical io_uring multishot req & provided buffer use case, such as
IORING_OP_READ_MULTISHOT.

Also the fetch command is completed in case of FETCH failure.

```
A multishot request will persist as long as no errors are encountered doing
handling of the request. For each CQE posted on behalf of this request, the
CQE flags will have IORING_CQE_F_MORE set if the application should expect
more completions from this request. If this flag isn’t set, then that signifies
termination of the multishot read request.
```

> like as long as the buffer selection and posting of the multishot CQE
> succeeds, the same UBLK_U_IO_FETCH_IO_CMDS will continue to be used.
> If additional UBLK_U_IO_FETCH_IO_CMDS commands are issued to the queue
> (e.g. by other threads), they won't be used until the first one fails
> to select a buffer or post the CQE? Seems like this would make it
> difficult to load-balance incoming requests on a single ublk queue
> between multiple threads.


Thanks,
Ming

On Mon, Dec 01, 2025 at 01:46:19PM -0800, Caleb Sander Mateos wrote:
> On Thu, Nov 20, 2025 at 6:00 PM Ming Lei <ming.lei@redhat.com> wrote:
> >
> > Document feature UBLK_F_BATCH_IO.
> >
> > Signed-off-by: Ming Lei <ming.lei@redhat.com>
> > ---
> >  Documentation/block/ublk.rst | 60 +++++++++++++++++++++++++++++++++---
> >  1 file changed, 56 insertions(+), 4 deletions(-)
> >
> > diff --git a/Documentation/block/ublk.rst b/Documentation/block/ublk.rst
> > index 8c4030bcabb6..09a5604f8e10 100644
> > --- a/Documentation/block/ublk.rst
> > +++ b/Documentation/block/ublk.rst
> > @@ -260,9 +260,12 @@ The following IO commands are communicated via io_uring passthrough command,
> >  and each command is only for forwarding the IO and committing the result
> >  with specified IO tag in the command data:
> >
> > -- ``UBLK_IO_FETCH_REQ``
> > +Traditional Per-I/O Commands
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >
> > -  Sent from the server IO pthread for fetching future incoming IO requests
> > +- ``UBLK_U_IO_FETCH_REQ``
> > +
> > +  Sent from the server I/O pthread for fetching future incoming I/O requests
> >    destined to ``/dev/ublkb*``. This command is sent only once from the server
> >    IO pthread for ublk driver to setup IO forward environment.
> >
> > @@ -278,7 +281,7 @@ with specified IO tag in the command data:
> >    supported by the driver, daemons must be per-queue instead - i.e. all I/Os
> >    associated to a single qid must be handled by the same task.
> >
> > -- ``UBLK_IO_COMMIT_AND_FETCH_REQ``
> > +- ``UBLK_U_IO_COMMIT_AND_FETCH_REQ``
> >
> >    When an IO request is destined to ``/dev/ublkb*``, the driver stores
> >    the IO's ``ublksrv_io_desc`` to the specified mapped area; then the
> > @@ -293,7 +296,7 @@ with specified IO tag in the command data:
> >    requests with the same IO tag. That is, ``UBLK_IO_COMMIT_AND_FETCH_REQ``
> >    is reused for both fetching request and committing back IO result.
> >
> > -- ``UBLK_IO_NEED_GET_DATA``
> > +- ``UBLK_U_IO_NEED_GET_DATA``
> >
> >    With ``UBLK_F_NEED_GET_DATA`` enabled, the WRITE request will be firstly
> >    issued to ublk server without data copy. Then, IO backend of ublk server
> > @@ -322,6 +325,55 @@ with specified IO tag in the command data:
> >    ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy
> >    the server buffer (pages) read to the IO request pages.
> >
> > +Batch I/O Commands (UBLK_F_BATCH_IO)
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +The ``UBLK_F_BATCH_IO`` feature provides an alternative high-performance
> > +I/O handling model that replaces the traditional per-I/O commands with
> > +per-queue batch commands. This significantly reduces communication overhead
> > +and enables better load balancing across multiple server tasks.
> > +
> > +Key differences from traditional mode:
> > +
> > +- **Per-queue vs Per-I/O**: Commands operate on queues rather than individual I/Os
> > +- **Batch processing**: Multiple I/Os are handled in single operations
> > +- **Multishot commands**: Use io_uring multishot for reduced submission overhead
> > +- **Flexible task assignment**: Any task can handle any I/O (no per-I/O daemons)
> > +- **Better load balancing**: Tasks can adjust their workload dynamically
> > +
> > +Batch I/O Commands:
> > +
> > +- ``UBLK_U_IO_PREP_IO_CMDS``
> > +
> > +  Prepares multiple I/O commands in batch. The server provides a buffer
> > +  containing multiple I/O descriptors that will be processed together.
> > +  This reduces the number of individual command submissions required.
> > +
> > +- ``UBLK_U_IO_COMMIT_IO_CMDS``
> > +
> > +  Commits results for multiple I/O operations in batch. The server provides
> 
> And prepares the I/O descriptors to accept new requests?

Yeah, will add it in next version.

> 
> > +  a buffer containing the results of multiple completed I/Os, allowing
> > +  efficient bulk completion of requests.
> > +
> > +- ``UBLK_U_IO_FETCH_IO_CMDS``
> > +
> > +  **Multishot command** for fetching I/O commands in batch. This is the key
> > +  command that enables high-performance batch processing:
> > +
> > +  * Uses io_uring multishot capability for reduced submission overhead
> > +  * Single command can fetch multiple I/O requests over time
> > +  * Buffer size determines maximum batch size per operation
> > +  * Multiple fetch commands can be submitted for load balancing
> > +  * Only one fetch command is active at any time per queue
> 
> Can you clarify what the lifetime of the fetch command is? It looks
> like as long as the buffer selection and posting of the multishot CQE
> succeeds, the same UBLK_U_IO_FETCH_IO_CMDS will continue to be used.

Yeah, it means the provided buffer isn't full yet.

> If additional UBLK_U_IO_FETCH_IO_CMDS commands are issued to the queue
> (e.g. by other threads), they won't be used until the first one fails
> to select a buffer or post the CQE? Seems like this would make it
> difficult to load-balance incoming requests on a single ublk queue
> between multiple threads.

So far, fetch command is added in FIFO style, so new fetch command
can only be handled when old commands are done, ublk server can support
simple load balance by adjusting the fetch buffer size dynamically:

- if one pthread is close to saturate, the fetch buffer size can be reduced

- if one pthread has more capacity, the fetch buffer size can be increased

In future, it should be easy to introduce fetch command priority, so ublk
server can balance load by controlling either fetch buffer size or command
priority.


Thanks, 
Ming