This patch add a new hybrid poll at io_uring level, it also set a signal
"IORING_SETUP_HYBRID_IOPOLL" to application, aim to provide a interface for
users to enable hybrid polling.

Hybrid poll may appropriate for some performance bottlenecks due to CPU
resource constraints, such as some database applications. In a
high-concurrency state, not only polling takes up a lot of CPU time, but
also operations like calculation and processing also need to compete for
CPU time.

The MultiRead interface of Rocksdb has been adapted to io_uring. Here used
db_bench to construct a situation with high CPU pressure and compared the
performance. The test configuration is as follows,

-------------------------------------------------------------------
CPU Model       Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz
CPU Cores       8
Memory          16G
SSD             Samsung PM9A3
-------------------------------------------------------------------

Test case:
./db_bench --benchmarks=multireadrandom,stats
--duration=60
--threads=4/8/16
--use_direct_reads=true
--db=/mnt/rocks/test_db
--wal_dir=/mnt/rocks/test_db
--key_size=4
--value_size=4096
-cache_size=0
-use_existing_db=1
-batch_size=256
-multiread_batched=true
-multiread_stride=0
---------------------------------------------------------------
Test result:
          National        Optimization
thread    sops/sec        ops/sec        CPU Utilization
16        121953          160233         100%*8
8         120198          116087         90%*8
4         61302           59105          90%*8
---------------------------------------------------------------

The 9th version patch makes following changes:

1. change some member and function name

2. Avoid the expansion of io_kiocb structure. After checking, the hash_node
structure is used in asynchronous poll, while the iopoll only supports the
dirict io for disk, these two path are different and they will not be used
simultaneously, it also confirmed in the code. So I shared this space with
iopoll_start.

 union {
	/*
	 * for polled requests, i.e. IORING_OP_POLL_ADD and async armed
	 * poll
	 */
	struct hlist_node   hash_node;
	/* For IOPOLL setup queues, with hybrid polling */
	u64                     iopoll_start;
 };

3. Avoid the expansion of io_ring_ctx structure. Although there is an
8-byte hole in the first structure, the structure is basically constants
and some read-only hot data that will not be changed, that means this cache
does not need to be brushed down frequently, but the hybrid_poll_time of
the recorded run time had a chance to be modified several times. So I put
it in the second structure (submission data), which is still 24 bytes of
space, and some of its own variables also need to be modified.

4. Add the poll_state identity to the flags of req.

/* every req only blocks once in hybrid poll */
REQ_F_IOPOLL_STATE = IO_REQ_FLAG(REQ_F_HYBRID_IOPOLL_STATE_BIT)

--
changes since v7:
- rebase code on for-6.12/io_uring
- remove unused varibales

changes since v6:
- Modified IO path, distinct iopoll and uring_cmd_iopoll
- update test results

changes since v5:
- Remove cstime recorder
- Use minimize sleep time in different drivers
- Use the half of whole runtime to do schedule
- Consider as a suboptimal solution between
  regular poll and IRQ

changes since v4:
- Rewrote the commit
- Update the test results
- Reorganized the code basd on 6.11

changes since v3:
- Simplified the commit
- Add some comments on code

changes since v2:
- Modified some formatting errors
- Move judgement to poll path

changes since v1:
- Extend hybrid poll to async polled io

hexue (1):
  io_uring: releasing CPU resources when polling

 include/linux/io_uring_types.h | 19 ++++++-
 include/uapi/linux/io_uring.h  |  3 ++
 io_uring/io_uring.c            |  8 ++-
 io_uring/rw.c                  | 92 ++++++++++++++++++++++++++++++----
 4 files changed, 108 insertions(+), 14 deletions(-)

-- 
2.40.1

On Fri, 01 Nov 2024 17:19:56 +0800, hexue wrote:
> This patch add a new hybrid poll at io_uring level, it also set a signal
> "IORING_SETUP_HYBRID_IOPOLL" to application, aim to provide a interface for
> users to enable hybrid polling.
> 
> Hybrid poll may appropriate for some performance bottlenecks due to CPU
> resource constraints, such as some database applications. In a
> high-concurrency state, not only polling takes up a lot of CPU time, but
> also operations like calculation and processing also need to compete for
> CPU time.
> 
> [...]

Applied, thanks!

[1/1] io_uring: releasing CPU resources when polling
      commit: 71b51c2fb200c502626e433ac7e22bcb8a3ae00c

Best regards,
-- 
Jens Axboe

On 11/1/24 3:19 AM, hexue wrote:
> changes since v7:
> - rebase code on for-6.12/io_uring

Though not sure why you'd base it on a branch that's long dead,
for-6.13/io_uring is the appropriate branch. for-6.12/io_uring was
things queued up for 6.12, it went extinct as soon as the merge window
opened for 6.12 and it got queued up. Not a big deal as it can get hand
applied, but new features should always get based on the branch for the
next kernel, not the previous one.

-- 
Jens Axboe