[PATCH v3 00/17] 64bit block-layer

Vladimir Sementsov-Ogievskiy posted 17 patches 3 years, 12 months ago
Test FreeBSD failed
Test docker-mingw@fedora passed
Test checkpatch passed
Test asan failed
Test docker-quick@centos7 failed
Failed in applying to current master (apply log)
There is a newer version of this series
include/block/block.h           |  17 +++--
include/block/block_int.h       |  67 ++++++++---------
include/block/throttle-groups.h |   2 +-
include/sysemu/block-backend.h  |  26 +++----
block/backup-top.c              |  14 ++--
block/blkdebug.c                |  12 +--
block/blklogwrites.c            |  16 ++--
block/blkreplay.c               |   8 +-
block/blkverify.c               |  10 +--
block/block-backend.c           |  60 +++++++--------
block/bochs.c                   |   4 +-
block/cloop.c                   |   4 +-
block/commit.c                  |   2 +-
block/copy-on-read.c            |  14 ++--
block/crypto.c                  |   8 +-
block/curl.c                    |   3 +-
block/dmg.c                     |   4 +-
block/file-posix.c              |  46 ++++++++----
block/file-win32.c              |   8 +-
block/filter-compress.c         |  15 ++--
block/gluster.c                 |  14 ++--
block/io.c                      | 126 +++++++++++++++++---------------
block/iscsi.c                   |  34 ++++++---
block/mirror.c                  |   8 +-
block/nbd.c                     |  18 +++--
block/nfs.c                     |  12 +--
block/null.c                    |  18 +++--
block/nvme.c                    |  38 +++++++---
block/qcow.c                    |  16 ++--
block/qcow2.c                   |  34 +++++----
block/qed.c                     |  17 ++++-
block/quorum.c                  |   9 ++-
block/raw-format.c              |  36 ++++-----
block/rbd.c                     |  10 ++-
block/sheepdog.c                |  11 ++-
block/throttle-groups.c         |   5 +-
block/throttle.c                |  14 ++--
block/vdi.c                     |   8 +-
block/vmdk.c                    |  14 ++--
block/vpc.c                     |   8 +-
block/vvfat.c                   |  12 +--
block/vxhs.c                    |   8 +-
tests/test-bdrv-drain.c         |  16 ++--
tests/test-block-iothread.c     |  19 +++--
block/trace-events              |  14 ++--
45 files changed, 488 insertions(+), 371 deletions(-)
[PATCH v3 00/17] 64bit block-layer
Posted by Vladimir Sementsov-Ogievskiy 3 years, 12 months ago
Hi all!

We want 64bit write-zeroes, and for this, convert all io functions to
64bit.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

Please refer to initial cover-letter 
 https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg08723.html
for more info.

v3 is available at
 https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-64bit-block-layer-v3

v3: Based on "[PATCH v2 0/9] block/io: safer inc/dec in_flight sections"
Add Eric's r-bs, improve commit message with short reasoning of the whole
thing, and Eric's audits (if you don't like something, I'll change or drop for
next series).

Add "Series:" tag to each patch. Just an idea, if it's inappropriate thing,
I'll drop it.

01: add assertion that bytes > 0
02: fix indentation
06: refactor calculations in bdrv_co_write_req_prepare
09,10: simple rebase conflicts solved

Also, cover more drivers by driver-updating patches and fix int flags
to be BdrvRequestFlags flags.

Based-on: <20200427143907.5710-1-vsementsov@virtuozzo.com>
Series: 64bit-block-status

Vladimir Sementsov-Ogievskiy (17):
  block/throttle-groups: throttle_group_co_io_limits_intercept(): 64bit
    bytes
  block: use int64_t as bytes type in tracked requests
  block/io: use int64_t bytes parameter in bdrv_check_byte_request()
  block/io: use int64_t bytes in driver wrappers
  block/io: support int64_t bytes in bdrv_co_do_pwrite_zeroes()
  block/io: support int64_t bytes in bdrv_aligned_pwritev()
  block/io: support int64_t bytes in bdrv_co_do_copy_on_readv()
  block/io: support int64_t bytes in bdrv_aligned_preadv()
  block/io: support int64_t bytes in bdrv_co_p{read,write}v_part()
  block/io: support int64_t bytes in read/write wrappers
  block/io: use int64_t bytes in copy_range
  block/block-backend: convert blk io path to use int64_t parameters
  block: use int64_t instead of uint64_t in driver read handlers
  block: use int64_t instead of uint64_t in driver write handlers
  block: use int64_t instead of uint64_t in copy_range driver handlers
  block: use int64_t instead of int in driver write_zeroes handlers
  block: use int64_t instead of int in driver discard handlers

 include/block/block.h           |  17 +++--
 include/block/block_int.h       |  67 ++++++++---------
 include/block/throttle-groups.h |   2 +-
 include/sysemu/block-backend.h  |  26 +++----
 block/backup-top.c              |  14 ++--
 block/blkdebug.c                |  12 +--
 block/blklogwrites.c            |  16 ++--
 block/blkreplay.c               |   8 +-
 block/blkverify.c               |  10 +--
 block/block-backend.c           |  60 +++++++--------
 block/bochs.c                   |   4 +-
 block/cloop.c                   |   4 +-
 block/commit.c                  |   2 +-
 block/copy-on-read.c            |  14 ++--
 block/crypto.c                  |   8 +-
 block/curl.c                    |   3 +-
 block/dmg.c                     |   4 +-
 block/file-posix.c              |  46 ++++++++----
 block/file-win32.c              |   8 +-
 block/filter-compress.c         |  15 ++--
 block/gluster.c                 |  14 ++--
 block/io.c                      | 126 +++++++++++++++++---------------
 block/iscsi.c                   |  34 ++++++---
 block/mirror.c                  |   8 +-
 block/nbd.c                     |  18 +++--
 block/nfs.c                     |  12 +--
 block/null.c                    |  18 +++--
 block/nvme.c                    |  38 +++++++---
 block/qcow.c                    |  16 ++--
 block/qcow2.c                   |  34 +++++----
 block/qed.c                     |  17 ++++-
 block/quorum.c                  |   9 ++-
 block/raw-format.c              |  36 ++++-----
 block/rbd.c                     |  10 ++-
 block/sheepdog.c                |  11 ++-
 block/throttle-groups.c         |   5 +-
 block/throttle.c                |  14 ++--
 block/vdi.c                     |   8 +-
 block/vmdk.c                    |  14 ++--
 block/vpc.c                     |   8 +-
 block/vvfat.c                   |  12 +--
 block/vxhs.c                    |   8 +-
 tests/test-bdrv-drain.c         |  16 ++--
 tests/test-block-iothread.c     |  19 +++--
 block/trace-events              |  14 ++--
 45 files changed, 488 insertions(+), 371 deletions(-)

-- 
2.21.0


Re: [PATCH v3 00/17] 64bit block-layer
Posted by no-reply@patchew.org 3 years, 11 months ago
Patchew URL: https://patchew.org/QEMU/20200430111033.29980-1-vsementsov@virtuozzo.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      tests/test-timed-average.o
  CC      tests/test-util-filemonitor.o
  CC      tests/test-util-sockets.o
/tmp/qemu-test/src/tests/test-block-iothread.c:79:5: error: initialization from incompatible pointer type [-Werror]
     .bdrv_co_pdiscard       = bdrv_test_co_pdiscard,
     ^
/tmp/qemu-test/src/tests/test-block-iothread.c:79:5: error: (near initialization for 'bdrv_test.bdrv_co_pdiscard') [-Werror]
cc1: all warnings being treated as errors
  CC      tests/test-authz-simple.o
make: *** [tests/test-block-iothread.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=42df3799b5074b82a8a4b696cb76547f', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-t9q8cenf/src/docker-src.2020-04-30-16.47.42.6034:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=42df3799b5074b82a8a4b696cb76547f
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-t9q8cenf/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    3m38.963s
user    0m9.441s


The full log is available at
http://patchew.org/logs/20200430111033.29980-1-vsementsov@virtuozzo.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v3 00/17] 64bit block-layer
Posted by Vladimir Sementsov-Ogievskiy 3 years, 11 months ago
30.04.2020 23:51, no-reply@patchew.org wrote:
> Patchew URL: https://patchew.org/QEMU/20200430111033.29980-1-vsementsov@virtuozzo.com/
> 
> 
> 
> Hi,
> 
> This series failed the docker-quick@centos7 build test. Please find the testing commands and
> their output below. If you have Docker installed, you can probably reproduce it
> locally.
> 
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> make docker-image-centos7 V=1 NETWORK=1
> time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
> === TEST SCRIPT END ===
> 
>    CC      tests/test-timed-average.o
>    CC      tests/test-util-filemonitor.o
>    CC      tests/test-util-sockets.o
> /tmp/qemu-test/src/tests/test-block-iothread.c:79:5: error: initialization from incompatible pointer type [-Werror]
>       .bdrv_co_pdiscard       = bdrv_test_co_pdiscard,
>       ^

Oh yes. Shows me didn't run make check :(

> /tmp/qemu-test/src/tests/test-block-iothread.c:79:5: error: (near initialization for 'bdrv_test.bdrv_co_pdiscard') [-Werror]
> cc1: all warnings being treated as errors
>    CC      tests/test-authz-simple.o
> make: *** [tests/test-block-iothread.o] Error 1
> make: *** Waiting for unfinished jobs....
> Traceback (most recent call last):
>    File "./tests/docker/docker.py", line 664, in <module>
> ---
>      raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=42df3799b5074b82a8a4b696cb76547f', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-t9q8cenf/src/docker-src.2020-04-30-16.47.42.6034:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
> filter=--filter=label=com.qemu.instance.uuid=42df3799b5074b82a8a4b696cb76547f
> make[1]: *** [docker-run] Error 1
> make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-t9q8cenf/src'
> make: *** [docker-run-test-quick@centos7] Error 2
> 
> real    3m38.963s
> user    0m9.441s
> 
> 
> The full log is available at
> http://patchew.org/logs/20200430111033.29980-1-vsementsov@virtuozzo.com/testing.docker-quick@centos7/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-devel@redhat.com
> 


-- 
Best regards,
Vladimir

Re: [PATCH v3 00/17] 64bit block-layer
Posted by no-reply@patchew.org 3 years, 11 months ago
Patchew URL: https://patchew.org/QEMU/20200430111033.29980-1-vsementsov@virtuozzo.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

clang -iquote /tmp/qemu-test/build/tests -iquote tests -iquote /tmp/qemu-test/src/tcg/i386 -isystem /tmp/qemu-test/src/linux-headers -isystem /tmp/qemu-test/build/linux-headers -iquote . -iquote /tmp/qemu-test/src -iquote /tmp/qemu-test/src/accel/tcg -iquote /tmp/qemu-test/src/include -iquote /tmp/qemu-test/src/disas/libvixl -I/usr/include/pixman-1   -Werror -fsanitize=undefined -fsanitize=address  -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include  -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99  -Wno-string-plus-int -Wno-typedef-redefinition -Wno-initializer-overrides -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-definition -Wtype-limits -fstack-protector-strong   -I/usr/include/p11-kit-1   -DLEGACY_RDMA_REG_MR -DSTRUCT_IOVEC_DEFINED  -I/usr/include/libpng16  -I/usr/include/spice-1 -I/usr/include/spice-server -I/usr/include/cacard -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/uuid -I/usr/include/pixman-1   -I/tmp/qemu-test/src/tests -I/tmp/qemu-test/src/tests/qtest -MMD -MP -MT tests/test-crypto-secret.o -MF tests/test-crypto-secret.d -g   -c -o tests/test-crypto-secret.o /tmp/qemu-test/src/tests/test-crypto-secret.c
clang -iquote /tmp/qemu-test/build/tests -iquote tests -iquote /tmp/qemu-test/src/tcg/i386 -isystem /tmp/qemu-test/src/linux-headers -isystem /tmp/qemu-test/build/linux-headers -iquote . -iquote /tmp/qemu-test/src -iquote /tmp/qemu-test/src/accel/tcg -iquote /tmp/qemu-test/src/include -iquote /tmp/qemu-test/src/disas/libvixl -I/usr/include/pixman-1   -Werror -fsanitize=undefined -fsanitize=address  -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include  -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99  -Wno-string-plus-int -Wno-typedef-redefinition -Wno-initializer-overrides -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-definition -Wtype-limits -fstack-protector-strong   -I/usr/include/p11-kit-1   -DLEGACY_RDMA_REG_MR -DSTRUCT_IOVEC_DEFINED  -I/usr/include/libpng16  -I/usr/include/spice-1 -I/usr/include/spice-server -I/usr/include/cacard -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/uuid -I/usr/include/pixman-1   -I/tmp/qemu-test/src/tests -I/tmp/qemu-test/src/tests/qtest -MMD -MP -MT tests/test-crypto-tlscredsx509.o -MF tests/test-crypto-tlscredsx509.d -g   -c -o tests/test-crypto-tlscredsx509.o /tmp/qemu-test/src/tests/test-crypto-tlscredsx509.c
clang -iquote /tmp/qemu-test/build/tests -iquote tests -iquote /tmp/qemu-test/src/tcg/i386 -isystem /tmp/qemu-test/src/linux-headers -isystem /tmp/qemu-test/build/linux-headers -iquote . -iquote /tmp/qemu-test/src -iquote /tmp/qemu-test/src/accel/tcg -iquote /tmp/qemu-test/src/include -iquote /tmp/qemu-test/src/disas/libvixl -I/usr/include/pixman-1   -Werror -fsanitize=undefined -fsanitize=address  -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include  -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99  -Wno-string-plus-int -Wno-typedef-redefinition -Wno-initializer-overrides -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-definition -Wtype-limits -fstack-protector-strong   -I/usr/include/p11-kit-1   -DLEGACY_RDMA_REG_MR -DSTRUCT_IOVEC_DEFINED  -I/usr/include/libpng16  -I/usr/include/spice-1 -I/usr/include/spice-server -I/usr/include/cacard -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/uuid -I/usr/include/pixman-1   -I/tmp/qemu-test/src/tests -I/tmp/qemu-test/src/tests/qtest -MMD -MP -MT tests/crypto-tls-x509-helpers.o -MF tests/crypto-tls-x509-helpers.d -g   -c -o tests/crypto-tls-x509-helpers.o /tmp/qemu-test/src/tests/crypto-tls-x509-helpers.c
/tmp/qemu-test/src/tests/test-block-iothread.c:79:31: error: incompatible pointer types initializing 'int (*)(BlockDriverState *, int64_t, int64_t)' (aka 'int (*)(struct BlockDriverState *, long, long)') with an expression of type 'int (BlockDriverState *, int64_t, int)' (aka 'int (struct BlockDriverState *, long, int)') [-Werror,-Wincompatible-pointer-types]
    .bdrv_co_pdiscard       = bdrv_test_co_pdiscard,
                              ^~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [/tmp/qemu-test/src/rules.mak:69: tests/test-block-iothread.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=e0743bd266804107bb9089aec26150f7', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-39rsbv95/src/docker-src.2020-04-30-16.52.24.18470:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=e0743bd266804107bb9089aec26150f7
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-39rsbv95/src'
make: *** [docker-run-test-debug@fedora] Error 2

real    4m51.462s
user    0m8.644s


The full log is available at
http://patchew.org/logs/20200430111033.29980-1-vsementsov@virtuozzo.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v3 00/17] 64bit block-layer
Posted by Vladimir Sementsov-Ogievskiy 3 years, 4 months ago
Hi!

I'm sorry, I should have pinged it, or resend, or suggest to pull at least a half long ago :(

I've rebased it on master and make some fixes.

What to do next? I can just resend. But I'm afraid that Eric's careful audits may be out of date: time passed, there is no guarantee that callers are not changed. Really sorry :(
So r-b marks are not applicable as well, yes?

But if I just resend it with no r-bs, is it feasible to review/merge it in a finite time? So that audits of patches will not become outdated?

Any ideas?

I have an idea: instead of auditing each function callers, can we just make some good assumptions (like that the whole offset/bytes request being aligned to bs->request_alignement doesn't lay inside [0..INT64_MAX] region), check it once in bdrv_check_bytes_request() and assert in each function we convert to int64_t.

Then, if somewhere our assumption is wrong, we'll have a crash and fix the bug.

30.04.2020 14:10, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> We want 64bit write-zeroes, and for this, convert all io functions to
> 64bit.
> 
> We chose signed type, to be consistent with off_t (which is signed) and
> with possibility for signed return type (where negative value means
> error).
> 
> Please refer to initial cover-letter
>   https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg08723.html
> for more info.
> 
> v3 is available at
>   https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-64bit-block-layer-v3
> 
> v3: Based on "[PATCH v2 0/9] block/io: safer inc/dec in_flight sections"
> Add Eric's r-bs, improve commit message with short reasoning of the whole
> thing, and Eric's audits (if you don't like something, I'll change or drop for
> next series).
> 
> Add "Series:" tag to each patch. Just an idea, if it's inappropriate thing,
> I'll drop it.
> 
> 01: add assertion that bytes > 0
> 02: fix indentation
> 06: refactor calculations in bdrv_co_write_req_prepare
> 09,10: simple rebase conflicts solved
> 
> Also, cover more drivers by driver-updating patches and fix int flags
> to be BdrvRequestFlags flags.
> 
> Based-on: <20200427143907.5710-1-vsementsov@virtuozzo.com>
> Series: 64bit-block-status
> 
> Vladimir Sementsov-Ogievskiy (17):
>    block/throttle-groups: throttle_group_co_io_limits_intercept(): 64bit
>      bytes
>    block: use int64_t as bytes type in tracked requests
>    block/io: use int64_t bytes parameter in bdrv_check_byte_request()
>    block/io: use int64_t bytes in driver wrappers
>    block/io: support int64_t bytes in bdrv_co_do_pwrite_zeroes()
>    block/io: support int64_t bytes in bdrv_aligned_pwritev()
>    block/io: support int64_t bytes in bdrv_co_do_copy_on_readv()
>    block/io: support int64_t bytes in bdrv_aligned_preadv()
>    block/io: support int64_t bytes in bdrv_co_p{read,write}v_part()
>    block/io: support int64_t bytes in read/write wrappers
>    block/io: use int64_t bytes in copy_range
>    block/block-backend: convert blk io path to use int64_t parameters
>    block: use int64_t instead of uint64_t in driver read handlers
>    block: use int64_t instead of uint64_t in driver write handlers
>    block: use int64_t instead of uint64_t in copy_range driver handlers
>    block: use int64_t instead of int in driver write_zeroes handlers
>    block: use int64_t instead of int in driver discard handlers
> 
>   include/block/block.h           |  17 +++--
>   include/block/block_int.h       |  67 ++++++++---------
>   include/block/throttle-groups.h |   2 +-
>   include/sysemu/block-backend.h  |  26 +++----
>   block/backup-top.c              |  14 ++--
>   block/blkdebug.c                |  12 +--
>   block/blklogwrites.c            |  16 ++--
>   block/blkreplay.c               |   8 +-
>   block/blkverify.c               |  10 +--
>   block/block-backend.c           |  60 +++++++--------
>   block/bochs.c                   |   4 +-
>   block/cloop.c                   |   4 +-
>   block/commit.c                  |   2 +-
>   block/copy-on-read.c            |  14 ++--
>   block/crypto.c                  |   8 +-
>   block/curl.c                    |   3 +-
>   block/dmg.c                     |   4 +-
>   block/file-posix.c              |  46 ++++++++----
>   block/file-win32.c              |   8 +-
>   block/filter-compress.c         |  15 ++--
>   block/gluster.c                 |  14 ++--
>   block/io.c                      | 126 +++++++++++++++++---------------
>   block/iscsi.c                   |  34 ++++++---
>   block/mirror.c                  |   8 +-
>   block/nbd.c                     |  18 +++--
>   block/nfs.c                     |  12 +--
>   block/null.c                    |  18 +++--
>   block/nvme.c                    |  38 +++++++---
>   block/qcow.c                    |  16 ++--
>   block/qcow2.c                   |  34 +++++----
>   block/qed.c                     |  17 ++++-
>   block/quorum.c                  |   9 ++-
>   block/raw-format.c              |  36 ++++-----
>   block/rbd.c                     |  10 ++-
>   block/sheepdog.c                |  11 ++-
>   block/throttle-groups.c         |   5 +-
>   block/throttle.c                |  14 ++--
>   block/vdi.c                     |   8 +-
>   block/vmdk.c                    |  14 ++--
>   block/vpc.c                     |   8 +-
>   block/vvfat.c                   |  12 +--
>   block/vxhs.c                    |   8 +-
>   tests/test-bdrv-drain.c         |  16 ++--
>   tests/test-block-iothread.c     |  19 +++--
>   block/trace-events              |  14 ++--
>   45 files changed, 488 insertions(+), 371 deletions(-)
> 


-- 
Best regards,
Vladimir

Re: [PATCH v3 00/17] 64bit block-layer
Posted by Vladimir Sementsov-Ogievskiy 3 years, 4 months ago
01.12.2020 19:07, Vladimir Sementsov-Ogievskiy wrote:
> 
> I have an idea: instead of auditing each function callers, can we just make some good assumptions (like that the whole offset/bytes request being aligned to bs->request_alignement doesn't lay inside [0..INT64_MAX] region), check it once in bdrv_check_bytes_request() and assert in each function we convert to int64_t.


s/doesn't//

-- 
Best regards,
Vladimir

Re: [PATCH v3 00/17] 64bit block-layer
Posted by Eric Blake 3 years, 4 months ago
On 12/1/20 10:07 AM, Vladimir Sementsov-Ogievskiy wrote:
> Hi!
> 
> I'm sorry, I should have pinged it, or resend, or suggest to pull at
> least a half long ago :(
> 
> I've rebased it on master and make some fixes.
> 
> What to do next? I can just resend. But I'm afraid that Eric's careful
> audits may be out of date: time passed, there is no guarantee that
> callers are not changed. Really sorry :(
> So r-b marks are not applicable as well, yes?

If you think the rebase has fundamentally changed things, then dropping
the r-b is safest.  I will probably spend a good time on the audit
again, but this time, I want to see the project through to completion,
and am willing to take patches through my NBD tree if Kevin or other
block maintainers do not have enough time to take it through a broader
block tree.  I can justify it because I have a specific patch in NBD
that will benefit from this audit - I want to rever 890cbccb08 in favor
of using saner 64-bit APIs throughout the block layer.  But I am also
aware that your patches touch more than NBD, so even if Kevin can't
commit to a full review, I will at least try to get his Acked-by.

> 
> But if I just resend it with no r-bs, is it feasible to review/merge it
> in a finite time? So that audits of patches will not become outdated?

Yes, let's agree to put a lot more effort into getting this series in
for 6.0.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org