[PULL 00/10] Migration 20230509 patches

Juan Quintela posted 10 patches 11 months, 3 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20230509191724.86159-1-quintela@redhat.com
Maintainers: Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Hailiang Zhang <zhanghailiang@xfusion.com>, "Dr. David Alan Gilbert" <dave@treblig.org>, Juan Quintela <quintela@redhat.com>, Peter Xu <peterx@redhat.com>, Leonardo Bras <leobras@redhat.com>, Eric Blake <eblake@redhat.com>, Markus Armbruster <armbru@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>
There is a newer version of this series
block/meson.build              |  2 +-
docs/COLO-FT.txt               |  1 +
hmp-commands.hx                |  2 ++
include/migration/colo.h       |  9 +++++-
migration/colo.c               | 57 +++++++++++-----------------------
migration/meson.build          |  6 ++--
migration/migration-hmp-cmds.c |  2 ++
migration/migration.c          | 35 ++++++++++++++-------
migration/migration.h          |  2 --
migration/multifd.c            | 11 +++----
migration/multifd.h            |  2 ++
migration/options.c            |  6 ++--
migration/ram.c                | 19 ++++++++++--
migration/ram.h                |  1 +
qapi/migration.json            |  9 ++++--
stubs/colo.c                   | 39 +++++++++++++++++++++++
stubs/meson.build              |  1 +
17 files changed, 131 insertions(+), 73 deletions(-)
create mode 100644 stubs/colo.c
[PULL 00/10] Migration 20230509 patches
Posted by Juan Quintela 11 months, 3 weeks ago
The following changes since commit 271477b59e723250f17a7e20f139262057921b6a:

  Merge tag 'compression-code-pull-request' of https://gitlab.com/juan.quintela/qemu into staging (2023-05-08 20:38:05 +0100)

are available in the Git repository at:

  https://gitlab.com/juan.quintela/qemu.git tags/migration-20230509-pull-request

for you to fetch changes up to 5f43d297bc2b9530805ad8602c6e2ea284b08628:

  migration: block incoming colo when capability is disabled (2023-05-09 20:52:21 +0200)

----------------------------------------------------------------
Migration Pull request (20230509 vintage)

Hi
In this PULL request:
- 1st part of colo support for multifd (lukas)
- 1st part of disabling colo option (vladimir)

Please, apply.

----------------------------------------------------------------

Lukas Straub (3):
  ram: Add public helper to set colo bitmap
  ram: Let colo_flush_ram_cache take the bitmap_mutex
  multifd: Add the ramblock to MultiFDRecvParams

Vladimir Sementsov-Ogievskiy (7):
  block/meson.build: prefer positive condition for replication
  colo: make colo_checkpoint_notify static and provide simpler API
  build: move COLO under CONFIG_REPLICATION
  migration: drop colo_incoming_thread from MigrationIncomingState
  migration: process_incoming_migration_co: simplify code flow around
    ret
  migration: disallow change capabilities in COLO state
  migration: block incoming colo when capability is disabled

 block/meson.build              |  2 +-
 docs/COLO-FT.txt               |  1 +
 hmp-commands.hx                |  2 ++
 include/migration/colo.h       |  9 +++++-
 migration/colo.c               | 57 +++++++++++-----------------------
 migration/meson.build          |  6 ++--
 migration/migration-hmp-cmds.c |  2 ++
 migration/migration.c          | 35 ++++++++++++++-------
 migration/migration.h          |  2 --
 migration/multifd.c            | 11 +++----
 migration/multifd.h            |  2 ++
 migration/options.c            |  6 ++--
 migration/ram.c                | 19 ++++++++++--
 migration/ram.h                |  1 +
 qapi/migration.json            |  9 ++++--
 stubs/colo.c                   | 39 +++++++++++++++++++++++
 stubs/meson.build              |  1 +
 17 files changed, 131 insertions(+), 73 deletions(-)
 create mode 100644 stubs/colo.c

-- 
2.40.0
Re: [PULL 00/10] Migration 20230509 patches
Posted by Richard Henderson 11 months, 3 weeks ago
On 5/9/23 20:17, Juan Quintela wrote:
> The following changes since commit 271477b59e723250f17a7e20f139262057921b6a:
> 
>    Merge tag 'compression-code-pull-request' of https://gitlab.com/juan.quintela/qemu into staging (2023-05-08 20:38:05 +0100)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/juan.quintela/qemu.git tags/migration-20230509-pull-request
> 
> for you to fetch changes up to 5f43d297bc2b9530805ad8602c6e2ea284b08628:
> 
>    migration: block incoming colo when capability is disabled (2023-05-09 20:52:21 +0200)
> 
> ----------------------------------------------------------------
> Migration Pull request (20230509 vintage)
> 
> Hi
> In this PULL request:
> - 1st part of colo support for multifd (lukas)
> - 1st part of disabling colo option (vladimir)
> 
> Please, apply.

Build failures.

https://gitlab.com/qemu-project/qemu/-/jobs/4257605099#L2241

    85 | void colo_record_bitmap(RAMBlock *block, ram_addr_t *normal, uint normal_num);
       |                                                              ^~~~
       |                                                              u_int


r~
Re: [PULL 00/10] Migration 20230509 patches
Posted by Juan Quintela 11 months, 3 weeks ago
Richard Henderson <richard.henderson@linaro.org> wrote:
> On 5/9/23 20:17, Juan Quintela wrote:
>> The following changes since commit 271477b59e723250f17a7e20f139262057921b6a:
>>    Merge tag 'compression-code-pull-request' of
>> https://gitlab.com/juan.quintela/qemu into staging (2023-05-08
>> 20:38:05 +0100)
>> are available in the Git repository at:
>>    https://gitlab.com/juan.quintela/qemu.git
>> tags/migration-20230509-pull-request
>> for you to fetch changes up to
>> 5f43d297bc2b9530805ad8602c6e2ea284b08628:
>>    migration: block incoming colo when capability is disabled
>> (2023-05-09 20:52:21 +0200)
>> ----------------------------------------------------------------
>> Migration Pull request (20230509 vintage)
>> Hi
>> In this PULL request:
>> - 1st part of colo support for multifd (lukas)
>> - 1st part of disabling colo option (vladimir)
>> Please, apply.
>
> Build failures.
>
> https://gitlab.com/qemu-project/qemu/-/jobs/4257605099#L2241
>
>    85 | void colo_record_bitmap(RAMBlock *block, ram_addr_t *normal, uint normal_num);
>       |                                                              ^~~~
>       |                                                              u_int
>

Grrr

And the worst thing is that hate those types, tried to get then out
long, long ago for a similar problem.

Will resend, sorry for the noise.

Later, Juan.
Re: [PULL 00/10] Migration 20230509 patches
Posted by Richard Henderson 11 months, 3 weeks ago
On 5/10/23 13:20, Juan Quintela wrote:
> Richard Henderson <richard.henderson@linaro.org> wrote:
>> On 5/9/23 20:17, Juan Quintela wrote:
>>> The following changes since commit 271477b59e723250f17a7e20f139262057921b6a:
>>>     Merge tag 'compression-code-pull-request' of
>>> https://gitlab.com/juan.quintela/qemu into staging (2023-05-08
>>> 20:38:05 +0100)
>>> are available in the Git repository at:
>>>     https://gitlab.com/juan.quintela/qemu.git
>>> tags/migration-20230509-pull-request
>>> for you to fetch changes up to
>>> 5f43d297bc2b9530805ad8602c6e2ea284b08628:
>>>     migration: block incoming colo when capability is disabled
>>> (2023-05-09 20:52:21 +0200)
>>> ----------------------------------------------------------------
>>> Migration Pull request (20230509 vintage)
>>> Hi
>>> In this PULL request:
>>> - 1st part of colo support for multifd (lukas)
>>> - 1st part of disabling colo option (vladimir)
>>> Please, apply.
>>
>> Build failures.
>>
>> https://gitlab.com/qemu-project/qemu/-/jobs/4257605099#L2241
>>
>>     85 | void colo_record_bitmap(RAMBlock *block, ram_addr_t *normal, uint normal_num);
>>        |                                                              ^~~~
>>        |                                                              u_int
>>
> 
> Grrr
> 
> And the worst thing is that hate those types, tried to get then out
> long, long ago for a similar problem.

Where do these types come from, and can we poison them on the qemu side?


r~
Re: [PULL 00/10] Migration 20230509 patches
Posted by Juan Quintela 11 months, 3 weeks ago
Richard Henderson <richard.henderson@linaro.org> wrote:
> On 5/10/23 13:20, Juan Quintela wrote:
>> Richard Henderson <richard.henderson@linaro.org> wrote:
>>> On 5/9/23 20:17, Juan Quintela wrote:
>>>> The following changes since commit 271477b59e723250f17a7e20f139262057921b6a:
>>>>     Merge tag 'compression-code-pull-request' of
>>>> https://gitlab.com/juan.quintela/qemu into staging (2023-05-08
>>>> 20:38:05 +0100)
>>>> are available in the Git repository at:
>>>>     https://gitlab.com/juan.quintela/qemu.git
>>>> tags/migration-20230509-pull-request
>>>> for you to fetch changes up to
>>>> 5f43d297bc2b9530805ad8602c6e2ea284b08628:
>>>>     migration: block incoming colo when capability is disabled
>>>> (2023-05-09 20:52:21 +0200)
>>>> ----------------------------------------------------------------
>>>> Migration Pull request (20230509 vintage)
>>>> Hi
>>>> In this PULL request:
>>>> - 1st part of colo support for multifd (lukas)
>>>> - 1st part of disabling colo option (vladimir)
>>>> Please, apply.
>>>
>>> Build failures.
>>>
>>> https://gitlab.com/qemu-project/qemu/-/jobs/4257605099#L2241
>>>
>>>     85 | void colo_record_bitmap(RAMBlock *block, ram_addr_t *normal, uint normal_num);
>>>        |                                                              ^~~~
>>>        |                                                              u_int
>>>
>> Grrr
>> And the worst thing is that hate those types, tried to get then out
>> long, long ago for a similar problem.

While I was not looking, some of them were corrected:

commit d7df0b41dc38327388c3f19fdf4246793d4a1e4b
Author: Marc-André Lureau <marcandre.lureau@redhat.com>
Date:   Thu Jan 17 15:43:53 2019 +0400

    slirp: prefer c99 types over BSD kind

    Replace:
    - u_char -> uint8_t
    - u_short -> uint16_t
    - u_long -> uint32_t
    - u_int -> unsigned
    - caddr_t -> char *

> Where do these types come from, and can we poison them on the qemu side?

grep " uint;" on my system includes.  I know that there are more
creative ways to define it.

/usr/include/ffi-x86_64.h\0278:  ffi_arg   uint;
/usr/include/sys/types.h\0150:typedef unsigned int uint;

#ifdef __USE_MISC
/* Old compatibility names for C types.  */
typedef unsigned long int ulong;
typedef unsigned short int ushort;
typedef unsigned int uint;
#endif

I guess we get those someway.

/usr/include/nspr4/obsolete/protypes.h\052:typedef PRUintn uint;
/usr/include/mysql/server/my_global.h\0465:typedef unsigned int uint;
/usr/include/boost/iostreams/filter/zlib.hpp\047:typedef uint32_t uint;
/usr/include/qt5/QtCore/qglobal.h\0275:typedef unsigned int uint;

in qt it is defined for everything.

If I disable the ones in sys/types.h

I got:

cc -m64 -mcx16 -Ilibqemu-loongarch64-linux-user.fa.p -I. -I../../../../mnt/code/qemu/full -Itarget/loongarch -I../../../../mnt/code/qemu/full/target/loongarch -I../../../../mnt/code/qemu/full/common-user/host/x86_64 -I../../../../mnt/code/qemu/full/linux-user/include/host/x86_64 -I../../../../mnt/code/qemu/full/linux-user/include -Ilinux-user -I../../../../mnt/code/qemu/full/linux-user -I../../../../mnt/code/qemu/full/linux-user/loongarch64 -Iqapi -Itrace -Iui -Iui/shader -I/usr/include/capstone -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/sysprof-4 -fdiagnostics-color=auto -Wall -Winvalid-pch -Werror -std=gnu11 -O2 -g -isystem /mnt/code/qemu/full/linux-headers -isystem linux-headers -iquote . -iquote /mnt/code/qemu/full -iquote /mnt/code/qemu/full/include -iquote /mnt/code/qemu/full/tcg/i386 -pthread -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -fno-strict-aliasing -fno-common -fwrapv -Wundef -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wredundant-decls -Wold-style-declaration -Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -Wexpansion-to-defined -Wimplicit-fallthrough=2 -Wmissing-format-attribute -Wno-missing-include-dirs -Wno-shift-negative-value -Wno-psabi -fstack-protector-strong -fPIE -isystem../../../../mnt/code/qemu/full/linux-headers -isystemlinux-headers -DNEED_CPU_H '-DCONFIG_TARGET="loongarch64-linux-user-config-target.h"' '-DCONFIG_DEVICES="loongarch64-linux-user-config-devices.h"' -MD -MQ libqemu-loongarch64-linux-user.fa.p/linux-user_syscall.c.o -MF libqemu-loongarch64-linux-user.fa.p/linux-user_syscall.c.o.d -o libqemu-loongarch64-linux-user.fa.p/linux-user_syscall.c.o -c ../../../../mnt/code/qemu/full/linux-user/syscall.c
../../../../mnt/code/qemu/full/linux-user/syscall.c:317:32: error: unknown type name ‘uint’; did you mean ‘guint’?
  317 | _syscall3(int, sys_getdents64, uint, fd, struct linux_dirent64 *, dirp, uint, count);
      |                                ^~~~

There are some other friends that fail in linux-user/syscall.c

I will post an RFC with my findings.

Later, Juan.
Re: [PULL 00/10] Migration 20230509 patches
Posted by Richard Henderson 11 months, 3 weeks ago
On 5/10/23 15:08, Juan Quintela wrote:
> grep " uint;" on my system includes.  I know that there are more
> creative ways to define it.
> 
> /usr/include/ffi-x86_64.h\0278:  ffi_arg   uint;

Thankfully only a structure member.  :-)

> /usr/include/sys/types.h\0150:typedef unsigned int uint;

Oof.

> /usr/include/nspr4/obsolete/protypes.h\052:typedef PRUintn uint;
> /usr/include/mysql/server/my_global.h\0465:typedef unsigned int uint;
> /usr/include/boost/iostreams/filter/zlib.hpp\047:typedef uint32_t uint;
> /usr/include/qt5/QtCore/qglobal.h\0275:typedef unsigned int uint;
> 
> in qt it is defined for everything.

Ok.

> ../../../../mnt/code/qemu/full/linux-user/syscall.c:317:32: error: unknown type name ‘uint’; did you mean ‘guint’?
>    317 | _syscall3(int, sys_getdents64, uint, fd, struct linux_dirent64 *, dirp, uint, count);
>        |                                ^~~~

Fixable.

> I will post an RFC with my findings.

Thanks.


r~