[PULL 00/23] QEMU changes for 7.0 soft freeze

Paolo Bonzini posted 23 patches 2 years, 2 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220307181633.596898-1-pbonzini@redhat.com
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, Thomas Huth <thuth@redhat.com>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Beraldo Leal <bleal@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Alex Williamson <alex.williamson@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Cornelia Huck <cohuck@redhat.com>, Michael Roth <michael.roth@amd.com>, Konstantin Kostiuk <kkostiuk@redhat.com>, Marcelo Tosatti <mtosatti@redhat.com>, Richard Henderson <richard.henderson@linaro.org>, Eduardo Habkost <eduardo@habkost.net>, Sunil Muthuswamy <sunilmut@microsoft.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>
There is a newer version of this series
.gitlab-ci.d/buildtest.yml           |   2 -
accel/kvm/kvm-all.c                  |   7 +-
accel/stubs/kvm-stub.c               |   2 +-
hw/misc/ivshmem.c                    |   5 +-
hw/vfio/pci.c                        |   5 +-
hw/virtio/virtio-pci.c               |   4 +-
include/sysemu/kvm.h                 |  23 ++++++-
linux-headers/asm-x86/kvm.h          |   3 +
linux-headers/linux/kvm.h            |   4 ++
meson.build                          |   5 +-
qga/meson.build                      |   2 +-
qga/vss-win32/install.cpp            |   4 ++
qga/vss-win32/provider.cpp           |   4 ++
qga/vss-win32/vss-common.h           |   3 +-
scripts/kvm/vmxcap                   |   1 +
scripts/meson-buildoptions.sh        |   2 +-
target/i386/cpu.c                    |  84 +++++++++++++++++++++--
target/i386/cpu.h                    |  43 +++++++++++-
target/i386/kvm/kvm-cpu.c            |  11 +--
target/i386/kvm/kvm.c                | 125 ++++++++++++++++++++++++++++++-----
target/i386/kvm/kvm_i386.h           |   1 +
target/i386/machine.c                |  46 +++++++++++++
target/i386/tcg/seg_helper.c         |  49 +++++++++++++-
target/i386/tcg/sysemu/excp_helper.c |  40 +----------
target/i386/whpx/whpx-all.c          |  30 ++++++++-
target/i386/xsave_helper.c           |  28 ++++++++
tests/check-block.sh                 |   6 +-
tests/qemu-iotests/check             |   6 +-
tests/qemu-iotests/meson.build       |   1 -
tests/qemu-iotests/testenv.py        |  30 ++++-----
tests/qemu-iotests/testrunner.py     |  54 +++++++--------
31 files changed, 494 insertions(+), 136 deletions(-)
[PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Paolo Bonzini 2 years, 2 months ago
The following changes since commit 99c53410bc9d50e556f565b0960673cccb566452:

  Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2022-02-28' into staging (2022-03-01 13:25:54 +0000)

are available in the Git repository at:

  https://gitlab.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to d170dbb84f8ed8f8c9daa359d6d95cd02fd959b7:

  gitlab-ci: do not run tests with address sanitizer (2022-03-07 17:41:51 +0100)

----------------------------------------------------------------
* whpx fixes in preparation for GDB support (Ivan)
* VSS header fixes (Marc-André)
* 5-level EPT support (Vitaly)
* AMX support (Jing Liu & Yang Zhong)
* Bundle changes to MSI routes (Longpeng)
* More precise emulation of #SS (Gareth)
* Disable TAP output for check-block
* Disable ASAN testing
----------------------------------------------------------------

Gareth Webb (1):
      target/i386: Throw a #SS when loading a non-canonical IST

Ivan Shcherbakov (2):
      whpx: Fixed reporting of the CPU context to GDB for 64-bit
      whpx: Fixed incorrect CR8/TPR synchronization

Jing Liu (5):
      x86: Fix the 64-byte boundary enumeration for extended state
      x86: Add AMX XTILECFG and XTILEDATA components
      x86: Add XFD faulting bit for state components
      x86: Add AMX CPUIDs enumeration
      x86: add support for KVM_CAP_XSAVE2 and AMX state migration

Longpeng (Mike) (2):
      kvm-irqchip: introduce new API to support route change
      kvm/msi: do explicit commit when adding msi routes

Marc-André Lureau (3):
      meson: fix generic location of vss headers
      qga/vss-win32: check old VSS SDK headers
      qga/vss: update informative message about MinGW

Maxim Levitsky (1):
      KVM: SVM: always set MSR_AMD64_TSC_RATIO to default value

Paolo Bonzini (5):
      update meson-buildoptions.sh
      target/i386: only include bits in pg_mode if they are not ignored
      linux-headers: include missing changes from 5.17
      check-block: revert TAP output and reintroduce -makecheck
      gitlab-ci: do not run tests with address sanitizer

Vitaly Kuznetsov (2):
      vmxcap: Add 5-level EPT bit
      i386: Add Icelake-Server-v6 CPU model with 5-level EPT support

Yang Zhong (1):
      x86: Grant AMX permission for guest

Zeng Guang (1):
      x86: Support XFD and AMX xsave data migration

 .gitlab-ci.d/buildtest.yml           |   2 -
 accel/kvm/kvm-all.c                  |   7 +-
 accel/stubs/kvm-stub.c               |   2 +-
 hw/misc/ivshmem.c                    |   5 +-
 hw/vfio/pci.c                        |   5 +-
 hw/virtio/virtio-pci.c               |   4 +-
 include/sysemu/kvm.h                 |  23 ++++++-
 linux-headers/asm-x86/kvm.h          |   3 +
 linux-headers/linux/kvm.h            |   4 ++
 meson.build                          |   5 +-
 qga/meson.build                      |   2 +-
 qga/vss-win32/install.cpp            |   4 ++
 qga/vss-win32/provider.cpp           |   4 ++
 qga/vss-win32/vss-common.h           |   3 +-
 scripts/kvm/vmxcap                   |   1 +
 scripts/meson-buildoptions.sh        |   2 +-
 target/i386/cpu.c                    |  84 +++++++++++++++++++++--
 target/i386/cpu.h                    |  43 +++++++++++-
 target/i386/kvm/kvm-cpu.c            |  11 +--
 target/i386/kvm/kvm.c                | 125 ++++++++++++++++++++++++++++++-----
 target/i386/kvm/kvm_i386.h           |   1 +
 target/i386/machine.c                |  46 +++++++++++++
 target/i386/tcg/seg_helper.c         |  49 +++++++++++++-
 target/i386/tcg/sysemu/excp_helper.c |  40 +----------
 target/i386/whpx/whpx-all.c          |  30 ++++++++-
 target/i386/xsave_helper.c           |  28 ++++++++
 tests/check-block.sh                 |   6 +-
 tests/qemu-iotests/check             |   6 +-
 tests/qemu-iotests/meson.build       |   1 -
 tests/qemu-iotests/testenv.py        |  30 ++++-----
 tests/qemu-iotests/testrunner.py     |  54 +++++++--------
 31 files changed, 494 insertions(+), 136 deletions(-)
-- 
2.34.1


Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Thomas Huth 2 years, 1 month ago
On 07/03/2022 19.16, Paolo Bonzini wrote:
> The following changes since commit 99c53410bc9d50e556f565b0960673cccb566452:
> 
>    Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2022-02-28' into staging (2022-03-01 13:25:54 +0000)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/bonzini/qemu.git tags/for-upstream
> 
> for you to fetch changes up to d170dbb84f8ed8f8c9daa359d6d95cd02fd959b7:
> 
>    gitlab-ci: do not run tests with address sanitizer (2022-03-07 17:41:51 +0100)
> 
> ----------------------------------------------------------------
> * whpx fixes in preparation for GDB support (Ivan)
> * VSS header fixes (Marc-André)
> * 5-level EPT support (Vitaly)
> * AMX support (Jing Liu & Yang Zhong)
> * Bundle changes to MSI routes (Longpeng)
> * More precise emulation of #SS (Gareth)
> * Disable TAP output for check-block
> * Disable ASAN testing
> ----------------------------------------------------------------
[...]
> Paolo Bonzini (5):
>        update meson-buildoptions.sh
>        target/i386: only include bits in pg_mode if they are not ignored
>        linux-headers: include missing changes from 5.17
>        check-block: revert TAP output and reintroduce -makecheck

You now need to revert commit db4b2133b8d ('Quote "case not run" lines in 
TAP mode'), too, which has just been merged yesterday.

But instead of reverting, couldn't we simply go ahead with my series here 
instead to restore the output of failed iotests:

  https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg05031.html

?

  Thomas


Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Paolo Bonzini 2 years, 1 month ago
On 3/8/22 09:14, Thomas Huth wrote:
> You now need to revert commit db4b2133b8d ('Quote "case not run" lines 
> in TAP mode'), too, which has just been merged yesterday.
> 
> But instead of reverting, couldn't we simply go ahead with my series 
> here instead to restore the output of failed iotests:
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg05031.html

That one also drops the progress report of non-failed tests, so I'm not 
sure it's an improvement.  Also the commit message for that one should 
mention that it's a meson bug, IMO we should ask distros to backport the 
trivial fix to --verbose 
(https://github.com/mesonbuild/meson/commit/c3f145ca2b9f5a4cfac3ffe1de7d901a4e8aba10.patch).

Paolo

Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Thomas Huth 2 years, 1 month ago
On 08/03/2022 12.31, Paolo Bonzini wrote:
> On 3/8/22 09:14, Thomas Huth wrote:
>> You now need to revert commit db4b2133b8d ('Quote "case not run" lines in 
>> TAP mode'), too, which has just been merged yesterday.
>>
>> But instead of reverting, couldn't we simply go ahead with my series here 
>> instead to restore the output of failed iotests:
>>
>>   https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg05031.html
> 
> That one also drops the progress report of non-failed tests, so I'm not sure 
> it's an improvement.

That only works here anyway if I don't run "make check" with the "-jX" 
option ... which I hardly do, since it then takes forever to finish the 
testing. So at least for me that's not really a reason.

>  Also the commit message for that one should mention 
> that it's a meson bug, IMO we should ask distros to backport the trivial fix 
> to --verbose 
> (https://github.com/mesonbuild/meson/commit/c3f145ca2b9f5a4cfac3ffe1de7d901a4e8aba10.patch). 

Ok ... could you maybe ask the meson folks to include the fix in the 
upcoming stable releases?
... the meson master branch has been switched to Python 3.7 already, but 
AFAIU we still target Python 3.6 in QEMU, so I'm not sure whether we will be 
able to jump on the next main release with QEMU... we might need to stick 
with a 0.61.x release for a while in the future?

Since the fix is not available in any meson release yet and we're in QEMU 
softfreeze now, I'm now also not quite sure anymore whether we really should 
go forward with the TAP approach right now... maybe it's indeed better to 
revert the patches for QEMU 7.0 and sort it out later?

  Thomas


Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Paolo Bonzini 2 years, 1 month ago
On 3/9/22 10:46, Thomas Huth wrote:
>>
>> That one also drops the progress report of non-failed tests, so I'm 
>> not sure it's an improvement.
> 
> That only works here anyway if I don't run "make check" with the "-jX" 
> option ... which I hardly do, since it then takes forever to finish the 
> testing. So at least for me that's not really a reason.

The point of having a separate "make check-block" was to have the 
progress report ("--verbose --num-processes 1" gives the progress 
report, the same with --print-errorlogs doesn't).

> Ok ... could you maybe ask the meson folks to include the fix in the upcoming stable releases?

Did it, hopefully will be in 0.61.3.

> ... the meson master branch has been switched to Python 3.7 already,
> but AFAIU we still target Python 3.6 in QEMU, so I'm not sure whether
> we will be able to jump on the next main release with QEMU... we
> might need to stick with a 0.61.x release for a while in the future?
3.6 was EOL'd in December, so I expect that QEMU will bump the 
requirement sometime this year too.

Paolo
Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Thomas Huth 2 years, 1 month ago
On 09/03/2022 17.07, Paolo Bonzini wrote:
> On 3/9/22 10:46, Thomas Huth wrote:
>>>
>>> That one also drops the progress report of non-failed tests, so I'm not 
>>> sure it's an improvement.
>>
>> That only works here anyway if I don't run "make check" with the "-jX" 
>> option ... which I hardly do, since it then takes forever to finish the 
>> testing. So at least for me that's not really a reason.
> 
> The point of having a separate "make check-block" was to have the progress 
> report ("--verbose --num-processes 1" gives the progress report, the same 
> with --print-errorlogs doesn't).

Yes, understood, but again, that only works for me if I run:

  make check-block

But if I run:

  make check-block -j8

I only get one single line saying:

  [1/1] 🌓 qemu:block / qemu-iotests qcow2                   2s

And the progress report is only printed in one go at the end, after all 
tests have finished. Since I'm using -jX with X > 1 by default, the progress 
report that you get with -j1 is simply no advantage for me. Is your build 
behaving differently? ... then this maybe depends on the meson version? I 
think my build is using the one from the submodule, version 0.59.3.

>> Ok ... could you maybe ask the meson folks to include the fix in the 
>> upcoming stable releases?
> 
> Did it, hopefully will be in 0.61.3.

Great!

... but I guess this will be too late for QEMU 7.0 ?

I just sent out

  "[PATCH v4] tests: Do not treat the iotests as separate meson test target 
anymore"

in case we want to continue with the TAP mode for 7.0 (only updated the 
patch description) ... but since we're in freeze mode already, I'm also fine 
if you say that this is not appropriate for QEMU 7.0 anymore and that we 
should revert the TAP patches instead.

>> ... the meson master branch has been switched to Python 3.7 already,
>> but AFAIU we still target Python 3.6 in QEMU, so I'm not sure whether
>> we will be able to jump on the next main release with QEMU... we
>> might need to stick with a 0.61.x release for a while in the future?
> 3.6 was EOL'd in December, so I expect that QEMU will bump the requirement 
> sometime this year too.

Yes, that makes sense ... I was just thinking about RHEL 8 and its clones, 
which have Python 3.6 as default ... but I think they also ship Python 3.8 
as alternative, so I guess it should be fine to increase the minimum Python 
version for QEMU 7.1 ?

  Thomas


Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Paolo Bonzini 2 years, 1 month ago
On 3/10/22 08:56, Thomas Huth wrote:
> 
> Yes, understood, but again, that only works for me if I run:
> 
>   make check-block
> 
> But if I run:
> 
>   make check-block -j8
> 
> I only get one single line saying:
> 
>   [1/1] 🌓 qemu:block / qemu-iotests qcow2                   2s
> 
> And the progress report is only printed in one go at the end, after all 
> tests have finished. Since I'm using -jX with X > 1 by default, the 
> progress report that you get with -j1 is simply no advantage for me. Is 
> your build behaving differently? ... then this maybe depends on the 
> meson version? I think my build is using the one from the submodule, 
> version 0.59.3.

Ok, I'll check that out.

Paolo

Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Peter Maydell 2 years, 1 month ago
On Wed, 9 Mar 2022 at 09:47, Thomas Huth <thuth@redhat.com> wrote:
> Since the fix is not available in any meson release yet and we're in QEMU
> softfreeze now, I'm now also not quite sure anymore whether we really should
> go forward with the TAP approach right now... maybe it's indeed better to
> revert the patches for QEMU 7.0 and sort it out later?

I'm a bit lost about where this discussion thread ended up,
so to be clear: are you OK with me applying this pullreq
from Paolo, or do you want more time to talk through the
issue? (I have another half dozen pullreqs still to merge
so it's not like I'm waiting on just this one.)

thanks
-- PMM
Re: [PULL 00/23] QEMU changes for 7.0 soft freeze
Posted by Thomas Huth 2 years, 1 month ago
On 09/03/2022 19.44, Peter Maydell wrote:
> On Wed, 9 Mar 2022 at 09:47, Thomas Huth <thuth@redhat.com> wrote:
>> Since the fix is not available in any meson release yet and we're in QEMU
>> softfreeze now, I'm now also not quite sure anymore whether we really should
>> go forward with the TAP approach right now... maybe it's indeed better to
>> revert the patches for QEMU 7.0 and sort it out later?
> 
> I'm a bit lost about where this discussion thread ended up,
> so to be clear: are you OK with me applying this pullreq
> from Paolo, or do you want more time to talk through the
> issue? (I have another half dozen pullreqs still to merge
> so it's not like I'm waiting on just this one.)

This version of the pull request has a merge conflict and won't apply 
anyway, but Paolo already sent a v2 of the PR which does not have the TAP 
revert patch in it, so you can certainly go ahead and merge that v2. We 
should have a decision by next Tuesday hopefully whether we revert the TAP 
code or go ahead with it...

  Thomas