[Qemu-devel] [PATCH 0/9] spapr: Clean up pagesize handling

David Gibson posted 9 patches 5 years, 10 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20180618063606.2513-1-david@gibson.dropbear.id.au
Test checkpatch failed
Test docker-mingw@fedora passed
Test docker-quick@centos7 passed
Test s390x passed
hw/ppc/spapr.c          |  45 +++++++-----
hw/ppc/spapr_caps.c     | 156 ++++++++++++++++++++++++++++++++++++----
hw/ppc/spapr_cpu_core.c |   4 ++
include/hw/ppc/spapr.h  |  11 ++-
target/ppc/compat.c     |  27 +++++--
target/ppc/cpu.h        |   4 ++
target/ppc/kvm.c        | 146 ++++++++++++++++++-------------------
target/ppc/kvm_ppc.h    |  11 ++-
target/ppc/mmu-hash64.c |  59 +++++++++++++++
target/ppc/mmu-hash64.h |   3 +
10 files changed, 349 insertions(+), 117 deletions(-)
[Qemu-devel] [PATCH 0/9] spapr: Clean up pagesize handling
Posted by David Gibson 5 years, 10 months ago
Currently the "pseries" machine type will (usually) advertise
different pagesizes to the guest when running under KVM and TCG, which
is not how things are supposed to work.

This comes from poor handling of hardware limitations which mean that
under KVM HV the guest is unable to use pagesizes larger than those
backing the guest's RAM on the host side.

The new scheme turns things around by having an explicit machine
parameter controlling the largest page size that the guest is allowed
to use.  This limitation applies regardless of accelerator.  When
we're running on KVM HV we ensure that our backing pages are adequate
to supply the requested guest page sizes, rather than adjusting the
guest page sizes based on what KVM can supply.

This means that in order to use hugepages in a PAPR guest it's
necessary to add a "cap-hpt-max-page-size=16m" machine parameter as
well as setting the mem-path correctly.  This is a bit more work on
the user and/or management side, but results in consistent behaviour
so I think it's worth it.

Longer term, we might also use this parameter to control IOMMU page
sizes.  But, I'm still working out how restrictions deriving from the
guest kernel, host kernel and hardware capabilities all interact here.

This applies on top of my ppc-for-3.0 tree.

Changes since RFC:
 * Add preliminary cleanups to allow us to evaluate effective
   capabilities levels earlier.
 * Don't try to remove double resetting of cpus.  It doesn't quite
   work, and is no longer necessary with the above.
 * Some user-friendliness improvements: use "hpt-max-page-size"
   instead of the cryptic "hpt-mps", and take an actual page size
   (allowing k/m/g suffixies) instead of a shift

David Gibson (9):
  target/ppc: Allow cpu compatiblity checks based on type, not instance
  spapr: Compute effective capability values earlier
  spapr: Add cpu_apply hook to capabilities
  target/ppc: Add kvmppc_hpt_needs_host_contiguous_pages() helper
  spapr: Maximum (HPT) pagesize property
  spapr: Use maximum page size capability to simplify memory backend
    checking
  target/ppc: Add ppc_hash64_filter_pagesizes()
  spapr: Limit available pagesizes to provide a consistent guest
    environment
  spapr: Don't rewrite mmu capabilities in KVM mode

 hw/ppc/spapr.c          |  45 +++++++-----
 hw/ppc/spapr_caps.c     | 156 ++++++++++++++++++++++++++++++++++++----
 hw/ppc/spapr_cpu_core.c |   4 ++
 include/hw/ppc/spapr.h  |  11 ++-
 target/ppc/compat.c     |  27 +++++--
 target/ppc/cpu.h        |   4 ++
 target/ppc/kvm.c        | 146 ++++++++++++++++++-------------------
 target/ppc/kvm_ppc.h    |  11 ++-
 target/ppc/mmu-hash64.c |  59 +++++++++++++++
 target/ppc/mmu-hash64.h |   3 +
 10 files changed, 349 insertions(+), 117 deletions(-)

-- 
2.17.1


Re: [Qemu-devel] [PATCH 0/9] spapr: Clean up pagesize handling
Posted by David Gibson 5 years, 10 months ago
On Mon, Jun 18, 2018 at 04:35:57PM +1000, David Gibson wrote:
> Currently the "pseries" machine type will (usually) advertise
> different pagesizes to the guest when running under KVM and TCG, which
> is not how things are supposed to work.
> 
> This comes from poor handling of hardware limitations which mean that
> under KVM HV the guest is unable to use pagesizes larger than those
> backing the guest's RAM on the host side.
> 
> The new scheme turns things around by having an explicit machine
> parameter controlling the largest page size that the guest is allowed
> to use.  This limitation applies regardless of accelerator.  When
> we're running on KVM HV we ensure that our backing pages are adequate
> to supply the requested guest page sizes, rather than adjusting the
> guest page sizes based on what KVM can supply.
> 
> This means that in order to use hugepages in a PAPR guest it's
> necessary to add a "cap-hpt-max-page-size=16m" machine parameter as
> well as setting the mem-path correctly.  This is a bit more work on
> the user and/or management side, but results in consistent behaviour
> so I think it's worth it.
> 
> Longer term, we might also use this parameter to control IOMMU page
> sizes.  But, I'm still working out how restrictions deriving from the
> guest kernel, host kernel and hardware capabilities all interact here.
> 
> This applies on top of my ppc-for-3.0 tree.

Greg, Cédric, could you try to review this series pretty soon?

I'd really like to get it merged, because it's the basis for a number
of fixes for assorted problems with hugepage behaviour.

> 
> Changes since RFC:
>  * Add preliminary cleanups to allow us to evaluate effective
>    capabilities levels earlier.
>  * Don't try to remove double resetting of cpus.  It doesn't quite
>    work, and is no longer necessary with the above.
>  * Some user-friendliness improvements: use "hpt-max-page-size"
>    instead of the cryptic "hpt-mps", and take an actual page size
>    (allowing k/m/g suffixies) instead of a shift
> 
> David Gibson (9):
>   target/ppc: Allow cpu compatiblity checks based on type, not instance
>   spapr: Compute effective capability values earlier
>   spapr: Add cpu_apply hook to capabilities
>   target/ppc: Add kvmppc_hpt_needs_host_contiguous_pages() helper
>   spapr: Maximum (HPT) pagesize property
>   spapr: Use maximum page size capability to simplify memory backend
>     checking
>   target/ppc: Add ppc_hash64_filter_pagesizes()
>   spapr: Limit available pagesizes to provide a consistent guest
>     environment
>   spapr: Don't rewrite mmu capabilities in KVM mode
> 
>  hw/ppc/spapr.c          |  45 +++++++-----
>  hw/ppc/spapr_caps.c     | 156 ++++++++++++++++++++++++++++++++++++----
>  hw/ppc/spapr_cpu_core.c |   4 ++
>  include/hw/ppc/spapr.h  |  11 ++-
>  target/ppc/compat.c     |  27 +++++--
>  target/ppc/cpu.h        |   4 ++
>  target/ppc/kvm.c        | 146 ++++++++++++++++++-------------------
>  target/ppc/kvm_ppc.h    |  11 ++-
>  target/ppc/mmu-hash64.c |  59 +++++++++++++++
>  target/ppc/mmu-hash64.h |   3 +
>  10 files changed, 349 insertions(+), 117 deletions(-)
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [PATCH 0/9] spapr: Clean up pagesize handling
Posted by no-reply@patchew.org 5 years, 10 months ago
Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20180618063606.2513-1-david@gibson.dropbear.id.au
Subject: [Qemu-devel] [PATCH 0/9] spapr: Clean up pagesize handling

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]               patchew/20180618063606.2513-1-david@gibson.dropbear.id.au -> patchew/20180618063606.2513-1-david@gibson.dropbear.id.au
Switched to a new branch 'test'
0c66386fb1 spapr: Don't rewrite mmu capabilities in KVM mode
787addd355 spapr: Limit available pagesizes to provide a consistent guest environment
f6b4366e24 target/ppc: Add ppc_hash64_filter_pagesizes()
143ee46ef0 spapr: Use maximum page size capability to simplify memory backend checking
7bb6e85e44 spapr: Maximum (HPT) pagesize property
7a43ed59ce target/ppc: Add kvmppc_hpt_needs_host_contiguous_pages() helper
9bff9c15e0 spapr: Add cpu_apply hook to capabilities
4c9c79b967 spapr: Compute effective capability values earlier
1321ab423b target/ppc: Allow cpu compatiblity checks based on type, not instance

=== OUTPUT BEGIN ===
Checking PATCH 1/9: target/ppc: Allow cpu compatiblity checks based on type, not instance...
Checking PATCH 2/9: spapr: Compute effective capability values earlier...
Checking PATCH 3/9: spapr: Add cpu_apply hook to capabilities...
Checking PATCH 4/9: target/ppc: Add kvmppc_hpt_needs_host_contiguous_pages() helper...
Checking PATCH 5/9: spapr: Maximum (HPT) pagesize property...
Checking PATCH 6/9: spapr: Use maximum page size capability to simplify memory backend checking...
Checking PATCH 7/9: target/ppc: Add ppc_hash64_filter_pagesizes()...
Checking PATCH 8/9: spapr: Limit available pagesizes to provide a consistent guest environment...
WARNING: line over 80 characters
#28: FILE: hw/ppc/spapr_caps.c:337:
+static bool spapr_pagesize_cb(void *opaque, uint32_t seg_pshift, uint32_t pshift)

total: 0 errors, 1 warnings, 45 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 9/9: spapr: Don't rewrite mmu capabilities in KVM mode...
ERROR: Error messages should not contain newlines
#212: FILE: target/ppc/kvm.c:505:
+"KVM can't supply 64kiB CI pages, which guest expects\n");

total: 1 errors, 0 warnings, 201 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com