Hi all,

I would like feedback on whether Xen needs to flush every page in a
dom0less xen,static-mem bank before assigning it to a guest.

Static-memory support is currently enabled only on Arm, where I am using
and testing this change. Its implementation and acquisition path are in
common code, so this RFC is about the generic static-memory path.

The current code cleans and invalidates every page in the bank. This is a
safe default, but walking a large bank can add several seconds to boot even
though Xen only writes the pages used for the guest kernel, initrd, and
device tree.

Here, cleaning refers to cache maintenance, not boot scrubbing. With
bootscrub=1 or the default bootscrub=idle, Xen assigns dom0less static
banks before boot scrubbing starts. The pages are no longer free, so boot
scrubbing does not clear them.

My starting point is that if a platform permits the untouched pages to be
cleared before guest start, the guest cannot depend on their initial
contents. This is a platform policy assumption, not something Xen's boot
scrubbing establishes. Xen already cleans each page it writes during
domain construction.

Would it therefore be reasonable to let a platform skip the full-bank
flush, provided it can also guarantee that the untouched pages are absent
from all caches and that firmware, DMA, EL3 services, and other CPUs do not
write them before guest start?

The attached patch adds a default-on staticmem-cache-flush option.
Specifying no-staticmem-cache-flush opts into the faster path. The normal
flush remains the default, and static shared memory is unaffected.

Are these conditions enough to make the opt-out valid, or is there another
reason the full-bank flush is required?

I am using no-staticmem-cache-flush with bootscrub=0 on a 64-bit Arm
system with two dom0less guests. Both guests boot normally. Skipping the
flush reduced Xen startup from about five seconds to about one second.

Thanks for any advice or help with this.

Jan Setje-Eilers (1):
  static-memory: allow skipping the cache flush

 docs/misc/xen-command-line.pandoc      | 26 ++++++++++++++++++++++++++
 xen/common/device-tree/static-memory.c |  9 ++++++++-
 xen/common/page_alloc.c                |  6 ++++--
 xen/include/xen/mm.h                   |  2 ++
 4 files changed, 40 insertions(+), 3 deletions(-)


base-commit: 79225a0c77e13b693b4d2b903a88289704b79db6
-- 
2.47.3