From nobody Tue May 14 11:15:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1693905236; cv=none; d=zohomail.com; s=zohoarc; b=kHJqYbunb/5Sjp8bSSsjEBzfMLzMyUZ7YAxfgq0Te2uMu1+OAOQe8p71gQDm4w3oWt+hOGfKcMLt5sT/NjrRsBNIFIvBhjlD82DeIPqgADLuo40W2KXK1zRJ4j0p+apCXw9L2QoFga5mwQQwCFMovG37GN3qL9076P5lvaL0Z9A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1693905236; h=Content-Type:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Sender:Subject:To; bh=WXZlu4c/41I4Egc5HxrkBr2W+PgYgrbGSqLZOwsuFXk=; b=D/iKw3Tt77qupdSK1ofv0Us/VT58Z074GI/O4zVDfYkH9KU+HIgfFCJ2/1DcyaFtAIVGQRDjQLRUy7XsxC0sRAxm6DJC84MPNSXUGKTQ49idheWALcdyQNRJSXyVFRfvsBlCVR3nCKVbgtb3qpsOb43AnengV3JbYD091tHBcgo= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16939052362389.599741003241547; Tue, 5 Sep 2023 02:13:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdS7d-00025S-RO; Tue, 05 Sep 2023 05:13:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdS7c-00024e-7z; Tue, 05 Sep 2023 05:13:20 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdS7V-0003xN-E1; Tue, 05 Sep 2023 05:13:19 -0400 Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.201]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Rg0Cd50gpz689yQ; Tue, 5 Sep 2023 17:13:01 +0800 (CST) Received: from A2006125610.china.huawei.com (10.202.227.178) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 5 Sep 2023 10:13:03 +0100 To: , CC: , , , , , Subject: [PATCH v4] arm/kvm: Enable support for KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE Date: Tue, 5 Sep 2023 10:12:46 +0100 Message-ID: <20230905091246.1931-1-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.202.227.178] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=shameerali.kolothum.thodi@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) AC_FROM_MANY_DOTS=0.001, BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Shameer Kolothum From: Shameer Kolothum via Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1693905237421100003 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that we have Eager Page Split support added for ARM in the kernel, enable it in Qemu. This adds, -eager-split-size to -accel sub-options to set the eager page split chunk = size. -enable KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE. The chunk size specifies how many pages to break at a time, using a single allocation. Bigger the chunk size, more pages need to be allocated ahead of time. Reviewed-by: Gavin Shan Signed-off-by: Shameer Kolothum --- Changes: v3: https://lore.kernel.org/qemu-devel/20230830114818.641-1-shameerali.kolo= thum.thodi@huawei.com/ -Added R-by by Gavin and replaced kvm_arm_eager_split_size_valid() with a direct check. v2: https://lore.kernel.org/qemu-devel/20230815092709.1290-1-shameerali.kol= othum.thodi@huawei.com/ -Addressed commenst from Gavin. RFC v1: https://lore.kernel.org/qemu-devel/20230725150002.621-1-shameerali.= kolothum.thodi@huawei.com/ -Updated qemu-options.hx with description -Addressed review comments from Peter and Gavin(Thanks). --- accel/kvm/kvm-all.c | 1 + include/sysemu/kvm_int.h | 1 + qemu-options.hx | 15 ++++++++++ target/arm/kvm.c | 61 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 78 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 2ba7521695..ff1578bb32 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -3763,6 +3763,7 @@ static void kvm_accel_instance_init(Object *obj) /* KVM dirty ring is by default off */ s->kvm_dirty_ring_size =3D 0; s->kvm_dirty_ring_with_bitmap =3D false; + s->kvm_eager_split_size =3D 0; s->notify_vmexit =3D NOTIFY_VMEXIT_OPTION_RUN; s->notify_window =3D 0; s->xen_version =3D 0; diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index 511b42bde5..a5b9122cb8 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -116,6 +116,7 @@ struct KVMState uint64_t kvm_dirty_ring_bytes; /* Size of the per-vcpu dirty ring */ uint32_t kvm_dirty_ring_size; /* Number of dirty GFNs per ring */ bool kvm_dirty_ring_with_bitmap; + uint64_t kvm_eager_split_size; /* Eager Page Splitting chunk size */ struct KVMDirtyRingReaper reaper; NotifyVmexitOption notify_vmexit; uint32_t notify_window; diff --git a/qemu-options.hx b/qemu-options.hx index 29b98c3d4c..2e70704ee8 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -186,6 +186,7 @@ DEF("accel", HAS_ARG, QEMU_OPTION_accel, " split-wx=3Don|off (enable TCG split w^x mapping)\n" " tb-size=3Dn (TCG translation block cache size)\n" " dirty-ring-size=3Dn (KVM dirty ring GFN count, defaul= t 0)\n" + " eager-split-size=3Dn (KVM Eager Page Split chunk size= , default 0, disabled. ARM only)\n" " notify-vmexit=3Drun|internal-error|disable,notify-win= dow=3Dn (enable notify VM exit and set notify window, x86 only)\n" " thread=3Dsingle|multi (enable multi-threaded TCG)\n",= QEMU_ARCH_ALL) SRST @@ -244,6 +245,20 @@ SRST is disabled (dirty-ring-size=3D0). When enabled, KVM will instead record dirty pages in a bitmap. =20 + ``eager-split-size=3Dn`` + KVM implements dirty page logging at the PAGE_SIZE granularity and + enabling dirty-logging on a huge-page requires breaking it into + PAGE_SIZE pages in the first place. KVM on ARM does this splitting + lazily by default. There are performance benefits in doing huge-pa= ge + split eagerly, especially in situations where TLBI costs associated + with break-before-make sequences are considerable and also if guest + workloads are read intensive. The size here specifies how many pag= es + to break at a time and needs to be a valid block size which is + 1GB/2MB/4KB, 32MB/16KB and 512MB/64KB for 4KB/16KB/64KB PAGE_SIZE + respectively. Be wary of specifying a higher size as it will have = an + impact on the memory. By default, this feature is disabled + (eager-split-size=3D0). + ``notify-vmexit=3Drun|internal-error|disable,notify-window=3Dn`` Enables or disables notify VM exit support on x86 host and specify the corresponding notify window to trigger the VM exit if enabled. diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 23aeb09949..b66b936a95 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -30,6 +30,7 @@ #include "exec/address-spaces.h" #include "hw/boards.h" #include "hw/irq.h" +#include "qapi/visitor.h" #include "qemu/log.h" =20 const KVMCapabilityInfo kvm_arch_required_capabilities[] =3D { @@ -287,6 +288,26 @@ int kvm_arch_init(MachineState *ms, KVMState *s) } } =20 + if (s->kvm_eager_split_size) { + uint32_t sizes; + + sizes =3D kvm_vm_check_extension(s, KVM_CAP_ARM_SUPPORTED_BLOCK_SI= ZES); + if (!sizes) { + s->kvm_eager_split_size =3D 0; + warn_report("Eager Page Split support not available"); + } else if (!(s->kvm_eager_split_size & sizes)) { + error_report("Eager Page Split requested chunk size not valid"= ); + ret =3D -EINVAL; + } else { + ret =3D kvm_vm_enable_cap(s, KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZ= E, 0, + s->kvm_eager_split_size); + if (ret < 0) { + error_report("Enabling of Eager Page Split failed: %s", + strerror(-ret)); + } + } + } + kvm_arm_init_debug(s); =20 return ret; @@ -1069,6 +1090,46 @@ bool kvm_arch_cpu_check_are_resettable(void) return true; } =20 +static void kvm_arch_get_eager_split_size(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + KVMState *s =3D KVM_STATE(obj); + uint64_t value =3D s->kvm_eager_split_size; + + visit_type_size(v, name, &value, errp); +} + +static void kvm_arch_set_eager_split_size(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + KVMState *s =3D KVM_STATE(obj); + uint64_t value; + + if (s->fd !=3D -1) { + error_setg(errp, "Unable to set early-split-size after KVM has bee= n initialized"); + return; + } + + if (!visit_type_size(v, name, &value, errp)) { + return; + } + + if (value && !is_power_of_2(value)) { + error_setg(errp, "early-split-size must be a power of two"); + return; + } + + s->kvm_eager_split_size =3D value; +} + void kvm_arch_accel_class_init(ObjectClass *oc) { + object_class_property_add(oc, "eager-split-size", "size", + kvm_arch_get_eager_split_size, + kvm_arch_set_eager_split_size, NULL, NULL); + + object_class_property_set_description(oc, "eager-split-size", + "Eager Page Split chunk size for hugepages. (default: 0, disabled)= "); } --=20 2.34.1