From nobody Mon Apr 29 02:19:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1564738791; cv=none; d=zoho.com; s=zohoarc; b=ZmbxUV2DaHrR10yBbPd+cJrib/Yu5mayxQ0IX1t1Qm8XnB4lubBW5JSLf6cQkfUQAJ4qmVRMputB3WpIWqVW/Nht8JPKBj7dcR6A0NmI6C0RufsLai+jDP7dUuMbi1ydOFM0iCFJllw1T0YnzwkNq87xifmjZnIi9i09iD7HFqg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1564738791; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=2LzDeZc2KbtLznVCHz0RuGZnBvy5l1ZgV9vi5xRDkcQ=; b=hN4zFBUCz9nzb1hU04UvHI3cRSqiVcQ0LToaolUGaIoMDsbP4+VbaTseQqy146Z+vDFMa0yhRv4lY0jMJkUFNUbTpwwhTFoug4nBy2T7egnBeo6ZBZO311kuOe08H0BLpSY+YcbH/fc/lffEu15MsJP9SG8zhX6tPPSr83wpC2Q= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1564738791709249.9025070375801; Fri, 2 Aug 2019 02:39:51 -0700 (PDT) Received: from localhost ([::1]:33046 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU2O-0008FB-HZ for importer@patchew.org; Fri, 02 Aug 2019 05:39:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55588) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU1j-0006zI-BL for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1htU1h-0000x3-Kk for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37668) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1htU1h-0000wc-DW; Fri, 02 Aug 2019 05:39:05 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AED8CC0586AE; Fri, 2 Aug 2019 09:39:04 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4181519C6A; Fri, 2 Aug 2019 09:39:03 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Date: Fri, 2 Aug 2019 05:38:51 -0400 Message-Id: <20190802093854.5343-2-imammedo@redhat.com> In-Reply-To: <20190802093854.5343-1-imammedo@redhat.com> References: <20190802093854.5343-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 02 Aug 2019 09:39:04 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH RFC v2 1/4] hw: add compat machines for 4.2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Cornelia Huck Add 4.2 machine types for arm/i440fx/q35/s390x/spapr. For i440fx and q35, unversioned cpu models are still translated to -v1, as 0788a56bd1ae ("i386: Make unversioned CPU models be aliases") states this should only transition to the latest cpu model version in 4.3 (or later). Signed-off-by: Cornelia Huck --- include/hw/boards.h | 3 +++ include/hw/i386/pc.h | 3 +++ hw/arm/virt.c | 9 ++++++++- hw/core/machine.c | 3 +++ hw/i386/pc.c | 3 +++ hw/i386/pc_piix.c | 14 +++++++++++++- hw/i386/pc_q35.c | 13 ++++++++++++- hw/ppc/spapr.c | 15 +++++++++++++-- hw/s390x/s390-virtio-ccw.c | 14 +++++++++++++- 9 files changed, 71 insertions(+), 6 deletions(-) diff --git a/include/hw/boards.h b/include/hw/boards.h index a71d1a53a5..d9ec37d807 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -317,6 +317,9 @@ struct MachineState { } \ type_init(machine_initfn##_register_types) =20 +extern GlobalProperty hw_compat_4_1[]; +extern const size_t hw_compat_4_1_len; + extern GlobalProperty hw_compat_4_0[]; extern const size_t hw_compat_4_0_len; =20 diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index 859b64c51d..c840e39251 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -302,6 +302,9 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t); int e820_get_num_entries(void); bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *); =20 +extern GlobalProperty pc_compat_4_1[]; +extern const size_t pc_compat_4_1_len; + extern GlobalProperty pc_compat_4_0[]; extern const size_t pc_compat_4_0_len; =20 diff --git a/hw/arm/virt.c b/hw/arm/virt.c index d9496c9363..64e3fc3440 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -2046,10 +2046,17 @@ static void machvirt_machine_init(void) } type_init(machvirt_machine_init); =20 +static void virt_machine_4_2_options(MachineClass *mc) +{ +} +DEFINE_VIRT_MACHINE_AS_LATEST(4, 2) + static void virt_machine_4_1_options(MachineClass *mc) { + virt_machine_4_2_options(mc); + compat_props_add(mc->compat_props, hw_compat_4_1, hw_compat_4_1_len); } -DEFINE_VIRT_MACHINE_AS_LATEST(4, 1) +DEFINE_VIRT_MACHINE(4, 1) =20 static void virt_machine_4_0_options(MachineClass *mc) { diff --git a/hw/core/machine.c b/hw/core/machine.c index 28a475ad97..fd76dad859 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -27,6 +27,9 @@ #include "hw/pci/pci.h" #include "hw/mem/nvdimm.h" =20 +GlobalProperty hw_compat_4_1[] =3D {}; +const size_t hw_compat_4_1_len =3D G_N_ELEMENTS(hw_compat_4_1); + GlobalProperty hw_compat_4_0[] =3D { { "VGA", "edid", "false" }, { "secondary-vga", "edid", "false" }, diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 549c437050..2aa1c49701 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -116,6 +116,9 @@ struct hpet_fw_config hpet_cfg =3D {.count =3D UINT8_MA= X}; /* Physical Address of PVH entry point read from kernel ELF NOTE */ static size_t pvh_start_addr; =20 +GlobalProperty pc_compat_4_1[] =3D {}; +const size_t pc_compat_4_1_len =3D G_N_ELEMENTS(pc_compat_4_1); + GlobalProperty pc_compat_4_0[] =3D {}; const size_t pc_compat_4_0_len =3D G_N_ELEMENTS(pc_compat_4_0); =20 diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index c2280c72ef..24e71be1d2 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -433,7 +433,7 @@ static void pc_i440fx_machine_options(MachineClass *m) machine_class_allow_dynamic_sysbus_dev(m, TYPE_RAMFB_DEVICE); } =20 -static void pc_i440fx_4_1_machine_options(MachineClass *m) +static void pc_i440fx_4_2_machine_options(MachineClass *m) { PCMachineClass *pcmc =3D PC_MACHINE_CLASS(m); pc_i440fx_machine_options(m); @@ -442,6 +442,18 @@ static void pc_i440fx_4_1_machine_options(MachineClass= *m) pcmc->default_cpu_version =3D 1; } =20 +DEFINE_I440FX_MACHINE(v4_2, "pc-i440fx-4.2", NULL, + pc_i440fx_4_2_machine_options); + +static void pc_i440fx_4_1_machine_options(MachineClass *m) +{ + pc_i440fx_4_2_machine_options(m); + m->alias =3D NULL; + m->is_default =3D 0; + compat_props_add(m->compat_props, hw_compat_4_1, hw_compat_4_1_len); + compat_props_add(m->compat_props, pc_compat_4_1, pc_compat_4_1_len); +} + DEFINE_I440FX_MACHINE(v4_1, "pc-i440fx-4.1", NULL, pc_i440fx_4_1_machine_options); =20 diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index 397e1fdd2f..e61260762c 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -365,7 +365,7 @@ static void pc_q35_machine_options(MachineClass *m) m->max_cpus =3D 288; } =20 -static void pc_q35_4_1_machine_options(MachineClass *m) +static void pc_q35_4_2_machine_options(MachineClass *m) { PCMachineClass *pcmc =3D PC_MACHINE_CLASS(m); pc_q35_machine_options(m); @@ -373,6 +373,17 @@ static void pc_q35_4_1_machine_options(MachineClass *m) pcmc->default_cpu_version =3D 1; } =20 +DEFINE_Q35_MACHINE(v4_2, "pc-q35-4.2", NULL, + pc_q35_4_2_machine_options); + +static void pc_q35_4_1_machine_options(MachineClass *m) +{ + pc_q35_4_2_machine_options(m); + m->alias =3D NULL; + compat_props_add(m->compat_props, hw_compat_4_1, hw_compat_4_1_len); + compat_props_add(m->compat_props, pc_compat_4_1, pc_compat_4_1_len); +} + DEFINE_Q35_MACHINE(v4_1, "pc-q35-4.1", NULL, pc_q35_4_1_machine_options); =20 diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 821f0d4a49..6c8aa60545 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -4426,15 +4426,26 @@ static const TypeInfo spapr_machine_info =3D { } \ type_init(spapr_machine_register_##suffix) =20 +/* + * pseries-4.2 + */ +static void spapr_machine_4_2_class_options(MachineClass *mc) +{ + /* Defaults for the latest behaviour inherited from the base class */ +} + +DEFINE_SPAPR_MACHINE(4_2, "4.2", true); + /* * pseries-4.1 */ static void spapr_machine_4_1_class_options(MachineClass *mc) { - /* Defaults for the latest behaviour inherited from the base class */ + spapr_machine_4_2_class_options(mc); + compat_props_add(mc->compat_props, hw_compat_4_1, hw_compat_4_1_len); } =20 -DEFINE_SPAPR_MACHINE(4_1, "4.1", true); +DEFINE_SPAPR_MACHINE(4_1, "4.1", false); =20 /* * pseries-4.0 diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 5b6a9a4e55..593b34e0e2 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -660,14 +660,26 @@ bool css_migration_enabled(void) } = \ type_init(ccw_machine_register_##suffix) =20 +static void ccw_machine_4_2_instance_options(MachineState *machine) +{ +} + +static void ccw_machine_4_2_class_options(MachineClass *mc) +{ +} +DEFINE_CCW_MACHINE(4_2, "4.2", true); + static void ccw_machine_4_1_instance_options(MachineState *machine) { + ccw_machine_4_2_instance_options(machine); } =20 static void ccw_machine_4_1_class_options(MachineClass *mc) { + ccw_machine_4_2_class_options(mc); + compat_props_add(mc->compat_props, hw_compat_4_1, hw_compat_4_1_len); } -DEFINE_CCW_MACHINE(4_1, "4.1", true); +DEFINE_CCW_MACHINE(4_1, "4.1", false); =20 static void ccw_machine_4_0_instance_options(MachineState *machine) { --=20 2.18.1 From nobody Mon Apr 29 02:19:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1564738845; cv=none; d=zoho.com; s=zohoarc; b=jPz+sxbBi5TLI8pf+EC0zN4sqGLNCee2OA6QKGM1SWOOAdgE4O2KlA9GM3vomO43q9D+HsA1lEJKGcDl2iKOwnhbyoVLKMRRcjWJBjCDEVW3cIDPvT+xIenh1GGeevMRgFKTSHRDmd5cqYkZoOWkq//5JVfoohLIS2EcyXetE44= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1564738845; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=F2RlD8HRo8ZUH5KRgAu8vIRq0+F9CMj3FCWOdwuOWYY=; b=mu6VOWZZX2wodP1bLMlDpHtT6lTwHf22ZgYw58To1CI2EVjRQVaV/X0I5uUtVyidyH7UTV9ObSVMP+F36uUpgVRyxN649PNBlkANA/wj4LTqcZetMOLOVFOIUgp6Bu8XX4HtspQ0RtbTNZurhNaPkUW9Z4veXSAnOIzk3kTv3vk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1564738845061873.2783214588301; Fri, 2 Aug 2019 02:40:45 -0700 (PDT) Received: from localhost ([::1]:33064 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU3E-00022X-V7 for importer@patchew.org; Fri, 02 Aug 2019 05:40:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55612) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU1k-0006zt-WC for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1htU1j-0000yB-Bx for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54906) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1htU1j-0000xd-47; Fri, 02 Aug 2019 05:39:07 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 709C63082E09; Fri, 2 Aug 2019 09:39:06 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id F397519C6A; Fri, 2 Aug 2019 09:39:04 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Date: Fri, 2 Aug 2019 05:38:52 -0400 Message-Id: <20190802093854.5343-3-imammedo@redhat.com> In-Reply-To: <20190802093854.5343-1-imammedo@redhat.com> References: <20190802093854.5343-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Fri, 02 Aug 2019 09:39:06 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH RFC v2 2/4] kvm: s390: split too big memory section on several memslots X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Max memslot size supported by kvm on s390 is 8Tb, move logic of splitting RAM on chunks upto 8T to KVM code. This way it will hide KVM specific restrictions in KVM code and won't affect baord level design decisions. Which would allow us to avoid misusing memory_region_allocate_system_memory() API and eventually use a single hostmem backend for guest RAM. Signed-off-by: Igor Mammedov --- patch prepares only KVM side for switching to single RAM memory region other patche will take care of migration issues that improper use of memory_region_allocate_system_memory() introduced --- include/hw/s390x/s390-virtio-ccw.h | 10 ++++ include/sysemu/kvm_int.h | 1 + accel/kvm/kvm-all.c | 77 ++++++++++++++++++------------ hw/s390x/s390-virtio-ccw.c | 9 ---- target/s390x/kvm.c | 1 + 5 files changed, 58 insertions(+), 40 deletions(-) diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-vir= tio-ccw.h index 8aa27199c9..00632f94b4 100644 --- a/include/hw/s390x/s390-virtio-ccw.h +++ b/include/hw/s390x/s390-virtio-ccw.h @@ -21,6 +21,16 @@ #define S390_MACHINE_CLASS(klass) \ OBJECT_CLASS_CHECK(S390CcwMachineClass, (klass), TYPE_S390_CCW_MACHINE) =20 +/* + * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages + * as the dirty bitmap must be managed by bitops that take an int as + * position indicator. If we have a guest beyond that we will split off + * new subregions. The split must happen on a segment boundary (1MB). + */ +#define KVM_MEM_MAX_NR_PAGES ((1ULL << 31) - 1) +#define SEG_MSK (~0xfffffULL) +#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SE= G_MSK) + typedef struct S390CcwMachineState { /*< private >*/ MachineState parent_obj; diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index 31df465fdc..7f7520bce2 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -41,4 +41,5 @@ typedef struct KVMMemoryListener { void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, AddressSpace *as, int as_id); =20 +void kvm_set_max_memslot_size(hwaddr max_slot_size); #endif diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index f450f25295..9a5ee0dc4f 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -138,6 +138,7 @@ bool kvm_direct_msi_allowed; bool kvm_ioeventfd_any_length_allowed; bool kvm_msi_use_devid; static bool kvm_immediate_exit; +static hwaddr kvm_max_slot_size =3D ~0; =20 static const KVMCapabilityInfo kvm_required_capabilites[] =3D { KVM_CAP_INFO(USER_MEMORY), @@ -951,6 +952,12 @@ kvm_check_extension_list(KVMState *s, const KVMCapabil= ityInfo *list) return NULL; } =20 +void kvm_set_max_memslot_size(hwaddr max_slot_size) +{ + g_assert(ROUND_UP(max_slot_size, qemu_real_host_page_size) =3D=3D max_= slot_size); + kvm_max_slot_size =3D max_slot_size; +} + static void kvm_set_phys_mem(KVMMemoryListener *kml, MemoryRegionSection *section, bool add) { @@ -958,7 +965,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, int err; MemoryRegion *mr =3D section->mr; bool writeable =3D !mr->readonly && !mr->rom_device; - hwaddr start_addr, size; + hwaddr start_addr, size, slot_size; void *ram; =20 if (!memory_region_is_ram(mr)) { @@ -983,41 +990,49 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, kvm_slots_lock(kml); =20 if (!add) { - mem =3D kvm_lookup_matching_slot(kml, start_addr, size); - if (!mem) { - goto out; - } - if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { - kvm_physical_sync_dirty_bitmap(kml, section); - } + do { + slot_size =3D kvm_max_slot_size < size ? kvm_max_slot_size : s= ize; + mem =3D kvm_lookup_matching_slot(kml, start_addr, slot_size); + if (!mem) { + goto out; + } + if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { + kvm_physical_sync_dirty_bitmap(kml, section); + } =20 - /* unregister the slot */ - g_free(mem->dirty_bmap); - mem->dirty_bmap =3D NULL; - mem->memory_size =3D 0; - mem->flags =3D 0; - err =3D kvm_set_user_memory_region(kml, mem, false); - if (err) { - fprintf(stderr, "%s: error unregistering slot: %s\n", - __func__, strerror(-err)); - abort(); - } + /* unregister the slot */ + g_free(mem->dirty_bmap); + mem->dirty_bmap =3D NULL; + mem->memory_size =3D 0; + mem->flags =3D 0; + err =3D kvm_set_user_memory_region(kml, mem, false); + if (err) { + fprintf(stderr, "%s: error unregistering slot: %s\n", + __func__, strerror(-err)); + abort(); + } + start_addr +=3D slot_size; + } while ((size -=3D slot_size)); goto out; } =20 /* register the new slot */ - mem =3D kvm_alloc_slot(kml); - mem->memory_size =3D size; - mem->start_addr =3D start_addr; - mem->ram =3D ram; - mem->flags =3D kvm_mem_flags(mr); - - err =3D kvm_set_user_memory_region(kml, mem, true); - if (err) { - fprintf(stderr, "%s: error registering slot: %s\n", __func__, - strerror(-err)); - abort(); - } + do { + slot_size =3D kvm_max_slot_size < size ? kvm_max_slot_size : size; + mem =3D kvm_alloc_slot(kml); + mem->memory_size =3D slot_size; + mem->start_addr =3D start_addr; + mem->ram =3D ram; + mem->flags =3D kvm_mem_flags(mr); + + err =3D kvm_set_user_memory_region(kml, mem, true); + if (err) { + fprintf(stderr, "%s: error registering slot: %s\n", __func__, + strerror(-err)); + abort(); + } + start_addr +=3D slot_size; + } while ((size -=3D slot_size)); =20 out: kvm_slots_unlock(kml); diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 593b34e0e2..073672f9cb 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -151,15 +151,6 @@ static void virtio_ccw_register_hcalls(void) virtio_ccw_hcall_early_printk); } =20 -/* - * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages - * as the dirty bitmap must be managed by bitops that take an int as - * position indicator. If we have a guest beyond that we will split off - * new subregions. The split must happen on a segment boundary (1MB). - */ -#define KVM_MEM_MAX_NR_PAGES ((1ULL << 31) - 1) -#define SEG_MSK (~0xfffffULL) -#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SE= G_MSK) static void s390_memory_init(ram_addr_t mem_size) { MemoryRegion *sysmem =3D get_system_memory(); diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c index 6e814c230b..d0267da8e1 100644 --- a/target/s390x/kvm.c +++ b/target/s390x/kvm.c @@ -347,6 +347,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) */ /* kvm_vm_enable_cap(s, KVM_CAP_S390_AIS, 0); */ =20 + kvm_set_max_memslot_size(KVM_SLOT_MAX_BYTES); return 0; } =20 --=20 2.18.1 From nobody Mon Apr 29 02:19:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1564738879; cv=none; d=zoho.com; s=zohoarc; b=VF521KKSWlScLqHfFdq4X2vX0OuedHea1/pe0I6wWVkzAZxuOrDY+OUzcYZ3bKMe+y1o7HxgDZnPJSHww5FNOPJknr4OkrWOAaKv1lm2aUePl5cGbJhbnPQGePquVpV3xQFvstosW1XZhhdOxPIVOmGFn2R4+UphC+ja1Uu+mak= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1564738879; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=aP/sdJuSnCvSUHgb3GFacHcgyiHgTJ8gLSo989ENXM0=; b=fMPlfo7vum/2NYigA12BkoCHhCWqcln45z2jKmFUMAgbmXE1ueQjzDCwHXxcCdVy7OO/9GxBb7GDnwBq0Xejlbxlz4lO0l7qxQsS3zjdBpZvs51spjbPDdoNHhjqkdhVKJy3xO8oC8GofKWXChDBBt2BDat+nxah8kzGurWDPZo= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1564738879885206.61368219207668; Fri, 2 Aug 2019 02:41:19 -0700 (PDT) Received: from localhost ([::1]:33074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU3q-0002y3-VN for importer@patchew.org; Fri, 02 Aug 2019 05:41:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55631) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU1m-00072N-9L for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1htU1l-00010D-6W for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54896) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1htU1l-0000zn-1B; Fri, 02 Aug 2019 05:39:09 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 56CAF83F45; Fri, 2 Aug 2019 09:39:08 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id B56AB19C6A; Fri, 2 Aug 2019 09:39:06 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Date: Fri, 2 Aug 2019 05:38:53 -0400 Message-Id: <20190802093854.5343-4-imammedo@redhat.com> In-Reply-To: <20190802093854.5343-1-imammedo@redhat.com> References: <20190802093854.5343-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 02 Aug 2019 09:39:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH RFC v2 3/4] memory: make MemoryRegion alias migratable X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, dgilbert@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" use qemu_ram_alloc_from_ptr() to create aliased RAMBlock to the part of original memory region. Change is migration safe as we do not migrate every existing RAMBlock anymore, to make it migratable code has to explicitly call vmstate_register_ram() on MemoryRegion that owns RAMBlock. Signed-off-by: Igor Mammedov --- PS: tested ping-pong migration between new and old QEMU for x86 pc/q35 and s390 machines. CC: dgilbert@redhat.com exec.c | 9 +++++---- memory.c | 6 ++++++ 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/exec.c b/exec.c index 3e78de3b8f..f5e9699632 100644 --- a/exec.c +++ b/exec.c @@ -2313,7 +2313,7 @@ static void ram_block_add(RAMBlock *new_block, Error = **errp, bool shared) new_block->used_length, DIRTY_CLIENTS_ALL); =20 - if (new_block->host) { + if (new_block->host && !new_block->mr->alias) { qemu_ram_setup_dump(new_block->host, new_block->max_length); qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_HUG= EPAGE); /* MADV_DONTFORK is also needed by KVM in absence of synchronous M= MU */ @@ -2497,7 +2497,7 @@ void qemu_ram_free(RAMBlock *block) return; } =20 - if (block->host) { + if (block->host && !block->mr->alias) { ram_block_notify_remove(block->host, block->max_length); } =20 @@ -2671,7 +2671,8 @@ RAMBlock *qemu_ram_block_from_host(void *ptr, bool ro= und_offset, =20 rcu_read_lock(); block =3D atomic_rcu_read(&ram_list.mru_block); - if (block && block->host && host - block->host < block->max_length) { + if (block && !block->mr->alias && block->host && + host - block->host < block->max_length) { goto found; } =20 @@ -2680,7 +2681,7 @@ RAMBlock *qemu_ram_block_from_host(void *ptr, bool ro= und_offset, if (block->host =3D=3D NULL) { continue; } - if (host - block->host < block->max_length) { + if (!block->mr->alias && host - block->host < block->max_length) { goto found; } } diff --git a/memory.c b/memory.c index 5d8c9a9234..d7c3609ce3 100644 --- a/memory.c +++ b/memory.c @@ -1678,6 +1678,12 @@ void memory_region_init_alias(MemoryRegion *mr, memory_region_init(mr, owner, name, size); mr->alias =3D orig; mr->alias_offset =3D offset; + if (orig->ram_block && size) { + mr->destructor =3D memory_region_destructor_ram; + mr->ram_block =3D qemu_ram_alloc_from_ptr(size, + orig->ram_block->host + of= fset, + mr, &error_fatal); + } } =20 void memory_region_init_rom_nomigrate(MemoryRegion *mr, --=20 2.18.1 From nobody Mon Apr 29 02:19:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1564738796; cv=none; d=zoho.com; s=zohoarc; b=jDAmHBRpS3RYGU3ztn8v7Pp/b1DMK8p1ANwHe+0hcBQGJ+mAxsUXjgGQSWdlKT4104bo7ihkL9VgK1FljC/yoVedEb80fX2Bvso/KpZcqsc+4Mb1BgSwPo4W2WwwJ1BdwY3aRX9C66yZ/SsVBKkCSOIW85M9wU8/ELo/xMkyenQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1564738796; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=h+cprCDElNEqOQ3jm+XiOQHvAF11fjRTmEA2SbnKJp8=; b=Aj412pZF1KD699Aa6rW30oBa9mfZwEk6drcqs/xvPkPZmNyb/KiAjtogJLtHcgCec4wHHXERu7M7M8DNAiStmlnCU/zUwBiYKmbwZEvXjCMZYh9hXfD7hWM4svZjLkWNLXoBaZxKpmQxKmAy+cIDM7bAJ1a5xep7Pxm19JzJ3as= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1564738796416134.66950728290044; Fri, 2 Aug 2019 02:39:56 -0700 (PDT) Received: from localhost ([::1]:33048 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU2T-0008UW-Hu for importer@patchew.org; Fri, 02 Aug 2019 05:39:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55668) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htU1o-00077s-9u for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1htU1m-00011S-UH for qemu-devel@nongnu.org; Fri, 02 Aug 2019 05:39:12 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46116) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1htU1m-00010s-NT; Fri, 02 Aug 2019 05:39:10 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 13EBE3082B4B; Fri, 2 Aug 2019 09:39:10 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9A1BE19C6A; Fri, 2 Aug 2019 09:39:08 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Date: Fri, 2 Aug 2019 05:38:54 -0400 Message-Id: <20190802093854.5343-5-imammedo@redhat.com> In-Reply-To: <20190802093854.5343-1-imammedo@redhat.com> References: <20190802093854.5343-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Fri, 02 Aug 2019 09:39:10 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH RFC v2 4/4] s390: do not call memory_region_allocate_system_memory() multiple times X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" s390 was trying to solve limited KVM memslot size issue by abusing memory_region_allocate_system_memory(), which breaks API contract where the function might be called only once. s390 should have used memory aliases to fragment inital memory into smaller chunks to satisfy KVM's memslot limitation. But its a bit late now, since allocated pieces are transfered in migration stream separately, so it's not possible to just replace broken layout with correct one. To workaround issue, MemoryRegion alases are made migratable and this patch switches to use them to split big initial RAM chunk into smaller pieces (KVM_SLOT_MAX_BYTES max) and registers aliases for migration. That should keep migration compatible with previous QEMU versions. New machine types (since 4.2) will use single memory region, which will get transimitted in migration stream as a whole RAMBlock. Signed-off-by: Igor Mammedov --- I don't have access to a suitable system to test it, so I've simulated it with smaller chunks on x84 host. Ping-pong migration between old and new QEMU worked fine. KVM part should be fine as memslots using mapped MemoryRegions (in this case it would be aliases) as far as I know but is someone could test it on big enough host it would be nice. --- include/hw/s390x/s390-virtio-ccw.h | 4 +++ hw/s390x/s390-virtio-ccw.c | 56 +++++++++++++++++++++--------- 2 files changed, 43 insertions(+), 17 deletions(-) diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-vir= tio-ccw.h index 00632f94b4..f9ed3737f8 100644 --- a/include/hw/s390x/s390-virtio-ccw.h +++ b/include/hw/s390x/s390-virtio-ccw.h @@ -21,6 +21,9 @@ #define S390_MACHINE_CLASS(klass) \ OBJECT_CLASS_CHECK(S390CcwMachineClass, (klass), TYPE_S390_CCW_MACHINE) =20 +#define S390_MACHINE_GET_CLASS(obj) \ + OBJECT_GET_CLASS(S390CcwMachineClass, (obj), TYPE_S390_CCW_MACHINE) + /* * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages * as the dirty bitmap must be managed by bitops that take an int as @@ -50,6 +53,7 @@ typedef struct S390CcwMachineClass { bool cpu_model_allowed; bool css_migration_enabled; bool hpage_1m_allowed; + bool split_ram_layout; } S390CcwMachineClass; =20 /* runtime-instrumentation allowed by the machine */ diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 073672f9cb..9160c1ed0a 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -151,28 +151,47 @@ static void virtio_ccw_register_hcalls(void) virtio_ccw_hcall_early_printk); } =20 -static void s390_memory_init(ram_addr_t mem_size) +static void s390_memory_init(MachineState *ms) { + S390CcwMachineClass *s390mc =3D S390_MACHINE_GET_CLASS(ms); MemoryRegion *sysmem =3D get_system_memory(); - ram_addr_t chunk, offset =3D 0; + MemoryRegion *ram =3D g_new(MemoryRegion, 1); unsigned int number =3D 0; Error *local_err =3D NULL; - gchar *name; + ram_addr_t mem_size =3D ms->ram_size; + gchar *name =3D g_strdup_printf(s390mc->split_ram_layout ? + "s390.whole.ram" : "s390.ram"); =20 /* allocate RAM for core */ - name =3D g_strdup_printf("s390.ram"); - while (mem_size) { - MemoryRegion *ram =3D g_new(MemoryRegion, 1); - uint64_t size =3D mem_size; - - /* KVM does not allow memslots >=3D 8 TB */ - chunk =3D MIN(size, KVM_SLOT_MAX_BYTES); - memory_region_allocate_system_memory(ram, NULL, name, chunk); - memory_region_add_subregion(sysmem, offset, ram); - mem_size -=3D chunk; - offset +=3D chunk; - g_free(name); - name =3D g_strdup_printf("s390.ram.%u", ++number); + memory_region_allocate_system_memory(ram, NULL, name, mem_size); + + /* migration compatible RAM handling for 4.1 and older machines */ + if (s390mc->split_ram_layout) { + ram_addr_t chunk, offset =3D 0; + /* + * memory_region_allocate_system_memory() registers allocated RAM f= or + * migration, however for compat reasons the RAM should be passed o= ver + * as RAMBlocks of the size upto KVM_SLOT_MAX_BYTES. So unregister = just + * allocated RAM so it won't be migrated directly. Aliases will tak= e care + * of segmenting RAM into legacy chunks that migration compatible. + */ + vmstate_unregister_ram(ram, NULL); + name =3D g_strdup_printf("s390.ram"); + while (mem_size) { + MemoryRegion *alias =3D g_new(MemoryRegion, 1); + + /* KVM does not allow memslots >=3D 8 TB */ + chunk =3D MIN(mem_size, KVM_SLOT_MAX_BYTES); + memory_region_init_alias(alias, NULL, name, ram, offset, chunk); + vmstate_register_ram_global(alias); + memory_region_add_subregion(sysmem, offset, alias); + mem_size -=3D chunk; + offset +=3D chunk; + g_free(name); + name =3D g_strdup_printf("s390.ram.%u", ++number); + } + } else { + memory_region_add_subregion(sysmem, 0, ram); } g_free(name); =20 @@ -257,7 +276,7 @@ static void ccw_init(MachineState *machine) =20 s390_sclp_init(); /* init memory + setup max page size. Required for the CPU model */ - s390_memory_init(machine->ram_size); + s390_memory_init(machine); =20 /* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus(machine); @@ -667,8 +686,11 @@ static void ccw_machine_4_1_instance_options(MachineSt= ate *machine) =20 static void ccw_machine_4_1_class_options(MachineClass *mc) { + S390CcwMachineClass *s390mc =3D S390_MACHINE_CLASS(mc); + ccw_machine_4_2_class_options(mc); compat_props_add(mc->compat_props, hw_compat_4_1, hw_compat_4_1_len); + s390mc->split_ram_layout =3D true; } DEFINE_CCW_MACHINE(4_1, "4.1", false); =20 --=20 2.18.1