From nobody Thu May 2 21:47:03 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604598306; cv=none; d=zohomail.com; s=zohoarc; b=OAmHudA93sM9OBVpl6tEVrCDaPd9buu9+Wq6b3rDxzJNiWLjNXtVsugQhwmZn+bQBsg9ogf2JEYczkfZFBWTlgxf8AxJb/bPTd/xrbUJHXUSx3dVG+sSU74jmvePI1u23ztOG34auDhSzgdPH5wXP+D1btPSAfRAk917ZlMB8d4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604598306; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=45ISBooNchfp4mZ2wLSegIRWFFC9NxidA1tWtrenJMo=; b=ZZDRkuRzOKal3kkSaoiS0e95kNjcNetU4Qn8cJDbUqyl2iqBuS/hG9BnfiVQhtNbNes0DVIouAhs/Z9XTzvIavugfA1HovG7af7kss3Ea65O/NdJ40ezCgIdq6Bv31cnWYXPrX2BQlCwENEp94AKU+9WSvCAEUZHBKjv7+mwJRA= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1604598306134636.3553773139372; Thu, 5 Nov 2020 09:45:06 -0800 (PST) Received: from localhost ([::1]:39284 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kajJn-0001Qi-Sw for importer@patchew.org; Thu, 05 Nov 2020 12:45:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:32936) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kajJC-0000vj-06 for qemu-devel@nongnu.org; Thu, 05 Nov 2020 12:44:26 -0500 Received: from frasgout.his.huawei.com ([185.176.79.56]:2049) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kajJ9-0003ck-3A for qemu-devel@nongnu.org; Thu, 05 Nov 2020 12:44:25 -0500 Received: from fraeml703-chm.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4CRrTM3VDGz67Hmp; Fri, 6 Nov 2020 01:43:03 +0800 (CST) Received: from lhreml710-chm.china.huawei.com (10.201.108.61) by fraeml703-chm.china.huawei.com (10.206.15.52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Thu, 5 Nov 2020 18:44:18 +0100 Received: from lhrphicprd00229.huawei.com (10.123.41.22) by lhreml710-chm.china.huawei.com (10.201.108.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1913.5; Thu, 5 Nov 2020 17:44:17 +0000 From: Jonathan Cameron To: Subject: [PATCH v2] hw/arm/virt enable support for virtio-mem Date: Fri, 6 Nov 2020 01:43:11 +0800 Message-ID: <20201105174311.566751-1-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.123.41.22] X-ClientProxiedBy: lhreml705-chm.china.huawei.com (10.201.108.54) To lhreml710-chm.china.huawei.com (10.201.108.61) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=jonathan.cameron@huawei.com; helo=frasgout.his.huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/11/05 12:13:13 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , David Hildenbrand , linuxarm@huawei.com, Jonathan Cameron , "Michael S . Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Basically a cut and paste job from the x86 support with the exception of needing a larger block size as the Memory Block Size (MIN_SECTION_SIZE) on ARM64 in Linux is 1G. Tested: * In full emulation and with KVM on an arm64 server. * cold and hotplug for the virtio-mem-pci device. * Wide range of memory sizes, added at creation and later. * Fairly basic memory usage of memory added. Seems to function as normal. * NUMA setup with virtio-mem-pci devices on each node. * Simple migration test. Related kernel patch just enables the Kconfig item for ARM64 as an alternative to x86 in drivers/virtio/Kconfig The original patches from David Hildenbrand stated that he thought it should work for ARM64 but it wasn't enabled in the kernel [1] It appears he was correct and everything 'just works'. The build system related stuff is intended to ensure virtio-mem support is not built for arm32 (build will fail due no defined block size). If there is a more elegant way to do this, please point me in the right direction. [1] https://lore.kernel.org/linux-mm/20191212171137.13872-1-david@redhat.co= m/ Signed-off-by: Jonathan Cameron --- default-configs/devices/aarch64-softmmu.mak | 1 + hw/arm/Kconfig | 1 + hw/arm/virt.c | 64 ++++++++++++++++++++- hw/virtio/Kconfig | 4 ++ hw/virtio/virtio-mem.c | 2 + 5 files changed, 71 insertions(+), 1 deletion(-) diff --git a/default-configs/devices/aarch64-softmmu.mak b/default-configs/= devices/aarch64-softmmu.mak index 958b1e08e4..31d6128a29 100644 --- a/default-configs/devices/aarch64-softmmu.mak +++ b/default-configs/devices/aarch64-softmmu.mak @@ -6,3 +6,4 @@ include arm-softmmu.mak CONFIG_XLNX_ZYNQMP_ARM=3Dy CONFIG_XLNX_VERSAL=3Dy CONFIG_SBSA_REF=3Dy +CONFIG_ARCH_VIRTIO_MEM_SUPPORTED=3Dy diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index fdf4464b94..eeae77eee9 100644 --- a/hw/arm/Kconfig +++ b/hw/arm/Kconfig @@ -20,6 +20,7 @@ config ARM_VIRT select PLATFORM_BUS select SMBIOS select VIRTIO_MMIO + select VIRTIO_MEM_SUPPORTED if ARCH_VIRTIO_MEM_SUPPORTED select ACPI_PCI select MEM_DEVICE select DIMM diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 8abb385d4e..6c96d71106 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -73,9 +73,11 @@ #include "hw/acpi/acpi.h" #include "target/arm/internals.h" #include "hw/mem/pc-dimm.h" +#include "hw/mem/memory-device.h" #include "hw/mem/nvdimm.h" #include "hw/acpi/generic_event_device.h" #include "hw/virtio/virtio-iommu.h" +#include "hw/virtio/virtio-mem-pci.h" #include "hw/char/pl011.h" #include "qemu/guest-random.h" =20 @@ -2286,6 +2288,34 @@ static void virt_memory_plug(HotplugHandler *hotplug= _dev, dev, &error_abort); } =20 +static void virt_virtio_md_pci_pre_plug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + HotplugHandler *hotplug_dev2 =3D qdev_get_bus_hotplug_handler(dev); + Error *local_err =3D NULL; + + if (!hotplug_dev2 && dev->hotplugged) { + /* + * Without a bus hotplug handler, we cannot control the plug/unplug + * order. We should never reach this point when hotplugging, + * however, better add a safety net. + */ + error_setg(errp, "hotplug of virtio-mem based memory devices not" + " supported on this bus."); + return; + } + /* + * First, see if we can plug this memory device at all. If that + * succeeds, branch of to the actual hotplug handler. + */ + memory_device_pre_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev), NULL, + &local_err); + if (!local_err && hotplug_dev2) { + hotplug_handler_pre_plug(hotplug_dev2, dev, &local_err); + } + error_propagate(errp, local_err); +} + static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -2293,6 +2323,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHa= ndler *hotplug_dev, =20 if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { virt_memory_pre_plug(hotplug_dev, dev, errp); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + virt_virtio_md_pci_pre_plug(hotplug_dev, dev, errp); } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) { hwaddr db_start =3D 0, db_end =3D 0; char *resv_prop_str; @@ -2322,6 +2354,27 @@ static void virt_machine_device_pre_plug_cb(HotplugH= andler *hotplug_dev, } } =20 +static void virt_virtio_md_pci_plug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + HotplugHandler *hotplug_dev2 =3D qdev_get_bus_hotplug_handler(dev); + Error *local_err =3D NULL; + + /* + * Plug the memory device first and then branch off to the actual + * hotplug handler. If that one fails, we can easily undo the memory + * device bits. + */ + memory_device_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev)); + if (hotplug_dev2) { + hotplug_handler_plug(hotplug_dev2, dev, &local_err); + if (local_err) { + memory_device_unplug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev)); + } + } + error_propagate(errp, local_err); +} + static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -2336,6 +2389,9 @@ static void virt_machine_device_plug_cb(HotplugHandle= r *hotplug_dev, if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { virt_memory_plug(hotplug_dev, dev, errp); } + if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + virt_virtio_md_pci_plug(hotplug_dev, dev, errp); + } if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) { PCIDevice *pdev =3D PCI_DEVICE(dev); =20 @@ -2363,6 +2419,11 @@ static void virt_dimm_unplug_request(HotplugHandler = *hotplug_dev, goto out; } =20 + if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + error_setg(&local_err, + "virtio-mem based memory devices cannot be unplugged."); + goto out; + } hotplug_handler_unplug_request(HOTPLUG_HANDLER(vms->acpi_dev), dev, &local_err); out: @@ -2413,7 +2474,8 @@ static HotplugHandler *virt_machine_get_hotplug_handl= er(MachineState *machine, DeviceState *dev) { if (object_dynamic_cast(OBJECT(dev), TYPE_SYS_BUS_DEVICE) || - (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM))) { + object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) || + object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { return HOTPLUG_HANDLER(machine); } if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) { diff --git a/hw/virtio/Kconfig b/hw/virtio/Kconfig index 0eda25c4e1..00dbf2939e 100644 --- a/hw/virtio/Kconfig +++ b/hw/virtio/Kconfig @@ -48,6 +48,10 @@ config VIRTIO_PMEM depends on VIRTIO_PMEM_SUPPORTED select MEM_DEVICE =20 +config ARCH_VIRTIO_MEM_SUPPORTED + bool + default n + config VIRTIO_MEM_SUPPORTED bool =20 diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index 7c8ca9f28b..16f9de6ab6 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -53,6 +53,8 @@ */ #if defined(TARGET_X86_64) || defined(TARGET_I386) #define VIRTIO_MEM_USABLE_EXTENT (2 * (128 * MiB)) +#elif defined(TARGET_AARCH64) +#define VIRTIO_MEM_USABLE_EXTENT (2 * (1024 * MiB)) #else #error VIRTIO_MEM_USABLE_EXTENT not defined #endif --=20 2.19.1