From nobody Mon Feb 9 05:40:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1591973559; cv=none; d=zohomail.com; s=zohoarc; b=llR2mf7/4iwkoe4JTll+kYdF/utxhJRdE8Hp6t/iDj29jBimy4tN7hire6haUarHcglBZCQlRB2q7B2NNbFyI7+0WHf9ZT2ozWX58haTq532AsiUMeUhcxc2moojK+66sy5VNNME7RLTOpjGWB+qQTcflNqqhHFY7bd6n0N4rHA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591973559; h=Content-Type:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ezVaZ39w/Zt6S04FtdW+skZfwDczX4yGiWwy5FV/qKo=; b=SlGqh4SonIWjwF2Ra3ShHMwEIgJ7LphqGJymC6959Zm7do9MjziVlFoN+rX4altZfc/RvNPdlkxzcDCgOs0ydmPfOa7t4+b3sV1kBbo1i04HliIPp5kyafTwd3v8CcnZ83RHy1tkEXJArfGb0D3X+7v214OehFyHzM1H9aafX+8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591973559633728.7541727554321; Fri, 12 Jun 2020 07:52:39 -0700 (PDT) Received: from localhost ([::1]:37554 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jjl2s-00071k-5j for importer@patchew.org; Fri, 12 Jun 2020 10:52:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjl1X-0005Cg-UH for qemu-devel@nongnu.org; Fri, 12 Jun 2020 10:51:15 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:47308 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jjl1V-0001MI-MW for qemu-devel@nongnu.org; Fri, 12 Jun 2020 10:51:15 -0400 Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-30-b9gyMVJ9MiGJOKNPGi7sKA-1; Fri, 12 Jun 2020 10:51:11 -0400 Received: by mail-wm1-f70.google.com with SMTP id t145so2545028wmt.2 for ; Fri, 12 Jun 2020 07:51:10 -0700 (PDT) Received: from redhat.com (bzq-79-178-18-124.red.bezeqint.net. [79.178.18.124]) by smtp.gmail.com with ESMTPSA id y5sm10767253wrs.63.2020.06.12.07.51.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Jun 2020 07:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591973473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ezVaZ39w/Zt6S04FtdW+skZfwDczX4yGiWwy5FV/qKo=; b=ZzbiMyTfgVLqT7qu9Hy3w1MRBeWhRfiRht7Qu24WScpS7ia82m4KG5ZF4dRyMhGz8aXjRp b8YoSlOUVjoTlPcxkCpFZ7fMklxYqMMqeiMk/YL7z6xaVzKM34jdSR9C/r6wRb9SlL2Dn4 07lOxX1xRTXHqPZvFBt63YseyhknShU= X-MC-Unique: b9gyMVJ9MiGJOKNPGi7sKA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ezVaZ39w/Zt6S04FtdW+skZfwDczX4yGiWwy5FV/qKo=; b=cadq4J6pZc2IOE9Sfymdnuzq7HYzEtlhe0YzjoCF9nN9CbWxdoArP+vTFy4nzRHGJO 5S7Pgac2chJQDt/iOXoN3uocYQz++ziu5EK6E2St33fu9rRW78STTF9j2GXcYM1LXvrY PgxUoYCLe5UckRqOTNSydd1iqeB75+MYnamerpPZh5ss+pzSykYUb3ZGDRyHO9hF2rJJ m7JmqCbUYvkMRk9wp3OKCUAes/KhWuRmr9Qzb1mJx6eEoPEiqp2OCYIY1hafOlmy84MF B4ND+niHn8OKEbayTAvMTuM93gbzHsSQCJ0D5lXxHq7WVS7NXzhEUEzumYCUtN1H3cNV PF8w== X-Gm-Message-State: AOAM530Rj1fcy3TQEDfXUmA9J1nI5DOnRtTPDbb5OT7Mdz/iFHm/8QyX yxiYQSQO535Y+oBsz2hBjy65pTbzrVWyTQx2H4RbymB1WlRXIK5MFOtwRS2FMmuFrmy2jPXlFPI k8hTiMFWaky5rKdQ= X-Received: by 2002:a1c:2082:: with SMTP id g124mr14255826wmg.21.1591973469590; Fri, 12 Jun 2020 07:51:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzEG0pJXLkOLT0yypGv25+q3tAYfMF2FkJ+8vRADgp1GU0Xko87chojkSQxIx77EuSNTk9gPA== X-Received: by 2002:a1c:2082:: with SMTP id g124mr14255796wmg.21.1591973469321; Fri, 12 Jun 2020 07:51:09 -0700 (PDT) Date: Fri, 12 Jun 2020 10:51:07 -0400 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Subject: [PULL v2 03/58] hw/acpi/nvdimm: add a helper to augment SRAT generation Message-ID: <20200612141917.9446-4-mst@redhat.com> References: <20200612141917.9446-1-mst@redhat.com> MIME-Version: 1.0 In-Reply-To: <20200612141917.9446-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=mst@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/12 08:00:42 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Xiao Guangrong , Jingqi Liu , Eduardo Habkost , Shannon Zhao , Paolo Bonzini , Vishal Verma , Igor Mammedov , qemu-arm@nongnu.org, Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vishal Verma NVDIMMs can belong to their own proximity domains, as described by the NFIT. In such cases, the SRAT needs to have Memory Affinity structures in the SRAT for these NVDIMMs, otherwise Linux doesn't populate node data structures properly during NUMA initialization. See the following for an example failure case. https://lore.kernel.org/linux-nvdimm/20200416225438.15208-1-vishal.l.verma@= intel.com/ Introduce a new helper, nvdimm_build_srat(), and call it for both the i386 and arm versions of 'build_srat()' to augment the SRAT with memory affinity information for NVDIMMs. The relevant command line options to exercise this are below. Nodes 0-1 contain CPUs and regular memory, and nodes 2-3 are the NVDIMM address space. -object memory-backend-ram,id=3Dmem0,size=3D2048M -numa node,nodeid=3D0,memdev=3Dmem0, -numa cpu,node-id=3D0,socket-id=3D0 -object memory-backend-ram,id=3Dmem1,size=3D2048M -numa node,nodeid=3D1,memdev=3Dmem1, -numa cpu,node-id=3D1,socket-id=3D1 -numa node,nodeid=3D2, -object memory-backend-file,id=3Dnvmem0,share,mem-path=3Dnvdimm-0,size= =3D16384M,align=3D1G -device nvdimm,memdev=3Dnvmem0,id=3Dnv0,label-size=3D2M,node=3D2 -numa node,nodeid=3D3, -object memory-backend-file,id=3Dnvmem1,share,mem-path=3Dnvdimm-1,size= =3D16384M,align=3D1G -device nvdimm,memdev=3Dnvmem1,id=3Dnv1,label-size=3D2M,node=3D3 Cc: Jingqi Liu Cc: Michael S. Tsirkin Reviewed-by: Jingqi Liu Reviewed-by: Igor Mammedov Signed-off-by: Vishal Verma Message-Id: <20200606000911.9896-3-vishal.l.verma@intel.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/mem/nvdimm.h | 1 + hw/acpi/nvdimm.c | 23 +++++++++++++++++++++++ hw/arm/virt-acpi-build.c | 4 ++++ hw/i386/acpi-build.c | 5 +++++ 4 files changed, 33 insertions(+) diff --git a/include/hw/mem/nvdimm.h b/include/hw/mem/nvdimm.h index a3c08955e8..b67a1aedf6 100644 --- a/include/hw/mem/nvdimm.h +++ b/include/hw/mem/nvdimm.h @@ -155,6 +155,7 @@ typedef struct NVDIMMState NVDIMMState; void nvdimm_init_acpi_state(NVDIMMState *state, MemoryRegion *io, struct AcpiGenericAddress dsm_io, FWCfgState *fw_cfg, Object *owner); +void nvdimm_build_srat(GArray *table_data); void nvdimm_build_acpi(GArray *table_offsets, GArray *table_data, BIOSLinker *linker, NVDIMMState *state, uint32_t ram_slots); diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c index 9316d12b70..8f7cc16add 100644 --- a/hw/acpi/nvdimm.c +++ b/hw/acpi/nvdimm.c @@ -28,6 +28,7 @@ =20 #include "qemu/osdep.h" #include "qemu/uuid.h" +#include "qapi/error.h" #include "hw/acpi/acpi.h" #include "hw/acpi/aml-build.h" #include "hw/acpi/bios-linker-loader.h" @@ -1334,6 +1335,28 @@ static void nvdimm_build_ssdt(GArray *table_offsets,= GArray *table_data, free_aml_allocator(); } =20 +void nvdimm_build_srat(GArray *table_data) +{ + GSList *device_list =3D nvdimm_get_device_list(); + + for (; device_list; device_list =3D device_list->next) { + AcpiSratMemoryAffinity *numamem =3D NULL; + DeviceState *dev =3D device_list->data; + Object *obj =3D OBJECT(dev); + uint64_t addr, size; + int node; + + node =3D object_property_get_int(obj, PC_DIMM_NODE_PROP, &error_ab= ort); + addr =3D object_property_get_uint(obj, PC_DIMM_ADDR_PROP, &error_a= bort); + size =3D object_property_get_uint(obj, PC_DIMM_SIZE_PROP, &error_a= bort); + + numamem =3D acpi_data_push(table_data, sizeof *numamem); + build_srat_memory(numamem, addr, size, node, + MEM_AFFINITY_ENABLED | MEM_AFFINITY_NON_VOLATILE= ); + } + g_slist_free(device_list); +} + void nvdimm_build_acpi(GArray *table_offsets, GArray *table_data, BIOSLinker *linker, NVDIMMState *state, uint32_t ram_slots) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 1b0a584c7b..2cbccd5fe2 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -539,6 +539,10 @@ build_srat(GArray *table_data, BIOSLinker *linker, Vir= tMachineState *vms) } } =20 + if (ms->nvdimms_state->is_enabled) { + nvdimm_build_srat(table_data); + } + if (ms->device_memory) { numamem =3D acpi_data_push(table_data, sizeof *numamem); build_srat_memory(numamem, ms->device_memory->base, diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index 2e15f6848e..d996525e2c 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -2428,6 +2428,11 @@ build_srat(GArray *table_data, BIOSLinker *linker, M= achineState *machine) MEM_AFFINITY_ENABLED); } } + + if (machine->nvdimms_state->is_enabled) { + nvdimm_build_srat(table_data); + } + slots =3D (table_data->len - numa_start) / sizeof *numamem; for (; slots < pcms->numa_nodes + 2; slots++) { numamem =3D acpi_data_push(table_data, sizeof *numamem); --=20 MST