From nobody Wed Mar 5 02:31:25 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1740996557; cv=none; d=zohomail.com; s=zohoarc; b=b6uQmXfstszcbuSUxdGjT7pzJLnL7EODcchbYTbKPdaFGfNd6a1hlM3DHa/eMLFNUC3TQHtj0jHMAunMQQ0Hu/JJsBkq+nbxJE8cSLI3pTAFKrHvoHfQuaJehAsQKtcx62GD1WNgdYBAPKUa9ggBcmPhGirokG6iUwXZkGE+zQE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1740996557; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=W8t3J5ob90+FnP0JXzgWqmqxURfMzfIJFtjvOoa/kNo=; b=e+da/xjjvh6jw8gLhhwQtY42szAiGZOt5ntR73zgrqrxV+gNvROSmtibNWzdaA3vOTg0Kl6ndvLr3aYPXaE/sbf7ydfrTGbIdeD5gcO4iynmUoNqE8MQfR/pCIqLb3c3ikXuMnE4PVSKdQRFK0KO0+W4IL8byppG3mnrmvS3QOI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1740996557684665.5083675409674; Mon, 3 Mar 2025 02:09:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tp2ir-0001HS-Lj; Mon, 03 Mar 2025 05:08:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tp2iL-0001Cv-53; Mon, 03 Mar 2025 05:08:05 -0500 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tp2iI-0003D2-Ua; Mon, 03 Mar 2025 05:07:56 -0500 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-2fea8d8c322so7641141a91.2; Mon, 03 Mar 2025 02:07:48 -0800 (PST) Received: from wheely.local0.net ([118.208.151.101]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fea67a7ddesm8539779a91.27.2025.03.03.02.07.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Mar 2025 02:07:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740996466; x=1741601266; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W8t3J5ob90+FnP0JXzgWqmqxURfMzfIJFtjvOoa/kNo=; b=FFGjmf2wi7qAcH9Q4bqApZrvLL7F5cwZwgZQL1VRLWh68+KoINHP8lKSPO2Vg92v2p QDde7c/WO119tegCdiB4gDa2bpx5NSmuGWMRywh/vR878XAIHfoqLunA0JR/Mz1gxVsr jWx2VbfhzYZ0aoSyes6JYOpm1OT7sa2fmBDWyZ2PhTQ3ef7yx2IuAu+WE8k5h5BQfyk/ +NSyCkZkXJZAqgr4jhBMqQ05jOCwhbtwhEaeqVTHejbYDoaIhvbsvJ3RqARZwN73Aovf 8Ygfmp982OEEOx19CF3UrGscFA5/Xl4h7ywbAgygluPArZ97JT+nuiGFERVMxMFuwl/g PGNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740996466; x=1741601266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W8t3J5ob90+FnP0JXzgWqmqxURfMzfIJFtjvOoa/kNo=; b=c1Z0bjbUwAfYDAjj/TehhL0D0xytPK4DIApECvEclDXzQpfsOy4gXVQJUPGA1gZXJb dcI89gMxN90FIhE+bvXsYWlMCkZHvkfX7nRFweN7j5nngZSbRILJjyFyBGS+h1SP+xXj apP7RHU0WtNpDuGT7uiS0OzE9IFwhVrPHvlVK25WadSqY9+/HS9SKGLifrzGZJJ5oyFe fhShgFc7B/smJ9qoW2I6GxTRmcq00T+HvnF1pnP1wzBcEsNjEwAn9jdGYTtjcHrTXMLH U5wYHHDH6sFKDelvWe/oGN4PVxDS4ytjGcTUBDzvRMnSQ0inYwO6XpqI2y+DL5a+Nz5F jwZQ== X-Forwarded-Encrypted: i=1; AJvYcCXyGIoMfha2A3ufcto7z0IhggvHcGedFnn0RtI6P1hqjM+dD3Lv5X6wG8GwTR/NnLT7Lfh+7TyY3a2D@nongnu.org X-Gm-Message-State: AOJu0YzsttjpjNLuol+1ucNnhS49hbl7ZILhslr6biWDMc5nefRCE2dN 6GLDksTprSKpC7oMxzdgGufxcmR7bcb132dSHFg2y9x1pB7ghaGOScSb6g== X-Gm-Gg: ASbGncu+Kt09cabOf7rJ8KLswZrPeZa6n24G3zrNA7u6KPF6QvgqaX6NIvFGxVxEo3U JcTrpnv2iQ5/6/2ngTJmWuj55fGQ3Vi18JyJd0b6le7aKI3Xjoq6mHLhL+3OZzzaE2KXB0i/55n jR0i57WIpekHzG5YLoATAYoKJCqcHLfkTW7ffU9sr9tY50Z+IlVY/TM3A8Ia8wJqHVIlwCGGyDa yN9HBBBMePFWKYVrqhPLGnJnDvO5BX1+2pa/VMpkCazzFBqkf95uLT947xyEooaOi5c9rpTMvaj NqmjAgIV7QfodR+IT8pIW27IGUVWe70H1kxwP9/U6xMMb4UClP0= X-Google-Smtp-Source: AGHT+IHzSfE4fE1zFdFCLLYQZdCDCbKbGhtmAT8NHZXG+LRg2mssOD6Uh2QAAL8Isj+QvEPypC90eA== X-Received: by 2002:a17:90b:1cc6:b0:2fc:3264:3657 with SMTP id 98e67ed59e1d1-2febaa92594mr24335979a91.0.1740996466478; Mon, 03 Mar 2025 02:07:46 -0800 (PST) From: Nicholas Piggin To: qemu-ppc@nongnu.org Cc: Nicholas Piggin , qemu-devel@nongnu.org, Eduardo Habkost , Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Zhao Liu , =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20Barrat?= , Igor Mammedov Subject: [PATCH 1/3] ppc/pnv: Add support for NUMA configuration Date: Mon, 3 Mar 2025 20:07:30 +1000 Message-ID: <20250303100732.576457-2-npiggin@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250303100732.576457-1-npiggin@gmail.com> References: <20250303100732.576457-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=npiggin@gmail.com; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1740996560520019100 Content-Type: text/plain; charset="utf-8" Enable NUMA topology configuration for the powernv machine by filling the necessary attributes and methods. pnv_possible_cpu_arch_ids() runs before pnv_init(), so the hacky big-core topology adjustment has to be moved there. Signed-off-by: Nicholas Piggin --- hw/ppc/pnv.c | 101 +++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 89 insertions(+), 12 deletions(-) diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c index 11fd477b71b..5f2041f7f9d 100644 --- a/hw/ppc/pnv.c +++ b/hw/ppc/pnv.c @@ -1082,18 +1082,6 @@ static void pnv_init(MachineState *machine) exit(1); } =20 - if (pnv->big_core) { - /* - * powernv models PnvCore as a SMT4 core. Big-core requires 2xPnvC= ore - * per core, so adjust topology here. pnv_dt_core() processor - * device-tree and TCG SMT code make the 2 cores appear as one big= core - * from software point of view. pnv pervasive models and xscoms te= nd to - * see the big core as 2 small core halves. - */ - machine->smp.cores *=3D 2; - machine->smp.threads /=3D 2; - } - if (!is_power_of_2(machine->smp.threads)) { error_report("Cannot support %d threads/core on a powernv " "machine because it must be a power of 2", @@ -2865,6 +2853,87 @@ static void pnv_nmi(NMIState *n, int cpu_index, Erro= r **errp) } } =20 +/* find cpu slot in machine->possible_cpus by core_id */ +static CPUArchId *pnv_find_cpu_slot(MachineState *ms, uint32_t id, int *id= x) +{ + int index =3D id / ms->smp.threads; + + if (index >=3D ms->possible_cpus->len) { + return NULL; + } + if (idx) { + *idx =3D index; + } + return &ms->possible_cpus->cpus[index]; +} + +static CpuInstanceProperties +pnv_cpu_index_to_props(MachineState *machine, unsigned cpu_index) +{ + CPUArchId *core_slot; + MachineClass *mc =3D MACHINE_GET_CLASS(machine); + + /* make sure possible_cpu are intialized */ + mc->possible_cpu_arch_ids(machine); + /* get CPU core slot containing thread that matches cpu_index */ + core_slot =3D pnv_find_cpu_slot(machine, cpu_index, NULL); + assert(core_slot); + return core_slot->props; +} + +static const CPUArchIdList *pnv_possible_cpu_arch_ids(MachineState *machin= e) +{ + PnvMachineState *pnv =3D PNV_MACHINE(machine); + MachineClass *mc =3D MACHINE_GET_CLASS(machine); + unsigned int smp_cpus =3D machine->smp.cpus; + unsigned int smp_threads; + int max_cores; + int i; + + if (pnv->big_core && !machine->possible_cpus) { + /* + * powernv models PnvCore as a SMT4 core. Big-core requires 2xPnvC= ore + * per core, so adjust topology here the first time it is called. + * pnv_dt_core() processor device-tree and TCG SMT code make the 2 + * cores appear as one big core from software point of view. pnv + * pervasive models and xscoms tend to see the big core as 2 small= core + * halves. + */ + machine->smp.cores *=3D 2; + machine->smp.threads /=3D 2; + } + + smp_threads =3D machine->smp.threads; + max_cores =3D machine->smp.max_cpus / smp_threads; + + if (!mc->has_hotpluggable_cpus) { + max_cores =3D QEMU_ALIGN_UP(smp_cpus, smp_threads) / smp_threads; + } + if (machine->possible_cpus) { + assert(machine->possible_cpus->len =3D=3D max_cores); + return machine->possible_cpus; + } + + machine->possible_cpus =3D g_malloc0(sizeof(CPUArchIdList) + + sizeof(CPUArchId) * max_cores); + machine->possible_cpus->len =3D max_cores; + for (i =3D 0; i < machine->possible_cpus->len; i++) { + int core_id =3D i * smp_threads; + + machine->possible_cpus->cpus[i].type =3D machine->cpu_type; + machine->possible_cpus->cpus[i].vcpus_count =3D smp_threads; + machine->possible_cpus->cpus[i].arch_id =3D core_id; + machine->possible_cpus->cpus[i].props.has_core_id =3D true; + machine->possible_cpus->cpus[i].props.core_id =3D core_id; + } + return machine->possible_cpus; +} + +static int64_t pnv_get_default_cpu_node_id(const MachineState *ms, int idx) +{ + return idx / ms->smp.cores % ms->numa_state->num_nodes; +} + static void pnv_machine_class_init(ObjectClass *oc, void *data) { MachineClass *mc =3D MACHINE_CLASS(oc); @@ -2879,6 +2948,14 @@ static void pnv_machine_class_init(ObjectClass *oc, = void *data) mc->block_default_type =3D IF_IDE; mc->no_parallel =3D 1; mc->default_boot_order =3D NULL; + + mc->numa_mem_supported =3D true; + mc->auto_enable_numa =3D true; + + mc->cpu_index_to_instance_props =3D pnv_cpu_index_to_props; + mc->get_default_cpu_node_id =3D pnv_get_default_cpu_node_id; + mc->possible_cpu_arch_ids =3D pnv_possible_cpu_arch_ids; + /* * RAM defaults to less than 2048 for 32-bit hosts, and large * enough to fit the maximum initrd size at it's load address --=20 2.47.1 From nobody Wed Mar 5 02:31:25 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1740996557; cv=none; d=zohomail.com; s=zohoarc; b=OkNSOxXuSFmovVw/FLzsl5kVFhISfk8xlW8SpArDHhrJ2MFC9Es+biWsnI60lM9GaehTmOvTHYbrizunUOXfbcZoANqj8nriEUsAU09zzhGRj0K8aVmX+ffqTo/NRjad1X/aAt7W6wh7P3lPQr/CMM2UdArZYedjkNjNfKozUYY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1740996557; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xd66omHNK7ev0/glEwTCxxD6meGDUkOt6mpT1wI9J5Q=; b=jzaJZ3XGr33XKYDs7sEsE48PeWbm6qP/ssyDF5v4BcPcWgpRk3tEi0GZLTGtEeJoHxapLLk/Ahe+tsVD2oNLxkV+7F5vGLsl6UjO2SsVtbIPtQ4jL19fpaDKCOWRr6VzB0pfYk5iV42rRPLRVCVsD8inQL+IPqeHOKhKdbxgrDI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1740996557431764.1871159935375; Mon, 3 Mar 2025 02:09:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tp2ir-0001H0-IE; Mon, 03 Mar 2025 05:08:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tp2iN-0001D5-W2; Mon, 03 Mar 2025 05:08:09 -0500 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tp2iJ-0003Ds-Vx; Mon, 03 Mar 2025 05:07:58 -0500 Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-2feb91a25bdso5016967a91.1; Mon, 03 Mar 2025 02:07:52 -0800 (PST) Received: from wheely.local0.net ([118.208.151.101]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fea67a7ddesm8539779a91.27.2025.03.03.02.07.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Mar 2025 02:07:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740996471; x=1741601271; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xd66omHNK7ev0/glEwTCxxD6meGDUkOt6mpT1wI9J5Q=; b=XcdzipGFcCJrMIxypNeGx44TniYfm4mO3nLi2cxgVSfkJo1nEQwi8gbqnQcQp5wvlT svXVwqTu2qkJ0Yk8JEJ4Q/jpCFuwwkSii9wGYf63RXuXAoJ6vpKM1nPJsXGf7DuFN0lX degQzegFsbbJMMvX2Fmctnc1dLyTLCz2+nQSM3ubev9Yqq/gLJ3qmAsami5rmP3C+lKN fASjW8hl9h6dp6by82cMn+KarIhKxN824iwAmuTh7mmGoXY3w5QMx4SVOMV0+IgVLrPf 578sz9+GOE+hl+YUnBsL8dwvd5o94YrwGS6TEBUczgHKbk9VeZ87jVGojQI0j5Vpl4r3 v3oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740996471; x=1741601271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xd66omHNK7ev0/glEwTCxxD6meGDUkOt6mpT1wI9J5Q=; b=M5hCnUtJ2vCgozAJQHpApPFRdczuiNoJbdXpH+hUmhIBqvm+0UoVvP+0YwaObcdVH4 SAVf7RThJ0Y7PyatbJf7e1tzxxVIBVA4J+j7IRM0eiF2QhBPyBTMZQufW+I2HzxcPtvu dQYLq6hKghl2pMBWQUqc+yMjbXTMERwlobEWlgXmKeeVz4HDGNV4tW5IaIvvlXm0PvO9 fEeCPA5U+Ho52nJdBbohdqnYRmIGX3gPpXfwbuRCNT5HYZY9OG0ZjMKY7pgaxLk51Mb/ qVhUbrSZyz/Bg2EE8KeZgQj3vXsTrZIicWCLudZJEL027psHLioxy4JbNAKEC3HovgqE GM9Q== X-Forwarded-Encrypted: i=1; AJvYcCXdCwfalsSosji1J4eFxZ2JGxuodQaOciLh9DtIAJJngeuIPcDmhJvCBKQdXWHtY3zA1jgWxzfFsX4K@nongnu.org X-Gm-Message-State: AOJu0YyeSHZnkTmTo//FBaaJx25Wit6b9Gd87FpgPgivnTSdxKhhuy27 4oyeJCy+Sc49QDYBBySmrWQjc2C0RDNUv/lHAJYOTOiEi8dfVJyWJEyJXA== X-Gm-Gg: ASbGncs6avAHaEcYL6eVET3qQXKRV+QYAZBtjVXZelSS67woCLEszDXYm2o3IPK6CYa OVsW/bFZ4ZVrptnsHjRdB/Rpnqu3YABCu3W+eZniyYj1mgFHQdOml06F8HwrgK6zEyTjCX7YT3f fjZQ9ADSke/+s7P1ygou1f/AMndt5AScAZv3LqaciMqysHzOCr9KOU65bFQ6jK9S0lJKmzmEaf5 aQVN/uhayCYv1A6IsoFCuAXMur/DZy9bulAMrHBMo/osaXfQx2flyEO58psE/6U5FUJB5f6eJcu FeF70MNdXdgov9TqolCR3eIU8p2onDlVTzCfJ7PXDTYguzo1Klo= X-Google-Smtp-Source: AGHT+IELzAgnAvqAYo95llqguI2UGD3I8UgV3N182MKM7uOOwlbkxCxHHV5hBjkEq9wv0ymg8qZ+GQ== X-Received: by 2002:a17:90b:3a4c:b0:2f6:dcc9:38e0 with SMTP id 98e67ed59e1d1-2febaa861e5mr23275909a91.0.1740996470911; Mon, 03 Mar 2025 02:07:50 -0800 (PST) From: Nicholas Piggin To: qemu-ppc@nongnu.org Cc: Nicholas Piggin , qemu-devel@nongnu.org, Eduardo Habkost , Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Zhao Liu , =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20Barrat?= , Igor Mammedov Subject: [PATCH 2/3] hw/core/numa: add attribute to skip creation of MachineState.ram region Date: Mon, 3 Mar 2025 20:07:31 +1000 Message-ID: <20250303100732.576457-3-npiggin@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250303100732.576457-1-npiggin@gmail.com> References: <20250303100732.576457-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=npiggin@gmail.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1740996558648019100 Content-Type: text/plain; charset="utf-8" NUMA machines with sparse address topologies do not want all NUMA regions packed densely inside the MachineState.ram container region. Add a machine class attribute that skips creating this container region. Individual NUMA memory device regions are recorded in NodeInfo where the machine init can add them to the system address space itself. Signed-off-by: Nicholas Piggin --- include/hw/boards.h | 6 ++++++ include/system/numa.h | 1 + hw/core/numa.c | 44 +++++++++++++++++++++++++++++++++++-------- 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/include/hw/boards.h b/include/hw/boards.h index 9360d1ce394..9e6654ee9ca 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -233,6 +233,11 @@ typedef struct { * is not needed. * @numa_mem_supported: * true if '--numa node.mem' option is supported and false otherwise + * @numa_skip_ram_container: + * If false, numa memory init creates the MachineState.ram memory region + * with all numa node regions packed densely within it. If true, the .r= am + * region is not created. Machines can use this e.g., to place NUMA + * regions sparsely within the address space. * @hotplug_allowed: * If the hook is provided, then it'll be called for each device * hotplug to check whether the device hotplug is allowed. Return @@ -311,6 +316,7 @@ struct MachineClass { bool nvdimm_supported; bool numa_mem_supported; bool auto_enable_numa; + bool numa_skip_ram_container; bool cpu_cluster_has_numa_boundary; SMPCompatProps smp_props; const char *default_ram_id; diff --git a/include/system/numa.h b/include/system/numa.h index 1044b0eb6e9..001e872d33e 100644 --- a/include/system/numa.h +++ b/include/system/numa.h @@ -38,6 +38,7 @@ enum { typedef struct NodeInfo { uint64_t node_mem; struct HostMemoryBackend *node_memdev; + MemoryRegion *node_mr; bool present; bool has_cpu; bool has_gi; diff --git a/hw/core/numa.c b/hw/core/numa.c index 218576f7455..d84b2d70849 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -623,19 +623,46 @@ static void complete_init_numa_distance(MachineState = *ms) } } =20 -static void numa_init_memdev_container(MachineState *ms, MemoryRegion *ram) +/* + * Consume all NUMA memory backends and store the regions in NodeInfo.node= _mr. + */ +static void numa_init_memdev(MachineState *ms) { int i; - uint64_t addr =3D 0; =20 for (i =3D 0; i < ms->numa_state->num_nodes; i++) { - uint64_t size =3D ms->numa_state->nodes[i].node_mem; HostMemoryBackend *backend =3D ms->numa_state->nodes[i].node_memde= v; if (!backend) { continue; } MemoryRegion *seg =3D machine_consume_memdev(ms, backend); - memory_region_add_subregion(ram, addr, seg); + ms->numa_state->nodes[i].node_mr =3D seg; + } +} + +/* + * Consume all NUMA memory backends as with numa_init_memdev, packing them + * densely into a MachineState.ram "container" region. + */ +static void numa_init_memdev_container(MachineState *ms) +{ + int i; + MachineClass *mc =3D MACHINE_GET_CLASS(ms); + uint64_t addr =3D 0; + + ms->ram =3D g_new(MemoryRegion, 1); + memory_region_init(ms->ram, OBJECT(ms), mc->default_ram_id, + ms->ram_size); + + numa_init_memdev(ms); + + for (i =3D 0; i < ms->numa_state->num_nodes; i++) { + uint64_t size =3D ms->numa_state->nodes[i].node_mem; + MemoryRegion *seg =3D ms->numa_state->nodes[i].node_mr; + if (!seg) { + continue; + } + memory_region_add_subregion(ms->ram, addr, seg); addr +=3D size; } } @@ -706,10 +733,11 @@ void numa_complete_configuration(MachineState *ms) " properties are mutually exclusive"); exit(1); } - ms->ram =3D g_new(MemoryRegion, 1); - memory_region_init(ms->ram, OBJECT(ms), mc->default_ram_id, - ms->ram_size); - numa_init_memdev_container(ms, ms->ram); + if (mc->numa_skip_ram_container) { + numa_init_memdev(ms); + } else { + numa_init_memdev_container(ms); + } } /* QEMU needs at least all unique node pair distances to build * the whole NUMA distance table. QEMU treats the distance table --=20 2.47.1 From nobody Wed Mar 5 02:31:25 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1740996550; cv=none; d=zohomail.com; s=zohoarc; b=PMWOu05cYrrsiu53PXgDN8QoW5A+85iiyiLXuevcMNx07vrJH8Wx5JxXcY+Q7m0JGGyCdrc4eDigPJQpU4tGxy1mn4QNtUnbFuDAmIol1PoxTcacZZM2wMhBRjwJukCuHl1V0l/sQMb3YWBYBUpAdEh5bSXWq6SBC3GOluDUrVg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1740996550; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=e/sa2uc6or1aHtVvQRaym8GaDvBwv+dby+++CN327KM=; b=jzWbKWhE1K98JKaUsGFtuQSxwTf95nw5EKSxBTEPo9G4L4UjmEXEnKjn75sFJTzerh9J2VMYqQ5BQKcmQY/UVzRIAU1SnLRKakSC6Tgb6gBfhxtZZkHG7Vqej29EhNJwkkJxTRd7PRsDPe3wg84IF3e7lXWRzPyJWNNk9i62Q38= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17409965502101016.2106282341892; Mon, 3 Mar 2025 02:09:10 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tp2in-0001FP-79; Mon, 03 Mar 2025 05:08:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tp2iP-0001DJ-Pa; Mon, 03 Mar 2025 05:08:09 -0500 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tp2iL-0003EX-Pq; Mon, 03 Mar 2025 05:08:01 -0500 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2fe82414cf7so8258466a91.0; Mon, 03 Mar 2025 02:07:56 -0800 (PST) Received: from wheely.local0.net ([118.208.151.101]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fea67a7ddesm8539779a91.27.2025.03.03.02.07.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Mar 2025 02:07:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740996475; x=1741601275; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e/sa2uc6or1aHtVvQRaym8GaDvBwv+dby+++CN327KM=; b=UgvwSdnUR2PbbHx8UqK5hbmKyRTAyvbQwurcAxwPbPRcDLZYiIQPVQEaUgyJVsAaD0 /aCYfnCY2A4/uFMSW87vLM6fioz5ItnjcpBJJTsw82whifdolbs3kB8h3iBFW3RrNnFK DpEkQrHdWcUK6LIH69ddHOELm0yvAy0hvq/i9n1TYML/ur8OXjh1NFqGpVb24Yqzqqti y5qi5sd4HG6b9qqUWc2/VnfVOLiDz6ginMcu7ukInBDXNw5l1PWW5VCuMHtai+R/yMdp Ho/A4TFahSSn+Hxtxv4T++O7ijYs05DNFWly6bPYZjt2qsbmEBH8XAvj2b+E9hIJ0lOo Y/bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740996475; x=1741601275; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e/sa2uc6or1aHtVvQRaym8GaDvBwv+dby+++CN327KM=; b=lQ0gPEzRduniw1xjMFbW28XEDRll16aXsdPrKALmwaAEs8SJFczRIoSit1YWEE562B zS6lh0buFKC1TKKO0ui831bFauJ+N4VGgVGK1wSrCgEcDM7HeQIo0bDu5qHVN9RuHM7a 9ubeNDmIHL39E0gtJg5IWr48NUXPRhvQEu9KTIKz1b+GAqIuy4w7BjbwWTxWxhMQreQb xW5wQ7qLex3z3dSjPoQMySFWgMGms6cZOTkSr/cgazMuvrAMfvtZ5fj5y+wK+KGaibRs wYCloGuGXXJGEYI4BMLI9whWcTpFo+SN/h6dzsQGH66NJWuhaZCcfA/IQnDQXkIlXfQp LaJg== X-Forwarded-Encrypted: i=1; AJvYcCUVSzUcqsvmMSnfO2p32rzg9iftVH3dl+Pm2NF3kDthqDh6yP5ILA8DlDqOU7XHNd2t4gVenKRRXCo9@nongnu.org X-Gm-Message-State: AOJu0YwnBGsbCQgLYkz66l65PtdBpMBq3rQFdGKBVg4znFFqtpc1lVom nbS5rFarv7jH+lQvGUxq7GRhiHEY34lSeKlticYEXPh3+f0HsloHC2dXQg== X-Gm-Gg: ASbGncuuE6QRZ6XMm6wCDS8mMFLGMwxGvZqhmzqed1MofHZdOJCQ6Uq5tRXXl518v5p 3YAd+7MT2Jwd4sn4ptpM/TAHMAcRm01E8eLhStEBH7CFlZtZDhXmT6a4efwhoy227PrOft0pDWP cC5bP2n6CiHFfLw1lA6U5eQ47eTi+ECwK2iPq3oPrApSWaN6NJdW8TyQQkRXBF3IEYEcuMjwq4w HujBOHU5+wcZ8pnbG8DQAMtWb4z1MdyvH+mVj27uZu48l66JfLw1M9MDwwW+bkKoussNv5ZrbUn N+7k+IFH3Sr0R/JsMCaoakcGYKGVKuhWbIjRyHvhP9q7o6K4puk= X-Google-Smtp-Source: AGHT+IFoACEcmG+tdYphPQmFkTUd4jDuwORcnI1sdYpXaHN8bcxvLxBXvARrul/ZsoVFawWvO9Po4Q== X-Received: by 2002:a17:90b:2786:b0:2ee:ab11:fab2 with SMTP id 98e67ed59e1d1-2febabedbd6mr16537625a91.22.1740996475369; Mon, 03 Mar 2025 02:07:55 -0800 (PST) From: Nicholas Piggin To: qemu-ppc@nongnu.org Cc: Nicholas Piggin , qemu-devel@nongnu.org, Eduardo Habkost , Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Zhao Liu , =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20Barrat?= , Igor Mammedov Subject: [PATCH 3/3] ppc/pnv: Enable sparse chip RAM memory addresses Date: Mon, 3 Mar 2025 20:07:32 +1000 Message-ID: <20250303100732.576457-4-npiggin@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250303100732.576457-1-npiggin@gmail.com> References: <20250303100732.576457-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=npiggin@gmail.com; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1740996552693019000 Content-Type: text/plain; charset="utf-8" Power CPUs place RAM memory regions for each chip (NUMA node) at fixed locations in the real address space, resulting in a sparse (disjoint) RAM address layout. Use the new NUMA machine class attribute numa_skip_ram_container to allow pnv machine init to lay out NUMA node memory regions into the system address space in the proper location rather than packing them densely from address 0. With the following options: -smp 2,sockets=3D2 -m 4g -object memory-backend-ram,size=3D2G,id=3Dmem0 -object memory-backend-ram,size=3D2G,id=3Dmem1 -numa node,nodeid=3D0,memdev=3Dmem0,cpus=3D0 -numa node,nodeid=3D1,memdev=3Dmem1,cpus=3D1 Linux (PowerNV) now boots with: node 0: [mem 0x0000000000000000-0x000000007fffffff] node 1: [mem 0x0000100000000000-0x000010007fffffff] Prior to this change: node 0: [mem 0x0000000000000000-0x000000007fffffff] node 1: [mem 0x0000000000000000-0x00000000ffffffff] Signed-off-by: Nicholas Piggin --- hw/ppc/pnv.c | 37 ++++++++++++++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c index 5f2041f7f9d..b6308593335 100644 --- a/hw/ppc/pnv.c +++ b/hw/ppc/pnv.c @@ -966,7 +966,24 @@ static void pnv_init(MachineState *machine) exit(EXIT_FAILURE); } =20 - memory_region_add_subregion(get_system_memory(), 0, machine->ram); + if (machine->ram) { + memory_region_add_subregion(get_system_memory(), 0, machine->ram); + } else if (machine->numa_state) { + for (i =3D 0; i < machine->numa_state->num_nodes; i++) { + MemoryRegion *mr =3D machine->numa_state->nodes[i].node_mr; + + /* + * powernv uses numa_mem_align_shift to derive the base RAM ad= dress + * for each chip addr =3D Chip Number << shift. + */ + chip_ram_start =3D (uint64_t)i << mc->numa_mem_align_shift; + if (!mr) { + continue; + } + memory_region_add_subregion(get_system_memory(), chip_ram_star= t, + mr); + } + } =20 /* * Create our simple PNOR device @@ -1100,20 +1117,30 @@ static void pnv_init(MachineState *machine) exit(1); } =20 + chip_ram_start =3D 0; pnv->chips =3D g_new0(PnvChip *, pnv->num_chips); for (i =3D 0; i < pnv->num_chips; i++) { char chip_name[32]; Object *chip =3D OBJECT(qdev_new(chip_typename)); - uint64_t chip_ram_size =3D pnv_chip_get_ram_size(pnv, i); + uint64_t chip_ram_size; =20 pnv->chips[i] =3D PNV_CHIP(chip); =20 + if (machine->numa_state) { + chip_ram_start =3D (uint64_t)i << mc->numa_mem_align_shift; + chip_ram_size =3D machine->numa_state->nodes[i].node_mem; + } else { + chip_ram_size =3D pnv_chip_get_ram_size(pnv, i); + } + /* Distribute RAM among the chips */ object_property_set_int(chip, "ram-start", chip_ram_start, &error_fatal); object_property_set_int(chip, "ram-size", chip_ram_size, &error_fatal); - chip_ram_start +=3D chip_ram_size; + if (!machine->numa_state) { + chip_ram_start +=3D chip_ram_size; + } =20 snprintf(chip_name, sizeof(chip_name), "chip[%d]", i); object_property_add_child(OBJECT(pnv), chip_name, chip); @@ -2680,6 +2707,7 @@ static void pnv_machine_power8_class_init(ObjectClass= *oc, void *data) =20 mc->desc =3D "IBM PowerNV (Non-Virtualized) POWER8"; mc->default_cpu_type =3D POWERPC_CPU_TYPE_NAME("power8_v2.0"); + mc->numa_mem_align_shift =3D 42; compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat= )); =20 xic->icp_get =3D pnv_icp_get; @@ -2709,6 +2737,7 @@ static void pnv_machine_power9_class_init(ObjectClass= *oc, void *data) =20 mc->desc =3D "IBM PowerNV (Non-Virtualized) POWER9"; mc->default_cpu_type =3D POWERPC_CPU_TYPE_NAME("power9_v2.2"); + mc->numa_mem_align_shift =3D 42; compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat= )); =20 xfc->match_nvt =3D pnv_match_nvt; @@ -2747,6 +2776,7 @@ static void pnv_machine_p10_common_class_init(ObjectC= lass *oc, void *data) }; =20 mc->default_cpu_type =3D POWERPC_CPU_TYPE_NAME("power10_v2.0"); + mc->numa_mem_align_shift =3D 44; compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat= )); =20 mc->alias =3D "powernv"; @@ -2951,6 +2981,7 @@ static void pnv_machine_class_init(ObjectClass *oc, v= oid *data) =20 mc->numa_mem_supported =3D true; mc->auto_enable_numa =3D true; + mc->numa_skip_ram_container =3D true; =20 mc->cpu_index_to_instance_props =3D pnv_cpu_index_to_props; mc->get_default_cpu_node_id =3D pnv_get_default_cpu_node_id; --=20 2.47.1