From nobody Fri Apr 26 17:21:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1631298002; cv=none; d=zohomail.com; s=zohoarc; b=FSR6cAD/8hWukBm7kv4Pa1wC4+FUKUf7Bg6lS/oV/CFGPNLurm37Isb8NRM+9ZN+s6Mke+Cv7FEsA2iV0kFFvcCtPD2eFVq9ELlhCGPyUzk+b7U26ZzyXeX4j7lZtTAKoPUJYRtHv77EykGywAKCCMgsqWnaQddVfi/Iv3bvtVE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1631298002; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=Pi6yuMUH/Hje2P5MmRyjScNmoovfYd7eFi3wUXINX8Y=; b=jJzuxiRg2k4a46mQUq/DawDlCM4YCaQnIk/1HO7zbl51MI8AQ2RPjbb+T6TvRYSLmsbXyxUwuanI2brGvsdOFoX4ZCRw6x6UJ7RnsAfY4zO5e+kHsXSzo8B2O4h9e6F5JqJ7yrlv5dOZnkz/XA46jdGz3re7Wc/CSekRblDLMkg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1631298002569746.0105962566192; Fri, 10 Sep 2021 11:20:02 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.184709.333424 (Exim 4.92) (envelope-from ) id 1mOl7b-0003Qo-Ga; Fri, 10 Sep 2021 18:19:31 +0000 Received: by outflank-mailman (output) from mailman id 184709.333424; Fri, 10 Sep 2021 18:19:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOl7b-0003Qh-Di; Fri, 10 Sep 2021 18:19:31 +0000 Received: by outflank-mailman (input) for mailman id 184709; Fri, 10 Sep 2021 18:19:31 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOl7a-0003AT-Vv for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:19:31 +0000 Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a408877b-c702-4904-a009-8007a6c3139b; Fri, 10 Sep 2021 18:19:25 +0000 (UTC) Received: by mail-lf1-x133.google.com with SMTP id t19so5738077lfe.13 for ; Fri, 10 Sep 2021 11:19:25 -0700 (PDT) Received: from otyshchenko.router ([212.22.223.21]) by smtp.gmail.com with ESMTPSA id a7sm625045lfi.15.2021.09.10.11.19.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Sep 2021 11:19:24 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a408877b-c702-4904-a009-8007a6c3139b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Pi6yuMUH/Hje2P5MmRyjScNmoovfYd7eFi3wUXINX8Y=; b=JyXhznYvxdb0bvxR3UsrKgnKsipi87KCT1G6tVcKV02z0YOthdqcv3HpPjSOuUnEJ7 h861M2f6pupojWC4neHnTfdHkXJK3Afm/m4NHlW25yD7bg1X6m5gl6mJfzS9Z5LdPdpz G3gF06YS+2QzNmvH37P6ZUxt2GDpXN3KrATRRkGhI4rUUbWLkrlDAVdCb4oW1EBZTbvo YOBCePVzygHtTfzrn6USJJ5ZogaLeG+nOQdvzotaSwMm/Nd9THm6PrVxRcQ407EJNiBO RaWVGHqLUV8AOJha3ij2fChrDqAtZpC1PLV5bT5ewLYepus1Ku1NbFBYAWUg5YVTI6wE jzUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Pi6yuMUH/Hje2P5MmRyjScNmoovfYd7eFi3wUXINX8Y=; b=XG+n3NJ//tePK0nQ9Vm0gkZD50quHIjYOjGLh/P3BlOUQSYrO6FNcNKOtazO6pG36x sHr8YcYcw3z7CMcxMoq6KueHMbRaG8BKGC8SwMzx1rgETa5gd/aSlQsGSELWnT4D/+ls H4d493RPVIMEmMeldfILT2++xvOSxwRp9kps47+47mcERF0g+maJJwRMnbTLJJdPR4Wk aRMoCJlrrXwqU02ErZyXDjW4dZKU6o2I+cJ6rc8Jbpi02HfyM2huyqjucMStv6LhQvO8 3EQtKoeoA8Cm0VbRAmYPgw5rVl5msZ848oDzUXXmJL0O/SmG2HKJPa/1g/Wk0yDpXMKJ i9+A== X-Gm-Message-State: AOAM533Ncr4//lIW/sCJYsj2Fp4fNstH/4tWgIrxDJLa0KEE9AEqwFLj s67jGAO95ywcFJ/EdfIVhHRseyRnzYg= X-Google-Smtp-Source: ABdhPJzNQCIIRu+uqLkS0O2ZDBsx9oexcKKbmKXJ78NwnvHvyco3jTOxEc+D9ChSiOnM07fIgLxwwQ== X-Received: by 2002:a19:c753:: with SMTP id x80mr1170968lff.267.1631297964434; Fri, 10 Sep 2021 11:19:24 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Cc: Oleksandr Tyshchenko , Ian Jackson , Wei Liu , Anthony PERARD , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Juergen Gross , Volodymyr Babchuk , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Henry Wang , Bertrand Marquis , Wei Chen Subject: [PATCH V2 1/3] xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo Date: Fri, 10 Sep 2021 21:18:42 +0300 Message-Id: <1631297924-8658-2-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1631298905763100001 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko We need to pass info about maximum supported guest address space size to the toolstack on Arm in order to properly calculate the base and size of the extended region (safe range) for the guest. The extended region is unused address space which could be safely used by domain for foreign/grant mappings on Arm. The extended region itself will be handled by the subsequents patch. Use p2m_ipa_bits variable on Arm, the x86 equivalent is hap_paddr_bits. As we change the size of structure bump the interface version. Suggested-by: Julien Grall Signed-off-by: Oleksandr Tyshchenko --- Please note, that recent review comments [1] haven't been addressed yet. It is not forgotten, some clarification is needed. It will be addressed for= the next version. [1] https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@= citrix.com/ Changes since RFC: - update patch subject/description - replace arch-specific sub-struct with common gpaddr_bits field and update code to reflect that --- tools/include/libxl.h | 7 +++++++ tools/libs/light/libxl.c | 2 ++ tools/libs/light/libxl_types.idl | 2 ++ xen/arch/arm/sysctl.c | 2 ++ xen/arch/x86/sysctl.c | 2 ++ xen/include/public/sysctl.h | 3 ++- 6 files changed, 17 insertions(+), 1 deletion(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index b9ba16d..da44944 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -855,6 +855,13 @@ typedef struct libxl__ctx libxl_ctx; */ #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1 =20 + /* + * LIBXL_HAVE_PHYSINFO_GPADDR_BITS + * + * If this is defined, libxl_physinfo has a "gpaddr_bits" field. + */ + #define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1 + /* * LIBXL_HAVE_DOMINFO_OUTSTANDING_MEMKB 1 * diff --git a/tools/libs/light/libxl.c b/tools/libs/light/libxl.c index 204eb0b..c86624f 100644 --- a/tools/libs/light/libxl.c +++ b/tools/libs/light/libxl.c @@ -405,6 +405,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *= physinfo) physinfo->cap_vmtrace =3D !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_vmtrace); =20 + physinfo->gpaddr_bits =3D xcphysinfo.gpaddr_bits; + GC_FREE; return 0; } diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_type= s.idl index 3f9fff6..f7437e4 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -1061,6 +1061,8 @@ libxl_physinfo =3D Struct("physinfo", [ ("cap_shadow", bool), ("cap_iommu_hap_pt_share", bool), ("cap_vmtrace", bool), + + ("gpaddr_bits", uint32), ], dir=3DDIR_OUT) =20 libxl_connectorinfo =3D Struct("connectorinfo", [ diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c index f87944e..91dca4f 100644 --- a/xen/arch/arm/sysctl.c +++ b/xen/arch/arm/sysctl.c @@ -15,6 +15,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi) { pi->capabilities |=3D XEN_SYSCTL_PHYSCAP_hvm | XEN_SYSCTL_PHYSCAP_hap; + + pi->gpaddr_bits =3D p2m_ipa_bits; } =20 long arch_do_sysctl(struct xen_sysctl *sysctl, diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c index aff52a1..7b14865 100644 --- a/xen/arch/x86/sysctl.c +++ b/xen/arch/x86/sysctl.c @@ -135,6 +135,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi) pi->capabilities |=3D XEN_SYSCTL_PHYSCAP_hap; if ( IS_ENABLED(CONFIG_SHADOW_PAGING) ) pi->capabilities |=3D XEN_SYSCTL_PHYSCAP_shadow; + + pi->gpaddr_bits =3D hap_paddr_bits; } =20 long arch_do_sysctl( diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index 039ccf8..f53b42e 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -35,7 +35,7 @@ #include "domctl.h" #include "physdev.h" =20 -#define XEN_SYSCTL_INTERFACE_VERSION 0x00000013 +#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014 =20 /* * Read console content from Xen buffer ring. @@ -120,6 +120,7 @@ struct xen_sysctl_physinfo { uint64_aligned_t outstanding_pages; uint64_aligned_t max_mfn; /* Largest possible MFN on this host */ uint32_t hw_cap[8]; + uint32_t gpaddr_bits; }; =20 /* --=20 2.7.4 From nobody Fri Apr 26 17:21:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1631298002; cv=none; d=zohomail.com; s=zohoarc; b=JoyB/cYPOKHbZ6Af0b8qJk/eSzZlAVzuTAjsmyTBCsQONOa2jOKnHyLuzkAzl9ndPBRmKRA2ZVLpvsGv9Jg3hem1csEHt6PsiBS2F43Mjs5x/xCbyv+iO7o5qz8mnhznGdMeuoyuXQYJ8/2wXb0hd+AFJ2LN32afHY2CrzE7ZdA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1631298002; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=6OqwJJkVPWfknF1ipcJOm+d5im8+QUIzbjtyLrkFb/M=; b=aI6iFaqaFiOdFW1YuOMLGMXnuewckdK7koddrpL1MCemLBu9xQsIZiCUapCtHxJqoh9ABJuPVcfwMNK4lWvv1NjjH7y+B0XKKeaUAiN0GHeGpaDJOi+dhB9VWwBp3y7B29AItBEGRsJ01LgsxJORz58NxtRIUpJWoJOx9+kasvM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1631298002211934.9329187730549; Fri, 10 Sep 2021 11:20:02 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.184710.333436 (Exim 4.92) (envelope-from ) id 1mOl7g-0003kn-Qz; Fri, 10 Sep 2021 18:19:36 +0000 Received: by outflank-mailman (output) from mailman id 184710.333436; Fri, 10 Sep 2021 18:19:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOl7g-0003ke-Mv; Fri, 10 Sep 2021 18:19:36 +0000 Received: by outflank-mailman (input) for mailman id 184710; Fri, 10 Sep 2021 18:19:36 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOl7f-0003AT-WB for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:19:36 +0000 Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id d8cdc84b-f567-4eae-bb76-6688abd738e9; Fri, 10 Sep 2021 18:19:26 +0000 (UTC) Received: by mail-lf1-x133.google.com with SMTP id k4so5789292lfj.7 for ; Fri, 10 Sep 2021 11:19:26 -0700 (PDT) Received: from otyshchenko.router ([212.22.223.21]) by smtp.gmail.com with ESMTPSA id a7sm625045lfi.15.2021.09.10.11.19.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Sep 2021 11:19:25 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d8cdc84b-f567-4eae-bb76-6688abd738e9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6OqwJJkVPWfknF1ipcJOm+d5im8+QUIzbjtyLrkFb/M=; b=W1rFwzZH8zTl0VZSHEaQMpOhOo3ip+nzB56r7+WBk0X0NEQ+1VqIuPamRv5m7WGHKq ZTE8m7ngwpqPWIOhMbglmfpRyUmI4WO/A3qpexBiLh9klKvPdVEA08/c4PmUPjaMwd59 ML/SD3wxAnA0XutCWMsq7tUea45boYezdoYcE5DCBrlLuDqbuf/OZB5FJOseCB3BMtEp qJX+2YXswxPMAauDRH0x7rBr+GpI0RfzCi/tGRsjfJ/Y4rjlWNlq0ZTROXytKQhcoB2p DTyuLryBdFnzUYOGnGhd0S0uDLyahDxQQSW/e67iculbiHvaDzOkZPRwQafgoc+RZSaC n/dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6OqwJJkVPWfknF1ipcJOm+d5im8+QUIzbjtyLrkFb/M=; b=YH3AHKjmXwQRp/i1wzAlvFh6c/s3hCN4VMQCCTYetwyw5rKVOwY5gLC9vrd4Kvct4o gKY5uYTY5cgxM2iK4EPXvZ99RAUe/Hku6iqva6wTF2Qgvzybenvmc07Nn9fj63bB5ELt XGBkHirk6A/JMo66QDR0s7oZwCEFefRJ4Nm2ND0NvTcNbjKNZsiOa9p9lP5E2P0PJVwW 3H//DBIb67bYlFL2AL1wqlFsysclgTsAPS/v9ixEpS4CYchtscHsLYQMgKxrDhnOQi5p 0laPPSH/DLBjOw1SJrUgJxobIAxooD+FPrZexVdP+jQIiOS2lKUcBMEmTobxJCYzTw5C UwhA== X-Gm-Message-State: AOAM531j7+eXRygBUl5c1Bq3zmNkO9MkH8HkV9o7pxwL/9gkVn5mebfc sO/t+bGNV4Jdc6/fMER99n23STuFGU0= X-Google-Smtp-Source: ABdhPJwOkAqyiJqstacgeYt9Ih7dmN6D82FgzbEmHKzP6tBr/Ie+o6OyzcVjMlFpoBHbXhUA1N4gcQ== X-Received: by 2002:a05:6512:3b9d:: with SMTP id g29mr4866022lfv.77.1631297965309; Fri, 10 Sep 2021 11:19:25 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Cc: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Henry Wang , Bertrand Marquis , Wei Chen Subject: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0 Date: Fri, 10 Sep 2021 21:18:43 +0300 Message-Id: <1631297924-8658-3-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1631298004436100002 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko The extended region (safe range) is a region of guest physical address space which is unused and could be safely used to create grant/foreign mappings instead of wasting real RAM pages from the domain memory for establishing these mappings. The extended regions are chosen at the domain creation time and advertised to it via "reg" property under hypervisor node in the guest device-tree. As region 0 is reserved for grant table space (always present), the indexes for extended regions are 1...N. If extended regions could not be allocated for some reason, Xen doesn't fail and behaves as usual, so only inserts region 0. Please note the following limitations: - The extended region feature is only supported for 64-bit domain. - The ACPI case is not covered. *** As Dom0 is direct mapped domain on Arm (e.g. MFN =3D=3D GFN) the algorithm to choose extended regions for it is different in comparison with the algorithm for non-direct mapped DomU. What is more, that extended regions should be chosen differently whether IOMMU is enabled or not. Provide RAM not assigned to Dom0 if IOMMU is disabled or memory holes found in host device-tree if otherwise. Make sure that extended regions are 2MB-aligned and located within maximum possible addressable physical memory range. The maximum number of extended regions is 128. Suggested-by: Julien Grall Signed-off-by: Oleksandr Tyshchenko --- Changes since RFC: - update patch description - drop uneeded "extended-region" DT property --- xen/arch/arm/domain_build.c | 226 ++++++++++++++++++++++++++++++++++++++++= +++- 1 file changed, 224 insertions(+), 2 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 206038d..070ec27 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct domai= n *d, return res; } =20 +static int __init add_ext_regions(unsigned long s, unsigned long e, void *= data) +{ + struct meminfo *ext_regions =3D data; + paddr_t start, size; + + if ( ext_regions->nr_banks >=3D ARRAY_SIZE(ext_regions->bank) ) + return 0; + + /* Both start and size of the extended region should be 2MB aligned */ + start =3D (s + SZ_2M - 1) & ~(SZ_2M - 1); + if ( start > e ) + return 0; + + size =3D (e - start + 1) & ~(SZ_2M - 1); + if ( !size ) + return 0; + + ext_regions->bank[ext_regions->nr_banks].start =3D start; + ext_regions->bank[ext_regions->nr_banks].size =3D size; + ext_regions->nr_banks ++; + + return 0; +} + +/* + * The extended regions will be prevalidated by the memory hotplug path + * in Linux which requires for any added address range to be within maximum + * possible addressable physical memory range for which the linear mapping + * could be created. + * For 48-bit VA space size the maximum addressable range are: + * 0x40000000 - 0x80003fffffff + */ +#define EXT_REGION_START 0x40000000ULL +#define EXT_REGION_END 0x80003fffffffULL + +static int __init find_unallocated_memory(const struct kernel_info *kinfo, + struct meminfo *ext_regions) +{ + const struct meminfo *assign_mem =3D &kinfo->mem; + struct rangeset *unalloc_mem; + paddr_t start, end; + unsigned int i; + int res; + + dt_dprintk("Find unallocated memory for extended regions\n"); + + unalloc_mem =3D rangeset_new(NULL, NULL, 0); + if ( !unalloc_mem ) + return -ENOMEM; + + /* Start with all available RAM */ + for ( i =3D 0; i < bootinfo.mem.nr_banks; i++ ) + { + start =3D bootinfo.mem.bank[i].start; + end =3D bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1; + res =3D rangeset_add_range(unalloc_mem, start, end); + if ( res ) + { + printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n", + start, end); + goto out; + } + } + + /* Remove RAM assigned to Dom0 */ + for ( i =3D 0; i < assign_mem->nr_banks; i++ ) + { + start =3D assign_mem->bank[i].start; + end =3D assign_mem->bank[i].start + assign_mem->bank[i].size - 1; + res =3D rangeset_remove_range(unalloc_mem, start, end); + if ( res ) + { + printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n", + start, end); + goto out; + } + } + + /* Remove reserved-memory regions */ + for ( i =3D 0; i < bootinfo.reserved_mem.nr_banks; i++ ) + { + start =3D bootinfo.reserved_mem.bank[i].start; + end =3D bootinfo.reserved_mem.bank[i].start + + bootinfo.reserved_mem.bank[i].size - 1; + res =3D rangeset_remove_range(unalloc_mem, start, end); + if ( res ) + { + printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n", + start, end); + goto out; + } + } + + /* Remove grant table region */ + start =3D kinfo->gnttab_start; + end =3D kinfo->gnttab_start + kinfo->gnttab_size - 1; + res =3D rangeset_remove_range(unalloc_mem, start, end); + if ( res ) + { + printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n", + start, end); + goto out; + } + + start =3D EXT_REGION_START; + end =3D min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END); + res =3D rangeset_report_ranges(unalloc_mem, start, end, + add_ext_regions, ext_regions); + if ( res ) + ext_regions->nr_banks =3D 0; + else if ( !ext_regions->nr_banks ) + res =3D -ENOENT; + +out: + rangeset_destroy(unalloc_mem); + + return res; +} + +static int __init find_memory_holes(const struct kernel_info *kinfo, + struct meminfo *ext_regions) +{ + struct dt_device_node *np; + struct rangeset *mem_holes; + paddr_t start, end; + unsigned int i; + int res; + + dt_dprintk("Find memory holes for extended regions\n"); + + mem_holes =3D rangeset_new(NULL, NULL, 0); + if ( !mem_holes ) + return -ENOMEM; + + /* Start with maximum possible addressable physical memory range */ + start =3D EXT_REGION_START; + end =3D min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END); + res =3D rangeset_add_range(mem_holes, start, end); + if ( res ) + { + printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n", + start, end); + goto out; + } + + /* Remove all regions described by "reg" property (MMIO, RAM, etc) */ + dt_for_each_device_node( dt_host, np ) + { + unsigned int naddr; + u64 addr, size; + + naddr =3D dt_number_of_address(np); + + for ( i =3D 0; i < naddr; i++ ) + { + res =3D dt_device_get_address(np, i, &addr, &size); + if ( res ) + { + printk(XENLOG_ERR "Unable to retrieve address %u for %s\n", + i, dt_node_full_name(np)); + goto out; + } + + start =3D addr & PAGE_MASK; + end =3D PAGE_ALIGN(addr + size) - 1; + res =3D rangeset_remove_range(mem_holes, start, end); + if ( res ) + { + printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64= "\n", + start, end); + goto out; + } + } + } + + start =3D EXT_REGION_START; + end =3D min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END); + res =3D rangeset_report_ranges(mem_holes, start, end, + add_ext_regions, ext_regions); + if ( res ) + ext_regions->nr_banks =3D 0; + else if ( !ext_regions->nr_banks ) + res =3D -ENOENT; + +out: + rangeset_destroy(mem_holes); + + return res; +} + static int __init make_hypervisor_node(struct domain *d, const struct kernel_info *kinfo, int addrcells, int sizecells) @@ -731,11 +921,13 @@ static int __init make_hypervisor_node(struct domain = *d, const char compat[] =3D "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\= 0" "xen,xen"; - __be32 reg[4]; + __be32 reg[(NR_MEM_BANKS + 1) * 4]; gic_interrupt_t intr; __be32 *cells; int res; void *fdt =3D kinfo->fdt; + struct meminfo *ext_regions; + unsigned int i; =20 dt_dprintk("Create hypervisor node\n"); =20 @@ -757,12 +949,42 @@ static int __init make_hypervisor_node(struct domain = *d, if ( res ) return res; =20 + ext_regions =3D xzalloc(struct meminfo); + if ( !ext_regions ) + return -ENOMEM; + + if ( is_32bit_domain(d) ) + printk(XENLOG_WARNING "The extended region is only supported for 6= 4-bit guest\n"); + else + { + if ( !is_iommu_enabled(d) ) + res =3D find_unallocated_memory(kinfo, ext_regions); + else + res =3D find_memory_holes(kinfo, ext_regions); + + if ( res ) + printk(XENLOG_WARNING "Failed to allocate extended regions\n"); + } + /* reg 0 is grant table space */ cells =3D ®[0]; dt_child_set_range(&cells, addrcells, sizecells, kinfo->gnttab_start, kinfo->gnttab_size); + /* reg 1...N are extended regions */ + for ( i =3D 0; i < ext_regions->nr_banks; i++ ) + { + u64 start =3D ext_regions->bank[i].start; + u64 size =3D ext_regions->bank[i].size; + + dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n", + i, start, start + size); + + dt_child_set_range(&cells, addrcells, sizecells, start, size); + } + xfree(ext_regions); + res =3D fdt_property(fdt, "reg", reg, - dt_cells_to_size(addrcells + sizecells)); + dt_cells_to_size(addrcells + sizecells) * (i + 1)); if ( res ) return res; =20 --=20 2.7.4 From nobody Fri Apr 26 17:21:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1631298003; cv=none; d=zohomail.com; s=zohoarc; b=hqYMa+joB0zpoY6y/H6WAi0FeoFrYhsRFnAWK0XP4gFJuqAFsDjMcajilPSQ/Pj+N4k/4m8EUAaWPS6lynWYpzkwaWgSEHKJB2UYq8RGb4RRCPFoigSKl/SPIBZYZZlW2tRw4tcIKK6R0/Xk8OLOHXnQu4AKMf/Zt08+3YzMH7Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1631298003; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=Eoy+Pchg4s0QithVevz+IoMG+lYh7VneYRIDbLe6C1s=; b=aZKnBhAPY6TqsehpCwC0N+ZOfSkUAVLUeZfayPALBFUZfeU3OM7LmdOzdYsPvTeGg/DPomHPs0pLVa37qt96IJSvW7qDaozo8KRWypnyqMQqRmxq/Xql2eohIr+Q5ibV2Nuxn2cyskrPVjkCSzHuN3s+tXEPEAVgsdRdT1ILDy8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1631298003450276.11187133836495; Fri, 10 Sep 2021 11:20:03 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.184711.333447 (Exim 4.92) (envelope-from ) id 1mOl7m-00049L-9R; Fri, 10 Sep 2021 18:19:42 +0000 Received: by outflank-mailman (output) from mailman id 184711.333447; Fri, 10 Sep 2021 18:19:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOl7m-00049A-4z; Fri, 10 Sep 2021 18:19:42 +0000 Received: by outflank-mailman (input) for mailman id 184711; Fri, 10 Sep 2021 18:19:41 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mOl7k-0003AT-WD for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:19:41 +0000 Received: from mail-lf1-x136.google.com (unknown [2a00:1450:4864:20::136]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 6dc5f12d-6d91-4378-a5aa-df471a9af16a; Fri, 10 Sep 2021 18:19:27 +0000 (UTC) Received: by mail-lf1-x136.google.com with SMTP id f18so5751061lfk.12 for ; Fri, 10 Sep 2021 11:19:27 -0700 (PDT) Received: from otyshchenko.router ([212.22.223.21]) by smtp.gmail.com with ESMTPSA id a7sm625045lfi.15.2021.09.10.11.19.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Sep 2021 11:19:25 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6dc5f12d-6d91-4378-a5aa-df471a9af16a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Eoy+Pchg4s0QithVevz+IoMG+lYh7VneYRIDbLe6C1s=; b=UBYQf8hSPuukUyAYT2w/QXM9jAPllIyX8yIQsyPXp3C0DClgZqz8w6lhVCJYdGYnsg wXWf7hb9DsR5gTB2nuJUaciQJW83S1ORICEsJeiI3s2gWGy5sHJcfM760Pfu5BxXZWwY zVhW2B/xUXLj1mz3cK9S/Oz+bFkBeI6Nxk+477wBEjNFL6ddowBMUTJX8LIE2syddjON bAWPyXbL7Q3ErzNrgElyqWjxOrx3z2U/yH4DkNRSn8blmVr7HP4IJuL6OzRUTCaoNoWl QX3v+SnDSIKzIWsoApkuzxXWq460TtXoC6QB9ns/00b6eUr9AhW/DFpAGexKypebWoDs bEbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Eoy+Pchg4s0QithVevz+IoMG+lYh7VneYRIDbLe6C1s=; b=NNsRaqZ57i/xucuXn7Nib4Azc+aKg80qLuAlJQ0N4jm5RxRbT6Z7JD3bGx8fT8FGbq q6EvfEWdbH1adwVGDsMQf9zjeCBOtoVZXVzUieK+IK0jetjfxBKa9MG4bN5r3ue4LWKL Yq5W3YHSwVZXewTFTeX0gA0Tq5xye2JzvvuLJI2WwcR+wFhgFgmoqiS+/Un406MBYS9i F/ITYv9pnyP5gpST1DV2FTL+4MSty4AEouE8flPbiNB7nHMK+wU1hS3dfQRwrLCGyMA3 d33fmovu/Ld/Ulu8elT0aIM2jIBHvjVUvytiyoFiqx91DtH99rilNNSz2oyCjJQvpHEf kySA== X-Gm-Message-State: AOAM530/XDZyjhjqzRa2Vigy0MVY8nC0ME1avj2rcf/w7PyqJFo61gGb CxKNKHa3rlqIjE7vzBxecfJsiwXng9M= X-Google-Smtp-Source: ABdhPJwE934eD6zKOi3fkeOuf9eSpR3/Gy5RT+rR/hMXdjN1ibo7klQLCDOl4lFOQ2K23gVgWM1y7g== X-Received: by 2002:a19:480c:: with SMTP id v12mr4779180lfa.60.1631297966164; Fri, 10 Sep 2021 11:19:26 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Cc: Oleksandr Tyshchenko , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross , Henry Wang , Bertrand Marquis , Wei Chen , Julien Grall , Stefano Stabellini Subject: [PATCH V2 3/3] libxl/arm: Add handling of extended regions for DomU Date: Fri, 10 Sep 2021 21:18:44 +0300 Message-Id: <1631297924-8658-4-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1631298004664100006 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko The extended region (safe range) is a region of guest physical address space which is unused and could be safely used to create grant/foreign mappings instead of wasting real RAM pages from the domain memory for establishing these mappings. The extended regions are chosen at the domain creation time and advertised to it via "reg" property under hypervisor node in the guest device-tree. As region 0 is reserved for grant table space (always present), the indexes for extended regions are 1...N. If extended regions could not be allocated for some reason, Xen doesn't fail and behaves as usual, so only inserts region 0. Please note the following limitations: - The extended region feature is only supported for 64-bit domain. - The ACPI case is not covered. *** The algorithm to choose extended regions for non-direct mapped DomU is simpler in comparison with the algorithm for direct mapped Dom0. As we have a lot of unused space above 4GB, provide single 1GB-aligned region from the second RAM bank taking into the account the maximum supported guest address space size and the amount of memory assigned to the guest. The maximum size of the region is 128GB. Suggested-by: Julien Grall Signed-off-by: Oleksandr Tyshchenko --- Changes since RFC: - update patch description - drop uneeded "extended-region" DT property - clear reg array in finalise_ext_region() and add a TODO --- tools/libs/light/libxl_arm.c | 89 ++++++++++++++++++++++++++++++++++++++++= +++- 1 file changed, 87 insertions(+), 2 deletions(-) diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index e3140a6..8c1d9d7 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -615,9 +615,12 @@ static int make_hypervisor_node(libxl__gc *gc, void *f= dt, "xen,xen"); if (res) return res; =20 - /* reg 0 is grant table space */ + /* + * reg 0 is a placeholder for grant table space, reg 1 is a placeholder + * for the extended region. + */ res =3D fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROO= T_SIZE_CELLS, - 1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE); + 2, 0, 0, 0, 0); if (res) return res; =20 /* @@ -1069,6 +1072,86 @@ static void finalise_one_node(libxl__gc *gc, void *f= dt, const char *uname, } } =20 +#define ALIGN_UP_TO_GB(x) (((x) + GB(1) - 1) & (~(GB(1) - 1))) + +#define EXT_REGION_SIZE GB(128) + +static void finalise_ext_region(libxl__gc *gc, struct xc_dom_image *dom) +{ + void *fdt =3D dom->devicetree_blob; + uint32_t regs[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2]; + be32 *cells =3D ®s[0]; + uint64_t region_size =3D 0, region_base, bank1end_align, bank1end_max; + uint32_t gpaddr_bits; + libxl_physinfo info; + int offset, rc; + + offset =3D fdt_path_offset(fdt, "/hypervisor"); + assert(offset > 0); + + if (strcmp(dom->guest_type, "xen-3.0-aarch64")) { + LOG(WARN, "The extended region is only supported for 64-bit guest"= ); + goto out; + } + + rc =3D libxl_get_physinfo(CTX, &info); + assert(!rc); + + gpaddr_bits =3D info.gpaddr_bits; + assert(gpaddr_bits >=3D 32 && gpaddr_bits <=3D 48); + + /* + * Try to allocate single 1GB-aligned extended region from the second = RAM + * bank (above 4GB) taking into the account the maximum supported guest + * address space size and the amount of memory assigned to the guest. + * The maximum size of the region is 128GB. + */ + bank1end_max =3D min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1= _SIZE); + bank1end_align =3D GUEST_RAM1_BASE + + ALIGN_UP_TO_GB((uint64_t)dom->rambank_size[1] << XC_PAGE_SHIFT); + + if (bank1end_max <=3D bank1end_align) { + LOG(WARN, "The extended region cannot be allocated, not enough spa= ce"); + goto out; + } + + if (bank1end_max - bank1end_align > EXT_REGION_SIZE) { + region_base =3D bank1end_max - EXT_REGION_SIZE; + region_size =3D EXT_REGION_SIZE; + } else { + region_base =3D bank1end_align; + region_size =3D bank1end_max - bank1end_align; + } + +out: + /* + * The first region for grant table space must be always present. + * If we managed to allocate the extended region then insert it as + * a second region. + * TODO If we failed to allocate the region, we end up inserting + * zero-sized region. This is because we don't know in advance when + * creating hypervisor node whether it would be possible to allocate + * a region, but we have to create a placeholder anyway. The Linux dri= ver + * is able to deal with by checking the region size. We cannot choose + * a region when creating hypervisor node because the guest memory lay= out + * is not know at that moment (and dom->rambank_size[1] is empty). + * We need to find a way not to expose invalid regions. + */ + memset(regs, 0, sizeof(regs)); + set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS, + GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE); + if (region_size > 0) { + LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n", + region_base, region_base + region_size); + + set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS, + region_base, region_size); + } + + rc =3D fdt_setprop_inplace(fdt, offset, "reg", regs, sizeof(regs)); + assert(!rc); +} + int libxl__arch_domain_finalise_hw_description(libxl__gc *gc, uint32_t domid, libxl_domain_config *d_conf= ig, @@ -1109,6 +1192,8 @@ int libxl__arch_domain_finalise_hw_description(libxl_= _gc *gc, =20 } =20 + finalise_ext_region(gc, dom); + for (i =3D 0; i < GUEST_RAM_BANKS; i++) { const uint64_t size =3D (uint64_t)dom->rambank_size[i] << XC_PAGE_= SHIFT; =20 --=20 2.7.4