From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518319; cv=none; d=zohomail.com; s=zohoarc; b=GcLyCqbZYqBiWzlK668NioOCvBulC5heFt4/hrmy5O2pJMK0NUqgWHRKrNNtmgwDJkM7LvwPfVdqpOWgFdK2YYoMIeOhPhu9oKkzAmkL0JwkJa2/m0PFbbvIP754ysE60eGUFJ0MhTkrsnLLj4ZnvwUTa7z7ABesS/akEmyuqhs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518319; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aWH5gx2xp10ySHfdsvtnv1tD2H4ADCtjpsnLlxSTmxQ=; b=B5gOmjUZQrNcr9nyx3laZXX5NBaB/3xhBd+qJ7rExo2+UyDHkSbCGgb48LD57vYenTisvrfWDThcZ/vBytXSqRYsfyGc6DePvSRIZ54DukvZDyxtxGuZj7Q31MaD0m+DOaOKyeo+XEFjZbj20fEwaiNTaZp3QE8NzGGQN2uK+po= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518319529365.6683728045357; Fri, 26 Aug 2022 05:51:59 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393753.632883 (Exim 4.92) (envelope-from ) id 1oRYo2-0001JS-Bt; Fri, 26 Aug 2022 12:51:26 +0000 Received: by outflank-mailman (output) from mailman id 393753.632883; Fri, 26 Aug 2022 12:51:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYo2-0001JL-7m; Fri, 26 Aug 2022 12:51:26 +0000 Received: by outflank-mailman (input) for mailman id 393753; Fri, 26 Aug 2022 12:51:25 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYo0-00013M-UW for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:25 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c9c21691-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:22 +0200 (CEST) Received: by mail-ej1-x632.google.com with SMTP id y3so3004762ejc.1 for ; Fri, 26 Aug 2022 05:51:22 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:21 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c9c21691-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=aWH5gx2xp10ySHfdsvtnv1tD2H4ADCtjpsnLlxSTmxQ=; b=n0Y4VIuhNdCrMLYsWkQhovINm/dmz/Ubo2HjYP3jqBflKwFX6BirFqgGmEaxrHfXi/ L5cFO+ZyED3AUKS1j5G8vRl6gk9LAt6xFeN4eckIxWoNhXcPnJYKaB8oXJOTskCxE1B1 ByrFI6ADZT9aWWGV2gSbbO42Yhiuj4uRXni5BH1CQBJl+DOjtpvOSPtm010y43UByOLy z+x2j3EGNFnoBrxYh41MK0V2LXiUxw2OFyVnwBLiMiRo7aM82CLOGdaLvdxzkmbbEizJ NG80NR4gxkLbOjVoyDExpYHuxf9rDod3BhwHOOiAue1L6qU/0QZ6NR5Uu5+ZHr/gZcex LzNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=aWH5gx2xp10ySHfdsvtnv1tD2H4ADCtjpsnLlxSTmxQ=; b=T8ZHbqGZuRNEkv/B9qHALcF5AWUhjVPYDk/EqT7tOIlm3GmxsUbPQDW2crgJ+PikWt J9Xu4exkAb4cpOPKxSCkUneVb8p2KqUG8CwXCi/rkBogEgfkDX+RuLB3P8YCwIP8jPrl oj/EVNifEvuXt14Po/tB0e0sEtFzIRv8iSEN2Pta69i7betV8wyQURV3kV+lPU/lk9Kp sjvuPl5RF8c/r01BLxeQF6Y7XVBZPgeHGj5HjqnJfS3mj7KHUeAhoDwDjWcD/gfeq4kK gqhVJOwKDtORSVruM9nmOzBH2ZykjUFr5EgPhD52+QHczXMwntlC/ptYnJgglV/RgogQ JREQ== X-Gm-Message-State: ACgBeo3VRaZrv6d3f3iCGZIHKekbTsGm8sgXsUvv/OUEo2xpA/x02ry/ te+rAncrxoa9tfF5UD7h0tx8d+hfY6uYDQ== X-Google-Smtp-Source: AA6agR5bs31ZxnvWb7Z3Ny3oGCQhWwPkMkD6rpoRzw3TIro4406Gijt4Q4cc52VEb7SLIejmx1/r3Q== X-Received: by 2002:a17:906:58ca:b0:73d:c7d5:bb39 with SMTP id e10-20020a17090658ca00b0073dc7d5bb39mr5432088ejs.305.1661518281780; Fri, 26 Aug 2022 05:51:21 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 01/12] xen/arm: add cache coloring initialization Date: Fri, 26 Aug 2022 14:51:00 +0200 Message-Id: <20220826125111.152261-2-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518321815100015 Content-Type: text/plain; charset="utf-8" This commit adds the cache coloring support initialization, Kconfig options, command line parameters and the initial documentation. The initialization consists of an auto probing of the cache layout necessary to retrieve the LLC way size which is used to compute the number of available colors. The Dom0 colors are then initialized with defau= lt colors (all available ones) if not provided from the command line, and they are checked for bad configuration. It also adds a debug-key to dump general cache coloring info. This includes LLC way size, total available colors and the mask used to extract colors from physical addresses. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- docs/misc/arm/cache-coloring.rst | 112 ++++++++++++++ docs/misc/xen-command-line.pandoc | 22 +++ xen/arch/arm/Kconfig | 16 ++ xen/arch/arm/Makefile | 1 + xen/arch/arm/coloring.c | 222 +++++++++++++++++++++++++++ xen/arch/arm/include/asm/coloring.h | 31 ++++ xen/arch/arm/include/asm/processor.h | 16 ++ xen/arch/arm/setup.c | 8 + 8 files changed, 428 insertions(+) create mode 100644 docs/misc/arm/cache-coloring.rst create mode 100644 xen/arch/arm/coloring.c create mode 100644 xen/arch/arm/include/asm/coloring.h diff --git a/docs/misc/arm/cache-coloring.rst b/docs/misc/arm/cache-colorin= g.rst new file mode 100644 index 0000000000..c7adcb0f1f --- /dev/null +++ b/docs/misc/arm/cache-coloring.rst @@ -0,0 +1,112 @@ +Xen cache coloring user guide +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D + +The cache coloring support in Xen allows to reserve Last Level Cache (LLC) +partition for Dom0, DomUs and Xen itself. Currently only ARM64 is supporte= d. + +In order to enable and use it, few steps are needed. + +- Enable cache coloring in Xen configuration file. + + CONFIG_CACHE_COLORING=3Dy +- If needed, change the maximum number of colors in Xen configuration file + (refer to menuconfig help for value meaning and when it should be change= d). + + CONFIG_MAX_CACHE_COLORS=3D +- Assign colors to Dom0 using the `Color selection format`_ (see + `Coloring parameters`_ for more documentation pointers). + +Background +********** + +Cache hierarchy of a modern multi-core CPU typically has first levels dedi= cated +to each core (hence using multiple cache units), while the last level is s= hared +among all of them. Such configuration implies that memory operations on one +core (e.g. running a DomU) are able to generate interference on another co= re +(e.g .hosting another DomU). Cache coloring allows eliminating this +mutual interference, and thus guaranteeing higher and more predictable +performances for memory accesses. +The key concept underlying cache coloring is a fragmentation of the memory +space into a set of sub-spaces called colors that are mapped to disjoint c= ache +partitions. Technically, the whole memory space is first divided into a nu= mber +of subsequent regions. Then each region is in turn divided into a number of +subsequent sub-colors. The generic i-th color is then obtained by all the +i-th sub-colors in each region. + +.. raw:: html + + 
+                            Region j            Region j+1
+                .....................   ............
+                .                     . .
+                .                       .
+            _ _ _______________ _ _____________________ _ _
+                |     |     |     |     |     |     |
+                | c_0 | c_1 |     | c_n | c_0 | c_1 |
+           _ _ _|_____|_____|_ _ _|_____|_____|_____|_ _ _
+                    :                       :
+                    :                       :...         ... .
+                    :                            color 0
+                    :...........................         ... .
+                                                :
+          . . ..................................:
+
+ +There are two pragmatic lesson to be learnt. + +1. If one wants to avoid cache interference between two domains, different + colors needs to be used for their memory. + +2. Color assignment must privilege contiguity in the partitioning. E.g., + assigning colors (0,1) to domain I and (2,3) to domain J is better th= an + assigning colors (0,2) to I and (1,3) to J. + +How to compute the number of colors +*********************************** + +To compute the number of available colors for a specific platform, the siz= e of +a LLC way and the page size used by Xen must be known. The first parameter= can +be found in the processor manual or can be also computed dividing the total +cache size by the number of its ways. The second parameter is the minimum = amount +of memory that can be mapped by the hypervisor, thus dividing the way size= by +the page size, the number of total cache partitions is found. So for examp= le, +an Arm Cortex-A53 with a 16-ways associative 1 MiB LLC, can isolate up to = 16 +colors when pages are 4 KiB in size. + +Cache layout is probed automatically by Xen itself, but a possibility to +manually set the way size it's left to the user to overcome failing situat= ions +or for debugging/testing purposes. See `Coloring parameters`_ section for = more +information on that. + +Colors selection format +*********************** + +Regardless of the memory pool that has to be colored (Xen, Dom0/DomUs), +the color selection can be expressed using the same syntax. In particular a +comma-separated list of colors or ranges of colors is used. +Ranges are hyphen-separated intervals (such as `0-4`) and are inclusive on= both +sides. + +Note that: + - no spaces are allowed between values. + - no overlapping ranges or duplicated colors are allowed. + - values must be written in ascending order. + +Examples: + ++---------------------+-----------------------------------+ +|**Configuration** |**Actual selection** | ++---------------------+-----------------------------------+ +| 1-2,5-8 | [1, 2, 5, 6, 7, 8] | ++---------------------+-----------------------------------+ +| 4-8,10,11,12 | [4, 5, 6, 7, 8, 10, 11, 12] | ++---------------------+-----------------------------------+ +| 0 | [0] | ++---------------------+-----------------------------------+ + +Coloring parameters +******************* + +LLC way size (as previously discussed) and Dom0 colors can be set using the +appropriate command line parameters. See the relevant documentation in +"docs/misc/xen-command-line.pandoc". \ No newline at end of file diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index 9a79385a37..910ebeb2eb 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -889,6 +889,14 @@ Controls for the dom0 IOMMU setup. =20 Incorrect use of this option may result in a malfunctioning system. =20 +### dom0-colors (arm64) +> `=3D List of [ | - ]` + +> Default: `All available colors` + +Specify dom0 color configuration. If the parameter is not set, all availab= le +colors are chosen and the user is warned on Xen's serial console. + ### dom0_ioports_disable (x86) > `=3D List of -` =20 @@ -1631,6 +1639,20 @@ This option is intended for debugging purposes only.= Enable MSR_DEBUGCTL.LBR in hypervisor context to be able to dump the Last Interrupt/Exception To/F= rom record with other registers. =20 +### llc-way-size (arm64) +> `=3D ` + +> Default: `Obtained from the hardware` + +Specify the way size of the Last Level Cache. This parameter is only usefu= l with +cache coloring support enabled. It is an optional, expert-only parameter a= nd it +is used to calculate the number of available colors on the platform. It ca= n be +obtained by dividing the total LLC size by the number of its associative w= ays. +By default, the value is automatically computed by probing the hardware, b= ut in +case of specific needs, it can be manually set. Those include failing prob= ing +and debugging/testing purposes so that it's possibile to emulate platforms= with +different number of supported colors. + ### loglvl > `=3D [/]` where level is `none | error | warn= ing | info | debug | all` =20 diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 33e004d702..8acff9682c 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -131,6 +131,22 @@ config ARM64_BTI Branch Target Identification support. This feature is not supported in Xen. =20 +config CACHE_COLORING + bool "Last Level Cache (LLC) coloring" if EXPERT + default n + depends on ARM_64 + +config MAX_CACHE_COLORS + int "Maximum number of cache colors" + default 128 + range 0 65536 + depends on CACHE_COLORING + help + This config value is an upper bound for the actual number of cache colo= rs + supported by the architecture. Xen preallocates this amount of cache + colors at boot. Note that if, at any time, a color configuration with m= ore + colors than the maximum will be employed an error will be produced. + config TEE bool "Enable TEE mediators support (UNSUPPORTED)" if UNSUPPORTED default n diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 4d076b278b..12940ba761 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -67,6 +67,7 @@ obj-$(CONFIG_SBSA_VUART_CONSOLE) +=3D vpl011.o obj-y +=3D vsmc.o obj-y +=3D vpsci.o obj-y +=3D vuart.o +obj-$(CONFIG_CACHE_COLORING) +=3D coloring.o =20 extra-y +=3D xen.lds =20 diff --git a/xen/arch/arm/coloring.c b/xen/arch/arm/coloring.c new file mode 100644 index 0000000000..c010ebc01b --- /dev/null +++ b/xen/arch/arm/coloring.c @@ -0,0 +1,222 @@ +/* + * xen/arch/arm/coloring.c + * + * Coloring support for ARM + * + * Copyright (C) 2019 Xilinx Inc. + * + * Authors: + * Luca Miccio + * Carlo Nonato + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +#include +#include +#include +#include + +#include +#include +#include + +/* Size of a LLC way */ +static unsigned int llc_way_size; +/* Number of colors available in the LLC */ +static unsigned int max_colors; +/* Mask to retrieve coloring relevant bits */ +static uint64_t addr_col_mask; + +#define addr_to_color(addr) (((addr) & addr_col_mask) >> PAGE_SHIFT) +#define addr_set_color(addr, color) (((addr) & ~addr_col_mask) \ + | ((color) << PAGE_SHIFT)) + +static unsigned int dom0_colors[CONFIG_MAX_CACHE_COLORS]; +static unsigned int dom0_num_colors; + +/* + * Parse the coloring configuration given in the buf string, following the + * syntax below. + * + * COLOR_CONFIGURATION ::=3D COLOR | RANGE,...,COLOR | RANGE + * RANGE ::=3D COLOR-COLOR + * + * Example: "0,2-6,15-16" represents the set of colors: 0,2,3,4,5,6,15,16. + */ +static int parse_color_config(const char *buf, unsigned int *colors, + unsigned int *num_colors) +{ + const char *s =3D buf; + + if ( !colors || !num_colors ) + return -EINVAL; + + while ( *s !=3D '\0' ) + { + if ( *s !=3D ',' ) + { + unsigned int color, start, end; + + start =3D simple_strtoul(s, &s, 0); + + if ( *s =3D=3D '-' ) /* Range */ + { + s++; + end =3D simple_strtoul(s, &s, 0); + } + else /* Single value */ + end =3D start; + + if ( start > end || + *num_colors + end - start >=3D CONFIG_MAX_CACHE_COLORS ) + return -EINVAL; + for ( color =3D start; color <=3D end; color++ ) + colors[(*num_colors)++] =3D color; + } + else + s++; + } + + return *s ? -EINVAL : 0; +} + +size_param("llc-way-size", llc_way_size); + +static int __init parse_dom0_colors(const char *s) +{ + return parse_color_config(s, dom0_colors, &dom0_num_colors); +} +custom_param("dom0-colors", parse_dom0_colors); + +/* Return the LLC way size by probing the hardware */ +static unsigned int get_llc_way_size(void) +{ + register_t ccsidr_el1; + register_t clidr_el1 =3D READ_SYSREG(CLIDR_EL1); + register_t csselr_el1 =3D READ_SYSREG(CSSELR_EL1); + register_t id_aa64mmfr2_el1 =3D READ_SYSREG(ID_AA64MMFR2_EL1); + uint32_t ccsidr_numsets_shift =3D CCSIDR_NUMSETS_SHIFT; + uint32_t ccsidr_numsets_mask =3D CCSIDR_NUMSETS_MASK; + unsigned int n, line_size, num_sets; + + for ( n =3D CLIDR_CTYPEn_LEVELS; + n !=3D 0 && !((clidr_el1 >> CLIDR_CTYPEn_SHIFT(n)) & CLIDR_CTYPE= n_MASK); + n-- ); + + if ( n =3D=3D 0 ) + return 0; + + WRITE_SYSREG(((n - 1) & CCSELR_LEVEL_MASK) << CCSELR_LEVEL_SHIFT, + CSSELR_EL1); + isb(); + + ccsidr_el1 =3D READ_SYSREG(CCSIDR_EL1); + + /* Arm ARM: (Log2(Number of bytes in cache line)) - 4 */ + line_size =3D 1 << ((ccsidr_el1 & CCSIDR_LINESIZE_MASK) + 4); + + /* If FEAT_CCIDX is enabled, CCSIDR_EL1 has a different bit layout */ + if ( (id_aa64mmfr2_el1 >> ID_AA64MMFR2_CCIDX_SHIFT) & 0x7 ) + { + ccsidr_numsets_shift =3D CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX; + ccsidr_numsets_mask =3D CCSIDR_NUMSETS_MASK_FEAT_CCIDX; + } + /* Arm ARM: (Number of sets in cache) - 1 */ + num_sets =3D ((ccsidr_el1 >> ccsidr_numsets_shift) & ccsidr_numsets_ma= sk) + 1; + + printk(XENLOG_INFO "LLC found: L%u (line size: %u bytes, sets num: %u)= \n", + n, line_size, num_sets); + + /* Restore value in CSSELR_EL1 */ + WRITE_SYSREG(csselr_el1, CSSELR_EL1); + isb(); + + return line_size * num_sets; +} + +static bool check_colors(unsigned int *colors, unsigned int num_colors) +{ + int i; + + if ( num_colors > max_colors ) + return false; + + for ( i =3D 0; i < num_colors; i++ ) + if ( colors[i] >=3D max_colors ) + return false; + + return true; +} + +static unsigned int set_default_domain_colors(unsigned int *colors) +{ + unsigned int i; + + if ( !colors ) + return 0; + + for ( i =3D 0; i < max_colors; i++ ) + colors[i] =3D i; + return max_colors; +} + +static void dump_coloring_info(unsigned char key) +{ + printk("'%c' pressed -> dumping coloring general info\n", key); + printk("LLC way size: %u KiB\n", llc_way_size >> 10); + printk("Number of LLC colors supported: %u\n", max_colors); + printk("Address color mask: 0x%lx\n", addr_col_mask); +} + +bool __init coloring_init(void) +{ + if ( !llc_way_size && !(llc_way_size =3D get_llc_way_size()) ) + { + printk(XENLOG_ERR + "Probed LLC way size is 0 and no custom value provided\n"); + return false; + } + + max_colors =3D llc_way_size / PAGE_SIZE; + /* The maximum number of colors must be a power of 2 in order to corre= ctly + map colors to bits of an address. */ + ASSERT((max_colors & (max_colors - 1)) =3D=3D 0); + addr_col_mask =3D (max_colors - 1) << PAGE_SHIFT; + + if ( !dom0_num_colors ) + { + printk(XENLOG_WARNING + "Dom0 color config not found. Using default (all colors)\n"= ); + dom0_num_colors =3D set_default_domain_colors(dom0_colors); + } + + if ( !check_colors(dom0_colors, dom0_num_colors) ) + { + printk(XENLOG_ERR "Bad color config for Dom0\n"); + return false; + } + + register_keyhandler('K', dump_coloring_info, "dump coloring info", 1); + + return true; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/include/asm/coloring.h b/xen/arch/arm/include/asm= /coloring.h new file mode 100644 index 0000000000..dd7eff5f07 --- /dev/null +++ b/xen/arch/arm/include/asm/coloring.h @@ -0,0 +1,31 @@ +/* + * xen/arm/include/asm/coloring.h + * + * Coloring support for ARM + * + * Copyright (C) 2019 Xilinx Inc. + * + * Authors: + * Luca Miccio + * Carlo Nonato + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +#ifndef __ASM_ARM_COLORING_H__ +#define __ASM_ARM_COLORING_H__ + +#include + +bool __init coloring_init(void); + +#endif /* !__ASM_ARM_COLORING_H__ */ diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/as= m/processor.h index 1dd81d7d52..85ff0caf1e 100644 --- a/xen/arch/arm/include/asm/processor.h +++ b/xen/arch/arm/include/asm/processor.h @@ -18,6 +18,22 @@ #define CTR_IDC_SHIFT 28 #define CTR_DIC_SHIFT 29 =20 +/* CCSIDR Current Cache Size ID Register */ +#define CCSIDR_LINESIZE_MASK 0x7 +#define CCSIDR_NUMSETS_SHIFT 13 +#define CCSIDR_NUMSETS_MASK 0x3FFF +#define CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX 32 +#define CCSIDR_NUMSETS_MASK_FEAT_CCIDX 0xFFFFFF + +/* CCSELR Cache Size Selection Register */ +#define CCSELR_LEVEL_MASK 0x7 +#define CCSELR_LEVEL_SHIFT 1 + +/* CLIDR Cache Level ID Register */ +#define CLIDR_CTYPEn_SHIFT(n) (3 * (n - 1)) +#define CLIDR_CTYPEn_MASK 0x7 +#define CLIDR_CTYPEn_LEVELS 7 + #define ICACHE_POLICY_VPIPT 0 #define ICACHE_POLICY_AIVIVT 1 #define ICACHE_POLICY_VIPT 2 diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 500307edc0..c02f21c0e6 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -53,6 +53,9 @@ #include #include #include +#ifdef CONFIG_CACHE_COLORING +#include +#endif =20 struct bootinfo __initdata bootinfo; =20 @@ -930,6 +933,11 @@ void __init start_xen(unsigned long boot_phys_offset, printk("Command line: %s\n", cmdline); cmdline_parse(cmdline); =20 +#ifdef CONFIG_CACHE_COLORING + if ( !coloring_init() ) + panic("Xen Coloring support: setup failed\n"); +#endif + setup_mm(); =20 /* Parse the ACPI tables for possible boot-time configuration */ --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518317; cv=none; d=zohomail.com; s=zohoarc; b=dcpbRldregd6s1U/g647EOWh0TjtlJ8hZ99ftw9TeOhWkDyOu7Sf8wEaanxUJYvP1YD5Ky3tSg0pHUTNqiRVuMwODO5jvTbGa8B7m0ZCA+DVYghBx6G2K42zPHqMDIzORGvOd1D0HGSAuMlVUY8x4FWoolQNjgcMbUydYqfpm00= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518317; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9CmOaXef3+3nQfZH6p8WWc7HRYLw9F9JZ6/dhzECsOk=; b=bnekIvWyIaWlB1dDPVh5nnVXv+BiCMybqY+70Qo2LNbGgh0x+JnyQMQHETyleJ/+08cK9hSYAAS0WQrgK5hnfGwEktVw5D0oFAChLSUqbN77l6QDLTOZL6UQuSkEQU2XnJDVczAOUrHKdi5g2/67InYnAeKMRANoshgTyHOhHkU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518317098155.65338734159593; Fri, 26 Aug 2022 05:51:57 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393754.632894 (Exim 4.92) (envelope-from ) id 1oRYo9-0001dx-Mn; Fri, 26 Aug 2022 12:51:33 +0000 Received: by outflank-mailman (output) from mailman id 393754.632894; Fri, 26 Aug 2022 12:51:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYo9-0001dm-Jz; Fri, 26 Aug 2022 12:51:33 +0000 Received: by outflank-mailman (input) for mailman id 393754; Fri, 26 Aug 2022 12:51:32 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYo8-00013M-1N for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:32 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id cecce6e8-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:31 +0200 (CEST) Received: by mail-ej1-x632.google.com with SMTP id y3so3005428ejc.1 for ; Fri, 26 Aug 2022 05:51:31 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:30 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cecce6e8-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=9CmOaXef3+3nQfZH6p8WWc7HRYLw9F9JZ6/dhzECsOk=; b=KFuWujFYJ5dgTKPPCnzmD+VtTxeGNekYvjc0hraXCXgwwHPGOR0lPxIZ/bKLjBSqnz YlY6tluzv9kLzG+R5MO1ztwcPjiRsQ0oXTKPAv9TBIrRLOBSExgS8Ne8oBoSWRHuRuuR tZCrdZQpmd/U8mgZsUCI/9OgbLVDHJPxDu/YKyf12YKy3hgkqAw/7WNXcVaAYuMMDde1 qBflC8Tk4J1+nRVd6kiWTVH3BRP2kFdYKJLOZXr+HrwSLdKhhw0BhFE4t4zhxlY4GUVi snpo5KGQOu3m3Qa6U+e2sXcc5f6IeHifAKlZLQbaxtM2/qOSGOOHEbYsY+F61O7CnvTv 7Hsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=9CmOaXef3+3nQfZH6p8WWc7HRYLw9F9JZ6/dhzECsOk=; b=wJEJbAtCS4r5Oj07y/0xsLpWX2Hwz2PB5fRWpDYW2qYs+fcLtv0jaW65jYwk6U4ZS5 xptiCDYMBe7+2MXUuO48DE3jn0NOF3Mfu85tZLcX+lMYZWiRFbx1DSzu8LJTe/i0hA8I We1ZXp0GqeZfoJ2dPlXxSQ0tiN8cAlV9CgSb7yWX+g3ISONOlSfbQra7fwEo6AdkykZY Qr6Oxd6a4Ttohn6/cjR79dhK8rtAPNMs5smXyovDo3O1S7eQB1+vgtEjD0E+mmO6xqY9 1A9yKwuVKm9dEUaENpV8VmE40qREHrgAScPskA1Sd0k/2So5F8wYx86zphrSgme4YkFI Hukg== X-Gm-Message-State: ACgBeo1vfFEcXiTp88s9VocQg+ra/QHjdhEoGPkkQ/WmxOxjfTlBZx5Y Mmf8LOqgipHeToGYxWD4Non0MgNr28Gh/w== X-Google-Smtp-Source: AA6agR4edpTZbwf/2AWI2wKj6vWtkwXyIccMd+6PdS55TRkhiMGoNOkpBV7wA4W9G9GoWIzDM5IIRA== X-Received: by 2002:a17:907:7619:b0:73e:28d8:eca9 with SMTP id jx25-20020a170907761900b0073e28d8eca9mr2086999ejc.490.1661518290452; Fri, 26 Aug 2022 05:51:30 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 02/12] xen/arm: add cache coloring initialization for domains Date: Fri, 26 Aug 2022 14:51:01 +0200 Message-Id: <20220826125111.152261-3-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518318389100005 Content-Type: text/plain; charset="utf-8" This commit adds array pointers to domains as well as to the hypercall and configuration structure employed in domain creation. The latter is used both by the toolstack and by Xen itself to pass configuration data to the domain creation function, so the XEN_GUEST_HANDLE macro must be adopted to = be able to access guest memory in the first case. This implies special care for the copy of the configuration data into the domain data, meaning that a discrimination variable for the two possible code paths (coming from Xen or from the toolstack) is needed. The initialization and free functions for colored domains are also added. The former is responsible for allocating and populating the color array of the domain and it also checks for configuration issues. One of those issues is enabling both coloring and directmap for the domain because they contradicts one another. Since that, Dom0 must not be created with the directmap flag. The latter instead frees allocated memory. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- docs/misc/arm/cache-coloring.rst | 7 ++-- xen/arch/arm/coloring.c | 56 +++++++++++++++++++++++++++++ xen/arch/arm/domain.c | 11 ++++++ xen/arch/arm/domain_build.c | 13 +++++-- xen/arch/arm/include/asm/coloring.h | 7 ++++ xen/arch/arm/include/asm/domain.h | 4 +++ xen/include/public/arch-arm.h | 8 +++++ 7 files changed, 102 insertions(+), 4 deletions(-) diff --git a/docs/misc/arm/cache-coloring.rst b/docs/misc/arm/cache-colorin= g.rst index c7adcb0f1f..345d97cb56 100644 --- a/docs/misc/arm/cache-coloring.rst +++ b/docs/misc/arm/cache-coloring.rst @@ -13,7 +13,7 @@ In order to enable and use it, few steps are needed. (refer to menuconfig help for value meaning and when it should be change= d). =20 CONFIG_MAX_CACHE_COLORS=3D -- Assign colors to Dom0 using the `Color selection format`_ (see +- Assign colors to domains using the `Color selection format`_ (see `Coloring parameters`_ for more documentation pointers). =20 Background @@ -109,4 +109,7 @@ Coloring parameters =20 LLC way size (as previously discussed) and Dom0 colors can be set using the appropriate command line parameters. See the relevant documentation in -"docs/misc/xen-command-line.pandoc". \ No newline at end of file +"docs/misc/xen-command-line.pandoc". + +Note that if no color configuration is provided for domains, they fallback= to +the default one, which corresponds simply to all available colors. \ No newline at end of file diff --git a/xen/arch/arm/coloring.c b/xen/arch/arm/coloring.c index c010ebc01b..2b37cda067 100644 --- a/xen/arch/arm/coloring.c +++ b/xen/arch/arm/coloring.c @@ -22,6 +22,7 @@ * along with this program. If not, see . */ #include +#include #include #include #include @@ -211,6 +212,61 @@ bool __init coloring_init(void) return true; } =20 +int domain_coloring_init(struct domain *d, + const struct xen_arch_domainconfig *config) +{ + if ( is_domain_direct_mapped(d) ) + { + printk(XENLOG_ERR + "Can't enable coloring and directmap at the same time for %= pd\n", + d); + return -EINVAL; + } + + if ( is_hardware_domain(d) ) + { + d->arch.colors =3D dom0_colors; + d->arch.num_colors =3D dom0_num_colors; + } + else if ( config->num_colors =3D=3D 0 ) + { + printk(XENLOG_WARNING + "Color config not found for %pd. Using default\n", d); + d->arch.colors =3D xzalloc_array(unsigned int, max_colors); + d->arch.num_colors =3D set_default_domain_colors(d->arch.colors); + } + else + { + d->arch.colors =3D xzalloc_array(unsigned int, config->num_colors); + d->arch.num_colors =3D config->num_colors; + if ( config->from_guest ) + copy_from_guest(d->arch.colors, config->colors, config->num_co= lors); + else + memcpy(d->arch.colors, config->colors.p, + sizeof(unsigned int) * config->num_colors); + } + + if ( !d->arch.colors ) + { + printk(XENLOG_ERR "Colors allocation failed for %pd\n", d); + return -ENOMEM; + } + + if ( !check_colors(d->arch.colors, d->arch.num_colors) ) + { + printk(XENLOG_ERR "Bad color config for %pd\n", d); + return -EINVAL; + } + + return 0; +} + +void domain_coloring_free(struct domain *d) +{ + if ( !is_hardware_domain(d) ) + xfree(d->arch.colors); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index 2d6253181a..c6fa8adc99 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -23,6 +23,9 @@ #include =20 #include +#ifdef CONFIG_CACHE_COLORING +#include +#endif #include #include #include @@ -712,6 +715,11 @@ int arch_domain_create(struct domain *d, ioreq_domain_init(d); #endif =20 +#ifdef CONFIG_CACHE_COLORING + if ( (rc =3D domain_coloring_init(d, &config->arch)) ) + goto fail; +#endif + /* p2m_init relies on some value initialized by the IOMMU subsystem */ if ( (rc =3D iommu_domain_init(d, config->iommu_opts)) !=3D 0 ) goto fail; @@ -807,6 +815,9 @@ void arch_domain_destroy(struct domain *d) get_order_from_bytes(d->arch.efi_acpi_len)); #endif domain_io_free(d); +#ifdef CONFIG_CACHE_COLORING + domain_coloring_free(d); +#endif } =20 void arch_domain_shutdown(struct domain *d) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 3fd1186b53..4d4cb692fc 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -33,6 +33,12 @@ #include #include =20 +#ifdef CONFIG_CACHE_COLORING +#define XEN_DOM0_CREATE_FLAGS CDF_privileged +#else +#define XEN_DOM0_CREATE_FLAGS CDF_privileged | CDF_directmap +#endif + static unsigned int __initdata opt_dom0_max_vcpus; integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); =20 @@ -3399,7 +3405,10 @@ static int __init construct_dom0(struct domain *d) /* type must be set before allocate_memory */ d->arch.type =3D kinfo.type; #endif - allocate_memory_11(d, &kinfo); + if ( IS_ENABLED(CONFIG_CACHE_COLORING) ) + allocate_memory(d, &kinfo); + else + allocate_memory_11(d, &kinfo); find_gnttab_region(d, &kinfo); =20 /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */ @@ -3455,7 +3464,7 @@ void __init create_dom0(void) if ( iommu_enabled ) dom0_cfg.flags |=3D XEN_DOMCTL_CDF_iommu; =20 - dom0 =3D domain_create(0, &dom0_cfg, CDF_privileged | CDF_directmap); + dom0 =3D domain_create(0, &dom0_cfg, XEN_DOM0_CREATE_FLAGS); if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) =3D=3D NULL) ) panic("Error creating domain 0\n"); =20 diff --git a/xen/arch/arm/include/asm/coloring.h b/xen/arch/arm/include/asm= /coloring.h index dd7eff5f07..60c8b1f079 100644 --- a/xen/arch/arm/include/asm/coloring.h +++ b/xen/arch/arm/include/asm/coloring.h @@ -25,7 +25,14 @@ #define __ASM_ARM_COLORING_H__ =20 #include +#include + +#include =20 bool __init coloring_init(void); =20 +int domain_coloring_init(struct domain *d, + const struct xen_arch_domainconfig *config); +void domain_coloring_free(struct domain *d); + #endif /* !__ASM_ARM_COLORING_H__ */ diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/d= omain.h index 26a8348eed..291f7c375d 100644 --- a/xen/arch/arm/include/asm/domain.h +++ b/xen/arch/arm/include/asm/domain.h @@ -58,6 +58,10 @@ struct arch_domain #ifdef CONFIG_ARM_64 enum domain_type type; #endif +#ifdef CONFIG_CACHE_COLORING + unsigned int *colors; + unsigned int num_colors; +#endif =20 /* Virtual MMU */ struct p2m_domain p2m; diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h index c8b6058d3a..adf843a7a1 100644 --- a/xen/include/public/arch-arm.h +++ b/xen/include/public/arch-arm.h @@ -314,6 +314,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t); #define XEN_DOMCTL_CONFIG_TEE_NONE 0 #define XEN_DOMCTL_CONFIG_TEE_OPTEE 1 =20 +__DEFINE_XEN_GUEST_HANDLE(color_t, unsigned int); + struct xen_arch_domainconfig { /* IN/OUT */ uint8_t gic_version; @@ -335,6 +337,12 @@ struct xen_arch_domainconfig { * */ uint32_t clock_frequency; + /* IN */ + uint8_t from_guest; + /* IN */ + uint16_t num_colors; + /* IN */ + XEN_GUEST_HANDLE(color_t) colors; }; #endif /* __XEN__ || __XEN_TOOLS__ */ =20 --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518318; cv=none; d=zohomail.com; s=zohoarc; b=QdRweiFAVpD0E15V7PbpYC7ImV2CfcAqdzSRs3a37z2JLXciL6TuFNqauPSYuRoGzo4SADzuK9RvnxZrVgGAfJafBv9WLhSnV4DsOpCad3z4M30QZwxyRrZXSIFzytGCpapEfjv4ZR9Ywn6ig2StdyYMusVDFN4XaNs0yKi9DLg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518318; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KUUG3oBu2sZvqX2D9rJkuOuQmnOvyHH8GrWoUzfp3Co=; b=TaGGsJB7mSjDdxoEvHnOmZ6MytQQ9NxI5AtrmoKcK7FKrJau8T4LPDrojPa3Kf6ksonYquZAubNhkHKVqjwMXWppbROH/pFMhD6KvUC93H04lJ2XXYY2BwwOm8ewvOhV8j5DDOzuLf3GtNxcXxW4zfBP81Q0sRmTmArgyZqxJwg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518318473532.1309172659227; Fri, 26 Aug 2022 05:51:58 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393755.632905 (Exim 4.92) (envelope-from ) id 1oRYoB-0001v6-U8; Fri, 26 Aug 2022 12:51:35 +0000 Received: by outflank-mailman (output) from mailman id 393755.632905; Fri, 26 Aug 2022 12:51:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoB-0001uz-R9; Fri, 26 Aug 2022 12:51:35 +0000 Received: by outflank-mailman (input) for mailman id 393755; Fri, 26 Aug 2022 12:51:34 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoA-0001dl-AC for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:34 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id cfbe9594-253d-11ed-bd2e-47488cf2e6aa; Fri, 26 Aug 2022 14:51:33 +0200 (CEST) Received: by mail-ej1-x632.google.com with SMTP id kk26so2943049ejc.11 for ; Fri, 26 Aug 2022 05:51:32 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:31 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cfbe9594-253d-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=KUUG3oBu2sZvqX2D9rJkuOuQmnOvyHH8GrWoUzfp3Co=; b=Oiy1yctzwOMWIvwKtGoMgkuywg12OKtQRBp9V+oUGbkP/OXN7ECgN0PcUiosnp39xk /hgkpd+VH3OPLhRyzwyyQfy6VhWw/bykqd+5GJ8Gc/YB34VHcUGPRNT5goTnsyioXAw4 rHxUti3ck5WigE+xhLWiEYYNBrlM/GwWrT5/z6XFBLy6rmg7Og85MgPoJ4nfQbv3x8XG GnafJ3xgf4DsFIPTWukH+c5G5odiQekl5fN4SST9uA2cUY/HgfspAG142jEKFkmpZ5rl w2DFEV0Qdo9omlVTeCoLSWbxCAX3IhP1LgyyXkGPNj0gXSYEKBegK01YZYXf7yLNbvpb rdog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=KUUG3oBu2sZvqX2D9rJkuOuQmnOvyHH8GrWoUzfp3Co=; b=cB4tl5RAoEHWYBGcDbXWbzw1SVqEyxhHpA03CCbgu3xqYNKd06Pq7e9tSdGhFBIJiQ 4MhrpZ5L35jZY/qiftctXXkQLaiJSthd1IPWtrNJa2eyG/W4VlGULH08iNEA3PTMIT3f 3zzCyfNq0bOZfv2hk+W7FYeIm4bbGWMDd8Iaia/rEBzK7oQLcaW7A5e0awiBjiVkWY9I CAmRFA/WXq+Jklxwa4CcP0WjZQl50nUrmJsiRP4UOkGXiF7oRG+Wf3B0L6YNFsFkl1xj Bt+GwLXg38txx7V4O5wPhjKh0gJgZZgOotOtAdiuFy5JtiOg1pdkMzLcECnV0cceIDfi to5g== X-Gm-Message-State: ACgBeo0zbLn6L6R5XVLCATRBPe+EumM6izMbKDh35MlqE+1i09qYZl2j Li5nXs+p97ZgIRuQ5DeOo+SPKFrkn7VS/g== X-Google-Smtp-Source: AA6agR4y4ZyBsKL9yN9HoaM+dJZ6Zq+gSYvqaQ/rT0daG/uYAEbT6gfIkKZ1U8JUKMq3hulYOlOALw== X-Received: by 2002:a17:907:7e8c:b0:73d:85ec:46c with SMTP id qb12-20020a1709077e8c00b0073d85ec046cmr5080773ejc.257.1661518292192; Fri, 26 Aug 2022 05:51:32 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 03/12] xen/arm: dump cache colors in domain info debug-key Date: Fri, 26 Aug 2022 14:51:02 +0200 Message-Id: <20220826125111.152261-4-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518319858100011 Content-Type: text/plain; charset="utf-8" This commit adds cache colors to the information dumped with the domain info debug-key. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- xen/arch/arm/coloring.c | 16 ++++++++++++++++ xen/arch/arm/domain.c | 3 +++ xen/arch/arm/include/asm/coloring.h | 1 + 3 files changed, 20 insertions(+) diff --git a/xen/arch/arm/coloring.c b/xen/arch/arm/coloring.c index 2b37cda067..40a07617cc 100644 --- a/xen/arch/arm/coloring.c +++ b/xen/arch/arm/coloring.c @@ -171,6 +171,16 @@ static unsigned int set_default_domain_colors(unsigned= int *colors) return max_colors; } =20 +static void print_colors(unsigned int *colors, unsigned int num_colors) +{ + unsigned int i; + + printk("[ "); + for ( i =3D 0; i < num_colors; i++ ) + printk("%u ", colors[i]); + printk("]\n"); +} + static void dump_coloring_info(unsigned char key) { printk("'%c' pressed -> dumping coloring general info\n", key); @@ -267,6 +277,12 @@ void domain_coloring_free(struct domain *d) xfree(d->arch.colors); } =20 +void domain_dump_coloring_info(struct domain *d) +{ + printk("Domain %pd has %u colors: ", d, d->arch.num_colors); + print_colors(d->arch.colors, d->arch.num_colors); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index c6fa8adc99..bca418e374 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -1087,6 +1087,9 @@ int domain_relinquish_resources(struct domain *d) void arch_dump_domain_info(struct domain *d) { p2m_dump_info(d); +#ifdef CONFIG_CACHE_COLORING + domain_dump_coloring_info(d); +#endif } =20 =20 diff --git a/xen/arch/arm/include/asm/coloring.h b/xen/arch/arm/include/asm= /coloring.h index 60c8b1f079..967031ae04 100644 --- a/xen/arch/arm/include/asm/coloring.h +++ b/xen/arch/arm/include/asm/coloring.h @@ -34,5 +34,6 @@ bool __init coloring_init(void); int domain_coloring_init(struct domain *d, const struct xen_arch_domainconfig *config); void domain_coloring_free(struct domain *d); +void domain_dump_coloring_info(struct domain *d); =20 #endif /* !__ASM_ARM_COLORING_H__ */ --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518316; cv=none; d=zohomail.com; s=zohoarc; b=bRmRA9c0FtYyj2IC+9Th5ml4mk4u/XhIHXXTygSvPf1+Kg+oozTrT8tnsuLBORLEsJJ3LHQZ4dH9509GmtogxMqzGkH5npgsT1QRhyBeZAl0OUBP2GFSlXLnFCkauHzkEWgtloqikBQPeaFIKZLe6SRIT5A952mpm5aS/nLRCHM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518316; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bZOFiRTK3AMcZ6w41ZvyaJ0FkDlksPHygOjR3JNEJS4=; b=jKn7wsH8Z4nu/VT5HGP4H2aQa0/2Op++qNBk7PdBhlgnuwagrqFv/vOXJKOdIX3HMCfVQKjTDzE6P9Q05XS1SKqQnd086tW3phStAdGqzGWUSd9p4/CoNMMDaXGMY7KaFToZTFetnz45wJ8g45uYV1sRBkZ1Cmbt7RL2fcInaIY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518316322116.65670762822049; Fri, 26 Aug 2022 05:51:56 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393756.632911 (Exim 4.92) (envelope-from ) id 1oRYoC-0001z0-AF; Fri, 26 Aug 2022 12:51:36 +0000 Received: by outflank-mailman (output) from mailman id 393756.632911; Fri, 26 Aug 2022 12:51:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoC-0001xz-2p; Fri, 26 Aug 2022 12:51:36 +0000 Received: by outflank-mailman (input) for mailman id 393756; Fri, 26 Aug 2022 12:51:35 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoB-00013M-4u for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:35 +0000 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [2a00:1450:4864:20::633]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d09a1530-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:34 +0200 (CEST) Received: by mail-ej1-x633.google.com with SMTP id w19so2977028ejc.7 for ; Fri, 26 Aug 2022 05:51:34 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:33 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d09a1530-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=bZOFiRTK3AMcZ6w41ZvyaJ0FkDlksPHygOjR3JNEJS4=; b=Jrd6pcEvA4VcGAsJwfRsDl5iFff89wJMQ4W9lsCt7f6IMzpMhm5mccRbrgvXgs9wwu fjOdOZ/2zz1SHBdQibqKtOKu1wnBL4K0G3zvs06pjb8CMBQrlYcmQX8S8EXM3r9JyNWH w5z3VO1PqarkUxX4Kt+vcIJ6pmz2QKFgV/jtDegaKIcmx9ubSsg21vnRcMUoZlkIMco+ RjBcoPn+zUq5Nyw63jXCgRVNpqz7uvHmHDsQ8AT0gbBP1pYDeSmmh4Ov3cKDcR8hgey2 vNfurPBBnIHlF4dzmp4IXZwHrJFb/h2NEcKRI7itJPxrW7wwsFfH86rZXudrws3j4YL8 iELQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=bZOFiRTK3AMcZ6w41ZvyaJ0FkDlksPHygOjR3JNEJS4=; b=foH09KNW9IAhjyd7Lajeh7MwIa6HycojNzpt0WdiMgSGFA1VDfJH9o0YXVmv6fV5I4 oYNAMN0hGtFB61qRHU/aVxuJIBBEZ6xqyof5huSz+AvQa7sFOITcOx6m1NmCixI+TDJc 7xeUOViDbo/uVgLlzdxo7isX3vHSXAa84pG0w0OmM7kTsFbjCjS7vDvmj/DidSkY1coc BA007DVh49JyOIMCbwjigbkxFHdg0/1f3bynWvlLr+7i//2WrXuAUbCp1OxK1pOctq6+ n2mzkB/QVEovGuNXFcNtjyYVlGYqPIsD/PMgctZUVVHbRXr89pmjSIsv31xPeoDqAbOc RMFw== X-Gm-Message-State: ACgBeo3XwSZ/L6EWcjOD0mNkUR1reke7mYL/D6VIWL5jmTuWYDPUBLdc oXh/DwNyZD928Lg6CeQhQ/6FxVw+ULQvnA== X-Google-Smtp-Source: AA6agR4PY0LYOzsFP/Yrv5FTmaf90sA8wSXjYW4UD83WIUhZVW78+R8x0XRaGcnoZtNujSu7OiYwIg== X-Received: by 2002:a17:907:a046:b0:730:9c7a:eab3 with SMTP id gz6-20020a170907a04600b007309c7aeab3mr5648959ejc.285.1661518293597; Fri, 26 Aug 2022 05:51:33 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 04/12] tools/xl: add support for cache coloring configuration Date: Fri, 26 Aug 2022 14:51:03 +0200 Message-Id: <20220826125111.152261-5-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518318201100001 Content-Type: text/plain; charset="utf-8" Add a new "colors" parameter that defines the color assignment for a domain. The user can specify one or more color ranges using the same syntax used everywhere else for color config described in the documentation. The parameter is defined as a list of strings that represent the color ranges. Also documentation is added. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- docs/man/xl.cfg.5.pod.in | 10 ++++++ tools/libs/light/libxl_create.c | 12 ++++++++ tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_parse.c | 52 ++++++++++++++++++++++++++++++-- 4 files changed, 73 insertions(+), 2 deletions(-) diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in index b2901e04cf..5f53cec8bf 100644 --- a/docs/man/xl.cfg.5.pod.in +++ b/docs/man/xl.cfg.5.pod.in @@ -2880,6 +2880,16 @@ Currently, only the "sbsa_uart" model is supported f= or ARM. =20 =3Dback =20 +=3Dover 4 + +=3Ditem B + +Specify the LLC color configuration for the guest. B can be = either +a single color value or a hypen-separated closed interval of colors +(such as "0-4"). + +=3Dback + =3Dhead3 x86 =20 =3Dover 4 diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_creat= e.c index b9dd2deedf..94c511912c 100644 --- a/tools/libs/light/libxl_create.c +++ b/tools/libs/light/libxl_create.c @@ -615,6 +615,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_conf= ig *d_config, struct xs_permissions rwperm[1]; struct xs_permissions noperm[1]; xs_transaction_t t =3D 0; + DECLARE_HYPERCALL_BUFFER(unsigned int, colors); =20 /* convenience aliases */ libxl_domain_create_info *info =3D &d_config->c_info; @@ -676,6 +677,16 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_con= fig *d_config, goto out; } =20 + if (d_config->b_info.num_colors) { + size_t bytes =3D sizeof(unsigned int) * d_config->b_info.num_c= olors; + colors =3D xc_hypercall_buffer_alloc(ctx->xch, colors, bytes); + memcpy(colors, d_config->b_info.colors, bytes); + set_xen_guest_handle(create.arch.colors, colors); + create.arch.num_colors =3D d_config->b_info.num_colors; + create.arch.from_guest =3D 1; + LOG(DEBUG, "Setup %u domain colors", d_config->b_info.num_colo= rs); + } + for (;;) { uint32_t local_domid; bool recent; @@ -922,6 +933,7 @@ retry_transaction: rc =3D 0; out: if (t) xs_transaction_end(ctx->xsh, t, 1); + if (colors) xc_hypercall_buffer_free(ctx->xch, colors); return rc; } =20 diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_type= s.idl index d634f304cd..642173af1a 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -557,6 +557,7 @@ libxl_domain_build_info =3D Struct("domain_build_info",[ ("ioports", Array(libxl_ioport_range, "num_ioports")), ("irqs", Array(uint32, "num_irqs")), ("iomem", Array(libxl_iomem_range, "num_iomem")), + ("colors", Array(uint32, "num_colors")), ("claim_mode", libxl_defbool), ("event_channels", uint32), ("kernel", string), diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index 1b5381cef0..7f8fbbfb4c 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -1220,8 +1220,9 @@ void parse_config_data(const char *config_source, XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms, *usbctrls, *usbdevs, *p9devs, *vdispls, *pvcallsifs_dev= s; XLU_ConfigList *channels, *ioports, *irqs, *iomem, *viridian, *dtdevs, - *mca_caps; - int num_ioports, num_irqs, num_iomem, num_cpus, num_viridian, num_mca_= caps; + *mca_caps, *colors; + int num_ioports, num_irqs, num_iomem, num_cpus, num_viridian, num_mca_= caps, + num_colors; int pci_power_mgmt =3D 0; int pci_msitranslate =3D 0; int pci_permissive =3D 0; @@ -1370,6 +1371,53 @@ void parse_config_data(const char *config_source, if (!xlu_cfg_get_long (config, "maxmem", &l, 0)) b_info->max_memkb =3D l * 1024; =20 + if (!xlu_cfg_get_list(config, "colors", &colors, &num_colors, 0)) { + int k, p, cur_index; + + b_info->num_colors =3D 0; + /* Get number of colors based on ranges */ + for (i =3D 0; i < num_colors; i++) { + uint32_t start =3D 0, end =3D 0; + + buf =3D xlu_cfg_get_listitem(colors, i); + if (!buf) { + fprintf(stderr, + "xl: Unable to get element %d in colors range list\n",= i); + exit(1); + } + + if (sscanf(buf, "%u-%u", &start, &end) !=3D 2) { + if (sscanf(buf, "%u", &start) !=3D 1) { + fprintf(stderr, "xl: Invalid color range: %s\n", buf); + exit(1); + } + end =3D start; + } + else if (start > end) { + fprintf(stderr, + "xl: Start color is greater than end color: %s\n",= buf); + exit(1); + } + + /* Check for overlaps */ + for (k =3D start; k <=3D end; k++) { + for (p =3D 0; p < b_info->num_colors; p++) + if (b_info->colors[p] =3D=3D k) { + fprintf(stderr, "xl: Overlapped ranges not allowed= \n"); + exit(1); + } + } + + cur_index =3D b_info->num_colors; + b_info->num_colors +=3D (end - start) + 1; + b_info->colors =3D (uint32_t *)realloc(b_info->colors, + sizeof(*b_info->colors) * b_info->num_colo= rs); + + for (k =3D start; k <=3D end; k++) + b_info->colors[cur_index++] =3D k; + } + } + if (!xlu_cfg_get_long (config, "vcpus", &l, 0)) { vcpus =3D l; if (libxl_cpu_bitmap_alloc(ctx, &b_info->avail_vcpus, l)) { --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518318; cv=none; d=zohomail.com; s=zohoarc; b=ELFNOJF7bHSlOlgwmyWpjsiv8rFchxXQvSGK7Euyr70x/ULu2LEODAUeZPi7TPaRNWlrw7i/Cvwh9hkxSuO7FuZxY0kv3UNaV69zZOMhrvGnmhGEC5YdyGKxY7/Qg9HieCQoBLmjYictmVSOK2f3u4phGiTmzj1hd1ER16j+xHY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518318; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/k6VvB3GlpUbTg4zEznZ7s2FszL8KBjsUm71X4Idjuk=; b=KI0CtCZKjsjXDGY/i1hn83YOxfGq3jauXqBXAmLlDHJyTlPc71ZuO8RXAvspcwHFehKyvadHFf5SM+Uzg4lq3mN22k31P8Ib9NgvRyxbwRzJqmfBUK2Ov2rx5fbZCWsvbTC+2Iiv4SQVIabc6kb4TWgKg5kBp9fsvM5UatFvd8E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518318615870.0188617520084; Fri, 26 Aug 2022 05:51:58 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393757.632927 (Exim 4.92) (envelope-from ) id 1oRYoE-0002VN-IK; Fri, 26 Aug 2022 12:51:38 +0000 Received: by outflank-mailman (output) from mailman id 393757.632927; Fri, 26 Aug 2022 12:51:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoE-0002V7-Cg; Fri, 26 Aug 2022 12:51:38 +0000 Received: by outflank-mailman (input) for mailman id 393757; Fri, 26 Aug 2022 12:51:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoD-00013M-3t for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:37 +0000 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [2a00:1450:4864:20::633]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d1c65ce9-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:36 +0200 (CEST) Received: by mail-ej1-x633.google.com with SMTP id h22so2987366ejk.4 for ; Fri, 26 Aug 2022 05:51:36 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:35 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d1c65ce9-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=/k6VvB3GlpUbTg4zEznZ7s2FszL8KBjsUm71X4Idjuk=; b=TkjcBMbK5k+ifi9o1Ju2dmUOhDYdZocxEf3v1aJeONp3N/Qm9YFiXb/9q9AMinR2kl YhnGZb95x3BQtTbpDeqkH491kkSW2V1K6ZdKXZyY4S6hG1GRen613zzRtrGgY059aQ5M B7phPR0z0xvb+U5Xhiy44RXY0MSCmhb/zAYbas8hb48iLI7ypAlslyI+sb3EnmhrBXhZ 6G+XFBtOmFB/vn3axX4wWZfiLzGMe2KM6RzDlcvmK/UZNWyV+EjDY3Mv05EAMmJr84a0 lWf3mbjvsxt7dgNILST3Br5ATKgl4Y86pALj2BQpIg9PsVZv0PB9XeZz0vFnHV0rqpa/ qycw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=/k6VvB3GlpUbTg4zEznZ7s2FszL8KBjsUm71X4Idjuk=; b=NM6vVkUnhW6fHQ+4YYx+9ufLwBgcv9NoT/YYEbFBbNMpk/4fuzSRv4H5yoQzIWkH9R /03G2KiYWsIXzeig+WGmzoznjkQATC7kCRjkd1BfZO4RxrFwwDsQtA02vQX57StOdhEG meG8ZuIhFh3dtLX6WOHOoOzH6bwk1AyIpYHC9cuX1IMRopwrSFLRWNVNrEZMclVDUEP6 poxbWxs5dw44ZEkAHqe8BUAKWqqeaQTgSN+lQ+6KwVIXQ7nt9Rw/dN/3z9kainnLwqV9 nu9dPDz8XONrvPuo8iE/GyashnwKUpGtar1Erw78rt/dFgc/GQi6k2UdkQ78rUeRo67R aqJA== X-Gm-Message-State: ACgBeo1QHgVqHXqzc7rTTxeY7LNBsfrRYnO4gEAxBeCXMZCz66VtIS23 hwtvbQheCDvwh2FPwysZyN1CIo1a7tsNRw== X-Google-Smtp-Source: AA6agR4cmYT6GhJBqHIWhFgE6ZHbq+wC+jPgGthC2iv6Rv7beNG7K5JqPb+y3XB5NXs2nrKOgup5Rw== X-Received: by 2002:a17:907:9484:b0:738:6f9f:6032 with SMTP id dm4-20020a170907948400b007386f9f6032mr5437827ejc.602.1661518295433; Fri, 26 Aug 2022 05:51:35 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 05/12] xen/arm: add support for cache coloring configuration via device-tree Date: Fri, 26 Aug 2022 14:51:04 +0200 Message-Id: <20220826125111.152261-6-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518319689100008 Content-Type: text/plain; charset="utf-8" This commit adds the "colors" Device Tree attribute that can be used for DomUs and Dom0less color configurations. The syntax is the same used for every color config. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- docs/misc/arm/cache-coloring.rst | 45 +++++++++++++++++++++++++++ docs/misc/arm/device-tree/booting.txt | 4 +++ xen/arch/arm/coloring.c | 17 ++++++++++ xen/arch/arm/domain_build.c | 9 ++++++ xen/arch/arm/include/asm/coloring.h | 3 ++ 5 files changed, 78 insertions(+) diff --git a/docs/misc/arm/cache-coloring.rst b/docs/misc/arm/cache-colorin= g.rst index 345d97cb56..763acd2d3f 100644 --- a/docs/misc/arm/cache-coloring.rst +++ b/docs/misc/arm/cache-coloring.rst @@ -111,5 +111,50 @@ LLC way size (as previously discussed) and Dom0 colors= can be set using the appropriate command line parameters. See the relevant documentation in "docs/misc/xen-command-line.pandoc". =20 +DomUs colors can be set either in the xl configuration file (relative +documentation at "docs/man/xl.cfg.pod.5.in") or via Device Tree, also for +Dom0less configurations, as in the following example: + +.. raw:: html + + 
+        xen,xen-bootargs =3D "console=3Ddtuart dtuart=3Dserial0 dom0_mem=
=3D1G dom0_max_vcpus=3D1 sched=3Dnull llc-way-size=3D64K xen-colors=3D0-1 d=
om0-colors=3D2-6";
+        xen,dom0-bootargs "console=3Dhvc0 earlycon=3Dxen earlyprintk=3Dxen=
 root=3D/dev/ram0"
+
+        dom0 {
+            compatible =3D "xen,linux-zimage" "xen,multiboot-module";
+            reg =3D <0x0 0x1000000 0x0 15858176>;
+        };
+
+        dom0-ramdisk {
+            compatible =3D "xen,linux-initrd" "xen,multiboot-module";
+            reg =3D <0x0 0x2000000 0x0 20638062>;
+        };
+
+        domU0 {
+            #address-cells =3D <0x1>;
+            #size-cells =3D <0x1>;
+            compatible =3D "xen,domain";
+            memory =3D <0x0 0x40000>;
+            colors =3D "4-8,10,11,12";
+            cpus =3D <0x1>;
+            vpl011 =3D <0x1>;
+
+            module@2000000 {
+                compatible =3D "multiboot,kernel", "multiboot,module";
+                reg =3D <0x2000000 0xffffff>;
+                bootargs =3D "console=3DttyAMA0";
+            };
+
+            module@30000000 {
+                compatible =3D "multiboot,ramdisk", "multiboot,module";
+                reg =3D <0x3000000 0xffffff>;
+            };
+        };
+
+ +Please refer to the relative documentation in +"docs/misc/arm/device-tree/booting.txt". + Note that if no color configuration is provided for domains, they fallback= to the default one, which corresponds simply to all available colors. \ No newline at end of file diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-t= ree/booting.txt index 98253414b8..82e8dd89c8 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -162,6 +162,10 @@ with the following properties: =20 An integer specifying the number of vcpus to allocate to the guest. =20 +- colors + A string specifying the color configuration for the guest. Refer to + "docs/misc/arm/cache_coloring.rst" for syntax. + - vpl011 =20 An empty property to enable/disable a virtual pl011 for the guest to diff --git a/xen/arch/arm/coloring.c b/xen/arch/arm/coloring.c index 40a07617cc..87e20b952e 100644 --- a/xen/arch/arm/coloring.c +++ b/xen/arch/arm/coloring.c @@ -252,8 +252,11 @@ int domain_coloring_init(struct domain *d, if ( config->from_guest ) copy_from_guest(d->arch.colors, config->colors, config->num_co= lors); else + { memcpy(d->arch.colors, config->colors.p, sizeof(unsigned int) * config->num_colors); + xfree(config->colors.p); + } } =20 if ( !d->arch.colors ) @@ -283,6 +286,20 @@ void domain_dump_coloring_info(struct domain *d) print_colors(d->arch.colors, d->arch.num_colors); } =20 +void prepare_color_domain_config(struct xen_arch_domainconfig *config, + const char *colors_str) +{ + unsigned int num; + + config->colors.p =3D xzalloc_array(unsigned int, max_colors); + if ( !config->colors.p ) + panic("Unable to allocate cache colors\n"); + + if ( parse_color_config(colors_str, config->colors.p, &num) ) + panic("Error parsing the color configuration\n"); + config->num_colors =3D (uint16_t)num; +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 4d4cb692fc..979b9c8906 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -34,6 +34,7 @@ #include =20 #ifdef CONFIG_CACHE_COLORING +#include #define XEN_DOM0_CREATE_FLAGS CDF_privileged #else #define XEN_DOM0_CREATE_FLAGS CDF_privileged | CDF_directmap @@ -3275,6 +3276,7 @@ void __init create_domUs(void) struct dt_device_node *node; const struct dt_device_node *cpupool_node, *chosen =3D dt_find_node_by_path("/chosen"= ); + const char * __maybe_unused colors_str; =20 BUG_ON(chosen =3D=3D NULL); dt_for_each_child_node(chosen, node) @@ -3354,6 +3356,13 @@ void __init create_domUs(void) d_cfg.cpupool_id =3D pool_id; } =20 +#ifdef CONFIG_CACHE_COLORING + if ( dt_find_property(node, "xen,static-mem", NULL) ) + panic("static-mem is not valid when cache coloring is enabled\= n"); + if ( !dt_property_read_string(node, "colors", &colors_str) ) + prepare_color_domain_config(&d_cfg.arch, colors_str); +#endif + /* * The variable max_init_domid is initialized with zero, so here i= t's * very important to use the pre-increment operator to call diff --git a/xen/arch/arm/include/asm/coloring.h b/xen/arch/arm/include/asm= /coloring.h index 967031ae04..b7fa323870 100644 --- a/xen/arch/arm/include/asm/coloring.h +++ b/xen/arch/arm/include/asm/coloring.h @@ -36,4 +36,7 @@ int domain_coloring_init(struct domain *d, void domain_coloring_free(struct domain *d); void domain_dump_coloring_info(struct domain *d); =20 +void prepare_color_domain_config(struct xen_arch_domainconfig *config, + const char *colors_str); + #endif /* !__ASM_ARM_COLORING_H__ */ --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518321; cv=none; d=zohomail.com; s=zohoarc; b=Nm/xhjZFPoCWSaRvRaSzzbuz18VZHTdY4WMRZdG2UpF1Lamx1yKJVYY1LysKbQD0lKoqFNJR7DqMarKL4w7r057Pb4E4v9QKtSyds54J9hapGJ5WHxUnBo+Xt6NNNWHAI+xoTK8fK3IKZnK1vhVzbp0YBItKjcJoEx2n2rqEYH4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518321; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=U19EH6QAwxhEkTeWIxszhQ+InALJqVaMoD+q8dGRBDI=; b=OW9LLx2kNyVsJQHHjCf9O2c4VZrC4p6XNPzBAIzIxj3YlXOPYqKb2T/ex168ieOu5LQlcl+D2qXv97WaIz14Um5F1JPXZPvogfIthiG2pIsm+jR7tlm8Z3X+JZID9dHC4A/PpDs8yfsDG6Jya37Yi/WLbeFW0QrT0dedgLIQ4F4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518321914840.546711825016; Fri, 26 Aug 2022 05:52:01 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393759.632945 (Exim 4.92) (envelope-from ) id 1oRYoH-0002vW-Lc; Fri, 26 Aug 2022 12:51:41 +0000 Received: by outflank-mailman (output) from mailman id 393759.632945; Fri, 26 Aug 2022 12:51:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoH-0002v8-Bm; Fri, 26 Aug 2022 12:51:41 +0000 Received: by outflank-mailman (input) for mailman id 393759; Fri, 26 Aug 2022 12:51:40 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoF-0001dl-O1 for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:39 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d2ca2647-253d-11ed-bd2e-47488cf2e6aa; Fri, 26 Aug 2022 14:51:38 +0200 (CEST) Received: by mail-ej1-x634.google.com with SMTP id gb36so2951283ejc.10 for ; Fri, 26 Aug 2022 05:51:37 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:36 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d2ca2647-253d-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=U19EH6QAwxhEkTeWIxszhQ+InALJqVaMoD+q8dGRBDI=; b=OEhuXHKa4EUgvKnKCL3I/lh2E1gm1vyFuhpuNyjQ4SQW8XEvAJiTpZcR3rL+fYvWF4 piNtjknBvC7gIarGTq3b8Nqjq8v2o2jaxC/k8RHqzLkVA1swbQbmFVSWZAomIB4ZbOrX HytBtYHb+A2jVS89INTytidDiDvaGYmDZJXZwSy5ka+s+BJBxdwqAXn4SEc4D1jlTxn/ 1tYSaWiW53syvxttlnp2KGjhV9mq088djluFtJ1YhClWsdEVkoRDFcbGDGhh4L/Y1SSM laxqDbkNUCwYywEd1TfzT1rHVVgr1JOfuLawIdPZjDWdvNJ0hOaZ6l4mBVA/S5cljkST 7eWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=U19EH6QAwxhEkTeWIxszhQ+InALJqVaMoD+q8dGRBDI=; b=S2i09H0tjZfAQchdiEtCJDpS2GqsH54gd22H3wPFVPalYPG7G2BjzRz8ZcaTVy3jku O3coM5UyLCkHXmqxBaKCxyoIMtRBEIfjB5jTZjHAqxepydMZ2SZoi7wcdkA7RnL0zEe0 J9j2H9Wwi6xMBU5gVEggp491zCBOHMKRAD+sDZsRZ08jwjweSMUa2BtQAyS+2qjMjN4P qpHhl0tJ8x2eTAwjZ3sr8kQ6ueS1LATmcyLPPf5j0vz2g2x4CDG9OoegDtkrxqGY3SPn AjsvcLMi76k9FN6sz0ctupsccvyJU6kfsXy0Ddlpr2pWP4vDmgj7MYzcdBz6q0mak1hF olKA== X-Gm-Message-State: ACgBeo2au8rF59MP5vO5Yx/ovXGp78RbpvgofmqxPFqJ9a3osGsZgrSI 5cqcbISptZJu3Wny2JOyU+eS0a2d1LiEEA== X-Google-Smtp-Source: AA6agR6zOQOOy13zYcedLOKfh3IPynjNphh93x6YWFOqV7oY0e9XjPwMdznT5ZczJ9Glh7H8uM4zRg== X-Received: by 2002:a17:906:8cb0:b0:730:d6f3:279e with SMTP id qr48-20020a1709068cb000b00730d6f3279emr5705871ejc.25.1661518296990; Fri, 26 Aug 2022 05:51:36 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 06/12] xen/common: add cache coloring allocator for domains Date: Fri, 26 Aug 2022 14:51:05 +0200 Message-Id: <20220826125111.152261-7-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518323671100020 Content-Type: text/plain; charset="utf-8" This commit adds a new memory page allocator that implements the cache coloring mechanism. The allocation algorithm follows the given color configuration of the domain and maximizes contiguity in the page selection. Pages are stored in a color-indexed data structure of lists, sorted by their machine addresses, that are collectively called the colored heap. A simple initialization function computes the color of any available page and inserts it in the corresponding list. When a domain requests a page, the allocator takes one from the subset of lists whose colors equals the domain configuration. It chooses the page with the highest machine address such th= at contiguous pages are sequentially allocated, if this is made possible by a color assignment which includes adjacent colors. The allocator can handle only requests with order equals to 0 since the single color granularity is represented in memory by one page. The buddy allocator must coexist with the colored one because the Xen heap isn't colored. For this reason a new Kconfig option and a command line parameter are added to let the user set the amount of memory reserved for the buddy allocator. Even when cache coloring is enabled, this memory isn't managed by the colored allocator. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- docs/misc/arm/cache-coloring.rst | 43 +++++- docs/misc/xen-command-line.pandoc | 14 ++ xen/arch/arm/Kconfig | 12 ++ xen/arch/arm/coloring.c | 10 ++ xen/arch/arm/include/asm/coloring.h | 6 + xen/arch/arm/include/asm/mm.h | 3 + xen/common/page_alloc.c | 213 ++++++++++++++++++++++++++-- 7 files changed, 290 insertions(+), 11 deletions(-) diff --git a/docs/misc/arm/cache-coloring.rst b/docs/misc/arm/cache-colorin= g.rst index 763acd2d3f..5f9132e525 100644 --- a/docs/misc/arm/cache-coloring.rst +++ b/docs/misc/arm/cache-coloring.rst @@ -13,6 +13,9 @@ In order to enable and use it, few steps are needed. (refer to menuconfig help for value meaning and when it should be change= d). =20 CONFIG_MAX_CACHE_COLORS=3D +- If needed, change the amount of memory reserved for the buddy allocator = either + from the Xen configuration file, via the CONFIG_BUDDY_ALLOCATOR_SIZE val= ue, + or with the command line option. See `Colored allocator and buddy alloca= tor`. - Assign colors to domains using the `Color selection format`_ (see `Coloring parameters`_ for more documentation pointers). =20 @@ -157,4 +160,42 @@ Please refer to the relative documentation in "docs/misc/arm/device-tree/booting.txt". =20 Note that if no color configuration is provided for domains, they fallback= to -the default one, which corresponds simply to all available colors. \ No newline at end of file +the default one, which corresponds simply to all available colors. + +Colored allocator and buddy allocator +************************************* + +The colored allocator distributes pages based on color configurations of +domains so that each domains only gets pages of its own colors. +The colored allocator is meant as an alternative to the buddy allocator be= cause +its allocation policy is by definition incompatible with the generic one. = Since +the Xen heap systems is not colored yet, we need to support the coexistenc= e of +the two allocators and some memory must be left for the buddy one. +The buddy allocator memory can be reserved from the Xen configuration file= or +with the help of a command-line option. + +Known issues and limitations +**************************** + +Colored allocator can only make use of order-0 pages +#################################################### + +The cache coloring technique relies on memory mappings and on the smallest +amount of memory that can be mapped to achieve the maximum number of colors +(cache partitions) possible. This amount is what is normally called a page= and, +in Xen terminology, the order-0 page is the smallest one. The fairly simple +colored allocator currently implemented, makes use only of such pages. +It must be said that a more complex one could, in theory, adopt higher ord= er +pages if the colors selection contained adjacent colors. Two subsequent co= lors, +for example, can be represented by a order-1 page, four colors correspond = to +a order-2 pages, etc. + +Fail to boot colored DomUs with large memory size +################################################# + +If the Linux kernel used for Dom0 does not contain the upstream commit +3941552aec1e04d63999988a057ae09a1c56ebeb and uses the hypercall buffer dev= ice, +colored DomUs with memory size larger then 127 MB cannot be created. This = is +caused by the default limit of this buffer of 64 pages. The solution is to +manually apply the above patch, or to check if there is an updated version= of +the kernel in use for Dom0 that contains this change. diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index 910ebeb2eb..4e85c4dfe4 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -299,6 +299,20 @@ can be maintained with the pv-shim mechanism. cause Xen not to use Indirect Branch Tracking even when support is available in hardware. =20 +### buddy-alloc-size (arm64) +> `=3D ` + +> Default: `64M` + +Amount of memory reserved for the buddy allocator when colored allocator is +active. This options is parsed only when cache coloring support is enabled. +The colored allocator is meant as an alternative to the buddy allocator, +because its allocation policy is by definition incompatible with the +generic one. Since the Xen heap systems is not colored yet, we need to +support the coexistence of the two allocators for now. This parameter, whi= ch is +optional and for expert only, it's used to set the amount of memory reserv= ed to +the buddy allocator. + ### clocksource (x86) > `=3D pit | hpet | acpi | tsc` =20 diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 8acff9682c..abce4bfc25 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -147,6 +147,18 @@ config MAX_CACHE_COLORS colors at boot. Note that if, at any time, a color configuration with m= ore colors than the maximum will be employed an error will be produced. =20 +config BUDDY_ALLOCATOR_SIZE + string "Buddy allocator reserved memory size" if CACHE_COLORING + default "64M" if CACHE_COLORING + default "0M" if !CACHE_COLORING + help + Amount of memory reserved for the buddy allocator to work alongside + the colored one. The colored allocator is meant as an alternative to the + buddy allocator because its allocation policy is by definition + incompatible with the generic one. Since the Xen heap systems is not + colored yet, we need to support the coexistence of the two allocators a= nd + some memory must be left for the buddy one. + config TEE bool "Enable TEE mediators support (UNSUPPORTED)" if UNSUPPORTED default n diff --git a/xen/arch/arm/coloring.c b/xen/arch/arm/coloring.c index 87e20b952e..3fb86043d1 100644 --- a/xen/arch/arm/coloring.c +++ b/xen/arch/arm/coloring.c @@ -300,6 +300,16 @@ void prepare_color_domain_config(struct xen_arch_domai= nconfig *config, config->num_colors =3D (uint16_t)num; } =20 +unsigned int page_to_color(struct page_info *pg) +{ + return addr_to_color(page_to_maddr(pg)); +} + +unsigned int get_max_colors(void) +{ + return max_colors; +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/include/asm/coloring.h b/xen/arch/arm/include/asm= /coloring.h index b7fa323870..0982bc9853 100644 --- a/xen/arch/arm/include/asm/coloring.h +++ b/xen/arch/arm/include/asm/coloring.h @@ -29,6 +29,8 @@ =20 #include =20 +struct page_info; + bool __init coloring_init(void); =20 int domain_coloring_init(struct domain *d, @@ -39,4 +41,8 @@ void domain_dump_coloring_info(struct domain *d); void prepare_color_domain_config(struct xen_arch_domainconfig *config, const char *colors_str); =20 +unsigned int page_to_color(struct page_info *pg); + +unsigned int get_max_colors(void); + #endif /* !__ASM_ARM_COLORING_H__ */ diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index da25251cda..a59fc3791a 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -143,6 +143,9 @@ struct page_info #define PGC_count_width PG_shift(10) #define PGC_count_mask ((1UL< +#endif =20 #ifndef PGC_static #define PGC_static 0 @@ -231,6 +234,9 @@ static bool __read_mostly scrub_debug; #define scrub_debug false #endif =20 +/* Memory required for buddy allocator to work with colored one */ +static unsigned long __initdata buddy_alloc_size; + /* * Bit width of the DMA heap -- used to override NUMA-node-first. * allocation strategy, which can otherwise exhaust low memory. @@ -440,7 +446,172 @@ mfn_t __init alloc_boot_pages(unsigned long nr_pfns, = unsigned long pfn_align) BUG(); } =20 +static DEFINE_SPINLOCK(heap_lock); =20 +/* Initialise fields which have other uses for free pages. */ +static void init_free_page_fields(struct page_info *pg) +{ + pg->u.inuse.type_info =3D PGT_TYPE_INFO_INITIALIZER; + page_set_owner(pg, NULL); +} + +static bool is_page_colored(struct page_info *pg) +{ + return pg->count_info & PGC_colored; +} + +#ifdef CONFIG_CACHE_COLORING +/************************* + * COLORED SIDE-ALLOCATOR + * + * Pages are stored by their color in separate lists. Each list defines a = color + * and it is initialized during end_boot_allocator, where each page's color + * is calculated and the page itself is put in the correct list. + * After initialization there will be N lists where N is the number of max= imum + * available colors on the platform. + */ +typedef struct page_list_head colored_pages_t; +static colored_pages_t __ro_after_init *color_heap; + +#define colored_pages(color) &color_heap[(color)] + +static void color_heap_insert_page(struct page_info *pg) +{ + struct page_info *pos; + struct page_list_head *head =3D colored_pages(page_to_color(pg)); + + pg->count_info |=3D PGC_colored; + + /* Insert page in list in ascending machine address order */ + page_list_for_each( pos, head ) + { + if ( page_to_maddr(pos) < page_to_maddr(pg) ) + { + head =3D &pos->list; + break; + } + } + + page_list_add_tail(pg, head); +} + +static void color_heap_remove_page(struct page_info *pg) +{ + page_list_del(pg, colored_pages(page_to_color(pg))); +} + +static void __init init_col_heap_pages(struct page_info *pg, + unsigned long nr_pages) +{ + unsigned int i; + + if ( !color_heap ) + { + unsigned int max_colors =3D get_max_colors(); + color_heap =3D xmalloc_array(colored_pages_t, max_colors); + BUG_ON(!color_heap); + + for ( i =3D 0; i < max_colors; i++ ) + INIT_PAGE_LIST_HEAD(colored_pages(i)); + + if ( !buddy_alloc_size ) + buddy_alloc_size =3D parse_size_and_unit(CONFIG_BUDDY_ALLOCATO= R_SIZE, + NULL); + } + + printk(XENLOG_INFO "Init color heap with %lu pages\n", nr_pages); + printk(XENLOG_INFO "Paging starting from: 0x%"PRIx64"\n", + page_to_maddr(pg)); + + for ( i =3D 0; i < nr_pages; i++ ) + color_heap_insert_page(pg++); +} + +/* Alloc one page based on domain color configuration */ +static struct page_info *alloc_col_heap_page(unsigned int memflags, + const unsigned int *colors, + unsigned int num_colors) +{ + struct page_info *pg =3D NULL; + unsigned int i; + bool need_tlbflush =3D false; + uint32_t tlbflush_timestamp =3D 0; + + spin_lock(&heap_lock); + + for ( i =3D 0; i < num_colors; i++ ) + { + struct page_info *tmp; + + if ( page_list_empty(colored_pages(colors[i])) ) + continue; + + tmp =3D page_list_first(colored_pages(colors[i])); + if ( !pg || page_to_maddr(tmp) > page_to_maddr(pg) ) + pg =3D tmp; + } + + if ( !pg ) + { + spin_unlock(&heap_lock); + return NULL; + } + + pg->count_info =3D PGC_state_inuse; + + if ( !(memflags & MEMF_no_tlbflush) ) + accumulate_tlbflush(&need_tlbflush, pg, &tlbflush_timestamp); + + init_free_page_fields(pg); + flush_page_to_ram(mfn_x(page_to_mfn(pg)), + !(memflags & MEMF_no_icache_flush)); + color_heap_remove_page(pg); + + spin_unlock(&heap_lock); + + if ( need_tlbflush ) + filtered_flush_tlb_mask(tlbflush_timestamp); + + return pg; +} + +static void free_col_domheap_page(struct page_info *pg) +{ + pg->count_info =3D PGC_state_free; + page_set_owner(pg, NULL); + color_heap_insert_page(pg); +} + +static struct page_info *alloc_col_domheap_page(struct domain *d, + unsigned int memflags) +{ + struct page_info *pg; + + ASSERT(!in_irq()); + + pg =3D alloc_col_heap_page(memflags, d->arch.colors, d->arch.num_color= s); + if ( !pg ) + { + printk(XENLOG_ERR "Colored page is null for domain %pd\n", d); + return NULL; + } + + if ( d && !(memflags & MEMF_no_owner) && assign_page(pg, 0, d, memflag= s) ) + { + free_col_domheap_page(pg); + return NULL; + } + + return pg; +} + +size_param("buddy-alloc-size", buddy_alloc_size); +#else +static void free_col_domheap_page(struct page_info *pg) +{ + return; +} +#endif /* CONFIG_CACHE_COLORING */ =20 /************************* * BINARY BUDDY ALLOCATOR @@ -462,7 +633,6 @@ static unsigned long node_need_scrub[MAX_NUMNODES]; static unsigned long *avail[MAX_NUMNODES]; static long total_avail_pages; =20 -static DEFINE_SPINLOCK(heap_lock); static long outstanding_claims; /* total outstanding claims by all domains= */ =20 unsigned long domain_adjust_tot_pages(struct domain *d, long pages) @@ -1027,10 +1197,7 @@ static struct page_info *alloc_heap_pages( accumulate_tlbflush(&need_tlbflush, &pg[i], &tlbflush_timestamp); =20 - /* Initialise fields which have other uses for free pages. */ - pg[i].u.inuse.type_info =3D PGT_TYPE_INFO_INITIALIZER; - page_set_owner(&pg[i], NULL); - + init_free_page_fields(&pg[i]); } =20 spin_unlock(&heap_lock); @@ -1926,6 +2093,7 @@ static unsigned long avail_heap_pages( void __init end_boot_allocator(void) { unsigned int i; + unsigned long buddy_pages =3D PFN_DOWN(buddy_alloc_size); =20 /* Pages that are free now go to the domain sub-allocator. */ for ( i =3D 0; i < nr_bootmem_regions; i++ ) @@ -1939,11 +2107,24 @@ void __init end_boot_allocator(void) break; } } - for ( i =3D nr_bootmem_regions; i-- > 0; ) + + for ( i =3D 0; i < nr_bootmem_regions; i++ ) { struct bootmem_region *r =3D &bootmem_region_list[i]; + + if ( buddy_pages && (r->s < r->e) ) + { + unsigned long pages =3D MIN(r->e - r->s, buddy_pages); + init_heap_pages(mfn_to_page(_mfn(r->s)), pages); + r->s +=3D pages; + buddy_pages -=3D pages; + } if ( r->s < r->e ) +#ifdef CONFIG_CACHE_COLORING + init_col_heap_pages(mfn_to_page(_mfn(r->s)), r->e - r->s); +#else init_heap_pages(mfn_to_page(_mfn(r->s)), r->e - r->s); +#endif } nr_bootmem_regions =3D 0; =20 @@ -2429,6 +2610,17 @@ struct page_info *alloc_domheap_pages( =20 ASSERT_ALLOC_CONTEXT(); =20 +#ifdef CONFIG_CACHE_COLORING + /* Only domains are supported for coloring */ + if ( d ) + { + /* Colored allocation must be done on 0 order */ + if ( order ) + return NULL; + return alloc_col_domheap_page(d, memflags); + } +#endif + bits =3D domain_clamp_alloc_bitsize(memflags & MEMF_no_owner ? NULL : = d, bits ? : (BITS_PER_LONG+PAGE_SHIFT)); =20 @@ -2546,7 +2738,10 @@ void free_domheap_pages(struct page_info *pg, unsign= ed int order) scrub =3D 1; } =20 - free_heap_pages(pg, order, scrub); + if ( is_page_colored(pg) ) + free_col_domheap_page(pg); + else + free_heap_pages(pg, order, scrub); } =20 if ( drop_dom_ref ) @@ -2759,9 +2954,7 @@ static struct page_info * __init acquire_staticmem_pa= ges(mfn_t smfn, * to PGC_state_inuse. */ pg[i].count_info =3D PGC_static | PGC_state_inuse; - /* Initialise fields which have other uses for free pages. */ - pg[i].u.inuse.type_info =3D PGT_TYPE_INFO_INITIALIZER; - page_set_owner(&pg[i], NULL); + init_free_page_fields(&pg[i]); } =20 spin_unlock(&heap_lock); --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518318; cv=none; d=zohomail.com; s=zohoarc; b=QYJNIUOB7xqkbwDoX0/O5ciWy55Dd8f2kirhYcqDXFepS0urYse7sP+WwS0U/QSUEoaEqMV+9i/5kpkLHxWMR/2CLTCQaUYRSZ/JQFNRMRNk68h5u08R60642VRUnWQXA87lnpDXZCg/oOp6kWfT1gVWWk4DYbgAT9y2+C18Zj4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518318; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8q8cSK0/ZbUuwajE+w5sunrFXcqx/YGVTD9wQaToEI0=; b=kWkPiAvhLewQRiXYFwRer4Vsu2KllzXngjUmJtEP9Cv+Ws9uXbRCRAmHJklp8b8XZcLRbREOmJLHBPTeYy/7ayezRN0heMc5TN7nMLoAvGJksmNVRmKG5ZxbbDXLp/4iPWkAGPDG55x8w/uSlU34LkNzyhyLlr+3yBoWYVki/lM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518318400527.9970613625567; Fri, 26 Aug 2022 05:51:58 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393758.632938 (Exim 4.92) (envelope-from ) id 1oRYoH-0002r6-4p; Fri, 26 Aug 2022 12:51:41 +0000 Received: by outflank-mailman (output) from mailman id 393758.632938; Fri, 26 Aug 2022 12:51:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoG-0002qw-Vv; Fri, 26 Aug 2022 12:51:40 +0000 Received: by outflank-mailman (input) for mailman id 393758; Fri, 26 Aug 2022 12:51:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoF-00013M-E0 for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:39 +0000 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [2a00:1450:4864:20::631]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d36904fd-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:38 +0200 (CEST) Received: by mail-ej1-x631.google.com with SMTP id fy31so2562260ejc.6 for ; Fri, 26 Aug 2022 05:51:38 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:38 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d36904fd-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=8q8cSK0/ZbUuwajE+w5sunrFXcqx/YGVTD9wQaToEI0=; b=nbqKRXAqk2Udr8kI/SUFCexCVkpyFcD7/XcAuGIl8cue6/F+BzHuaCCJkXY4RWOYs0 ZshlxV4FtJIKeJ9rTD+AG9T7olxSOwbPRR1Qp1okXYmUJCxous0RgjYSD6CLBmrfv+Go P9HVDtkKvMOtfhdbFk5Qh1Cf2CQ5VVDV4qzm/LQjNaxaAK5Glnv1Ku+UjXOlU5XC/ZWT 3Zcv6QEBbmjf/wSbprs9Tnazd3U1wt1goOn1essPXdpbipZE9lCJIw6kVmmA9f4XivUx tzhUkDTCLTBFV5vXKmXUD5BeRPY83McAzK/okdxy7TYZoIRdR7Uguh//rjthiLTBdIPE Qoxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=8q8cSK0/ZbUuwajE+w5sunrFXcqx/YGVTD9wQaToEI0=; b=PSLRuQT+2sg8GFcPeibLTNhdQgHcvC199NT3KSNuP3HePvd6FTwX21j7FpzFEN1yIX 1MwukTNVxd2cAI+6KF8C0bGAGDGV/JALVn7gC4RIsCEfiFTsN30Nrv8/CUwnXFoHfs11 inVRGspGwAjeJ+khiD5+82mZPfcZuz06N0Q80jPHPvKfZW/eLhC23S4f2KUae+CP2prj N8zQpG2jUOp2XCaOnwmiqP3G0D0cEeozA4sChh9gCyn9Z1rbB3h44vFj5dfwEtrwCSRc pr7QxiLOZVh/GXyW57r7E0h3Vd5Y2jTypxet5da10gXKoI7+d9Z6LnDmym9JXjYUM3Q4 ZIeA== X-Gm-Message-State: ACgBeo3wpcbzgKf8Mkd3s3Qkjq1rpzM1SLj4qrbu+EnvIbcVBh7jX4yr OhOmdXn7kM+Vo8naTfLMfxdDFE2Cm6sZ7g== X-Google-Smtp-Source: AA6agR5U5N1r4qkF/3vBc7BJpuniwJiwk4Drng/nfaijetjjGomI6h419PBY2XUs+KbJ4yUPlFIPgw== X-Received: by 2002:a17:907:d08:b0:72f:b107:c07a with SMTP id gn8-20020a1709070d0800b0072fb107c07amr5492067ejc.340.1661518298352; Fri, 26 Aug 2022 05:51:38 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 07/12] xen/common: add colored heap info debug-key Date: Fri, 26 Aug 2022 14:51:06 +0200 Message-Id: <20220826125111.152261-8-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518319644100007 Content-Type: text/plain; charset="utf-8" This commit adds a debug-key to let the user inspect the colored heap information. The number of pages stored for each available color is dumped. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- xen/common/page_alloc.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 4ae3cfe9a7..be6bb2b9a1 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -605,6 +605,27 @@ static struct page_info *alloc_col_domheap_page(struct= domain *d, return pg; } =20 +static void dump_col_heap(unsigned char key) +{ + struct page_info *pg; + unsigned long pages; + unsigned int color; + + printk("'%c' pressed -> dumping coloring heap info\n", key); + + for ( color =3D 0; color < get_max_colors(); color++ ) + { + printk("Heap[%u]: ", color); + pages =3D 0; + page_list_for_each( pg, colored_pages(color) ) + { + BUG_ON(!(page_to_color(pg) =3D=3D color)); + pages++; + } + printk("%lu pages\n", pages); + } +} + size_param("buddy-alloc-size", buddy_alloc_size); #else static void free_col_domheap_page(struct page_info *pg) @@ -2853,6 +2874,9 @@ static void cf_check dump_heap(unsigned char key) static __init int cf_check register_heap_trigger(void) { register_keyhandler('H', dump_heap, "dump heap info", 1); +#ifdef CONFIG_CACHE_COLORING + register_keyhandler('k', dump_col_heap, "dump coloring heap info", 1); +#endif return 0; } __initcall(register_heap_trigger); --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518319; cv=none; d=zohomail.com; s=zohoarc; b=F/AT0NsrYI8v5MON1H0Drbu3gE9tASonDYw+af6SNy1OCmRnW6OcXLBFe1Hovu/pnOtyykrMOrYC+VB+i2BdbacqAneorLnWrfARa0aZOoSdxYxUOk+H+OhM9JBEF/+0yQX/kgfDgW1FMpLL4hssxhZlK89IY2MjZZBD8TY/tdw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518319; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mumtEpYDzsfqqrj/UQbqekiy0HKBQR3otaMx18E1gBE=; b=lB/qmuchPN8Xxh3SkCUdKpRN8aHGmUaA2c8KISa0T9/r1WnHNgUs/kNYdln6Knwd3fNYIp5PR8VuklUOaGbO76udDamCqgjfTiHX5zs/5Yu4bNEF6ysxCZa/GzUagFm1oGIE6+gCMdbRQum5KfTvozjuC2AfLZTqlUqAFvAoX3M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518319697212.9993200497986; Fri, 26 Aug 2022 05:51:59 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393760.632958 (Exim 4.92) (envelope-from ) id 1oRYoJ-0003Lz-1R; Fri, 26 Aug 2022 12:51:43 +0000 Received: by outflank-mailman (output) from mailman id 393760.632958; Fri, 26 Aug 2022 12:51:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoI-0003L2-Rf; Fri, 26 Aug 2022 12:51:42 +0000 Received: by outflank-mailman (input) for mailman id 393760; Fri, 26 Aug 2022 12:51:41 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoH-0001dl-6a for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:41 +0000 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [2a00:1450:4864:20::533]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d448dcee-253d-11ed-bd2e-47488cf2e6aa; Fri, 26 Aug 2022 14:51:40 +0200 (CEST) Received: by mail-ed1-x533.google.com with SMTP id c93so1988472edf.5 for ; Fri, 26 Aug 2022 05:51:40 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:39 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d448dcee-253d-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=mumtEpYDzsfqqrj/UQbqekiy0HKBQR3otaMx18E1gBE=; b=xF6d3CycQqSXZr5mXMfex5FSMUku5QfnDhd1tcf6hVs4Oi+ZNAeDUydlhx8f0CfXyq dkaXsxCUrFBFNPJ7e6aPQciKup8yB72BgamDmYUhJjaH6XslapKtwA04Il5BUOxRFiYU PuHPim2R870aUIlOiBurGNfPxumFp/gBY0dwT1d3G+LxcUKIy2ItfJJ0JWIbrt3iqBNd zU1FPTnDmSJQ1L5ZFxQmEkfLpj29qa92k4Z0R84HswIdCQAObOc3whqV3XmRxPHrvKaE yTB23z2fUcfxiA6C8OVMaxOQDWx+GA+N1zrmqC5Ux4GQ4b0dFoLRJOaPkpQy4GiGN9OJ 15Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=mumtEpYDzsfqqrj/UQbqekiy0HKBQR3otaMx18E1gBE=; b=wNe7fNzriQgrKXp3aKEzb+16kSyeUU1PJbAcCCiWHMXpu6IMu+v6gYxfdcJSbhavos fa/ajXEUc+NSPhF5l5HvOLZDcsN4zVk1kjokiSpjZTkODD4SGs14T70gxQYxzx7dHA6H pd1G7YEsVuocUoBzExFinYLxNLssWY6poorr7E87jIzcIuydHMG3Ww/MEaT7dFr5i07P GHeNsVOum8lC8azIziIyzq0ZUuctq8V2269Sl/6NuOLhHblS3NxVuOucUD+Ctw3eSQzv b3LbflMZH3fOECpqtygfg97LQnx9hbc+AUkfSXYLjzbGJecgcpgxV5HnYnZPqsCq8qpj 2CIg== X-Gm-Message-State: ACgBeo0ki/0NUR0fzG+68lwB9CGUrF0ozrT96baYCkUsE80zikjiteiW FuG1UsPw+ujfkYPezOWPcEmRqRxhm6Gbqg== X-Google-Smtp-Source: AA6agR7lZPkiAL9EKsSHYPBtWPKEClMLiQYt9tN+2HJKmX5UBmoBaXeFRyLfatr9kwkHX1bebvJ6kQ== X-Received: by 2002:a05:6402:5292:b0:446:80b0:4a5d with SMTP id en18-20020a056402529200b0044680b04a5dmr6771606edb.285.1661518299763; Fri, 26 Aug 2022 05:51:39 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 08/12] Revert "xen/arm: setup: Add Xen as boot module before printing all boot modules" Date: Fri, 26 Aug 2022 14:51:07 +0200 Message-Id: <20220826125111.152261-9-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518321781100013 Content-Type: text/plain; charset="utf-8" From: Luca Miccio This reverts commit 48fb2a9deba11ee48dde21c5c1aa93b4d4e1043b. The cache coloring support has the command line parsing as a prerequisite because of the color configurations passed in this way. Also, the Xen boot module will be placed at an address that depends on the coloring initialization. This commit moves the Xen boot module after the coloring initialization to allow the order of operations previously described to take place. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- xen/arch/arm/setup.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index c02f21c0e6..611c93ad7d 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -921,12 +921,6 @@ void __init start_xen(unsigned long boot_phys_offset, "Please check your bootloader.\n", fdt_paddr); =20 - /* Register Xen's load address as a boot module. */ - xen_bootmodule =3D add_boot_module(BOOTMOD_XEN, - (paddr_t)(uintptr_t)(_start + boot_phys_offse= t), - (paddr_t)(uintptr_t)(_end - _start), false); - BUG_ON(!xen_bootmodule); - fdt_size =3D boot_fdt_info(device_tree_flattened, fdt_paddr); =20 cmdline =3D boot_fdt_cmdline(device_tree_flattened); @@ -938,6 +932,12 @@ void __init start_xen(unsigned long boot_phys_offset, panic("Xen Coloring support: setup failed\n"); #endif =20 + /* Register Xen's load address as a boot module. */ + xen_bootmodule =3D add_boot_module(BOOTMOD_XEN, + (paddr_t)(uintptr_t)(_start + boot_phys_offse= t), + (paddr_t)(uintptr_t)(_end - _start + 1), fals= e); + BUG_ON(!xen_bootmodule); + setup_mm(); =20 /* Parse the ACPI tables for possible boot-time configuration */ --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518321; cv=none; d=zohomail.com; s=zohoarc; b=LFkPSVtMNphzWaJhfPGOvEejauCK1fTPbLi0u5ifcxybPiD6CeBmoo0+c9sHVPNZAB7WoNSpddG1jW5yf54lI4O/U4lMIsgGuxyE8MYSLDbCrMBQxq9VTvWtx4WzNCN36jcfBDFPhWltqbMbvORjSxpQoei+e5qPH58YAPV9YEM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518321; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5ZYX74i6LNybppj1kPT1aVudjHbJkWqFfMxF9yVRhVY=; b=Elgt4Tx+bCpZBp4nJpXi0GzhVQ2yiVcU5eui8PAiB5FIzB3hOsntKHi+eBhonPe/thaMFLCvIEl20mu4CxchNEIhwTMUP2KkfdLdTzKZ2BHcc33L2607KfetRwm34972qmbF3sU5PqMRf3Mq6e8AUutqTdsR3VFiD4uUVI2stLM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 166151832142332.174750473436575; Fri, 26 Aug 2022 05:52:01 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393761.632966 (Exim 4.92) (envelope-from ) id 1oRYoJ-0003WU-Qp; Fri, 26 Aug 2022 12:51:43 +0000 Received: by outflank-mailman (output) from mailman id 393761.632966; Fri, 26 Aug 2022 12:51:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoJ-0003U6-Hg; Fri, 26 Aug 2022 12:51:43 +0000 Received: by outflank-mailman (input) for mailman id 393761; Fri, 26 Aug 2022 12:51:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoI-00013M-H9 for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:42 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d5106182-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:41 +0200 (CEST) Received: by mail-ed1-x52e.google.com with SMTP id w20so1965831edd.10 for ; Fri, 26 Aug 2022 05:51:41 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:40 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d5106182-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=5ZYX74i6LNybppj1kPT1aVudjHbJkWqFfMxF9yVRhVY=; b=O5sRob496fZhG2BZn4uIt5GZ3ja7k1UqYhko8MlIjEUtXueOnpsv2F1pQFcbOF21vZ o4EYz5MBaf6XzranrszO0sqEhgTv1T9KU+fWWk1v6dYI09ydgcfOAnY1UPLYGeSxZYYn dz9MvRNFShUbxZW4x2310DD4jfKF7oZumLS0VBfTHIKwzIN48Xi2Xd7kRBtiB+4z3rsU P40X+M14muXCGKd+0PGo/3YMol84t3/2xoMtwuUrnZe5rnTXlB6wQ98vJWTZqLTN4RIQ 3BhKh66ooKHwZ6wIc7Cy01dPbbZ3UIdaVvw2/XQEiZkF3HT/Z99x5+Usoz8edjxl5n0N kmsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=5ZYX74i6LNybppj1kPT1aVudjHbJkWqFfMxF9yVRhVY=; b=nMQ0f1sqF7uhPtW9U3ubBAZL+yw5Xd96a9Gmyr7vjePGYH1eMMv7mhDMLsUhXQNdlL nMleuO5dgAH368l/xX71dvMM31eYSmlGa8yqO1JOymmOM/j9ml7HRyFN7Uqzn6k7I2zt FvKQlKyTb+vydn7UDGdjfnavWpJAkm7Kv9xsDRAEn14NxWJerz/o+khQK9D9cgkoyiH8 /MCkWF0ASFtp2275Iq57xqIutoQDl/CpZ9K1zPMFuOYPc/cSvJGUY5kbR0PoSaHTa3ZE F/Vq8BzgiPHj+LUdeM5QKlgd/aP+KwQEP0RksZDxGL8TxnyLB14LFcEs/VZSgVpImL0N YQUg== X-Gm-Message-State: ACgBeo2ZDlVNHd6JOj8hi3GNUrMsFVLVAiiGWYhbdhR51SNmmgJXU2nH sbB053hHfydN/ivOQW6flmUT+UFxc1cdAA== X-Google-Smtp-Source: AA6agR53Q+BBK6cYj+MeqnJJqd2wgmTW0fOIWt3YkTx7Ek4v3VbQAc0emVnvQhW665dRlaNdmqwq9g== X-Received: by 2002:a05:6402:1d51:b0:41f:cf6c:35a5 with SMTP id dz17-20020a0564021d5100b0041fcf6c35a5mr6911189edb.25.1661518301097; Fri, 26 Aug 2022 05:51:41 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 09/12] Revert "xen/arm: mm: Initialize page-tables earlier" Date: Fri, 26 Aug 2022 14:51:08 +0200 Message-Id: <20220826125111.152261-10-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518321794100014 Content-Type: text/plain; charset="utf-8" From: Luca Miccio This reverts commit 3a5d341681af650825bbe3bee9be5d187da35080. The cache coloring support will be configurable within the Xen command line, but it will be initialized before the page-tables; this is necessary for coloring the hypervisor itself beacuse we will create a specific mapping for it that could be configured using some command line options. In order to parse all the needed information from the device tree, we need to revert the above commit and restore the previous order for page-tables initialization. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- xen/arch/arm/mm.c | 33 ++++++++++++++++++++------------- xen/arch/arm/setup.c | 4 ++-- 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index b42cddb1b4..1afa02b4af 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -368,6 +368,17 @@ static inline lpae_t pte_of_xenaddr(vaddr_t va) return mfn_to_xen_entry(maddr_to_mfn(ma), MT_NORMAL); } =20 +static void __init create_boot_mappings(unsigned long virt_offset, + mfn_t base_mfn) +{ + lpae_t pte; + + pte =3D mfn_to_xen_entry(base_mfn, MT_NORMAL); + write_pte(&boot_second[second_table_offset(virt_offset)], pte); + flush_xen_tlb_local(); +} + +/* Map the FDT in the early boot page table */ void * __init early_fdt_map(paddr_t fdt_paddr) { /* We are using 2MB superpage for mapping the FDT */ @@ -375,7 +386,6 @@ void * __init early_fdt_map(paddr_t fdt_paddr) paddr_t offset; void *fdt_virt; uint32_t size; - int rc; =20 /* * Check whether the physical FDT address is set and meets the minimum @@ -391,12 +401,7 @@ void * __init early_fdt_map(paddr_t fdt_paddr) /* The FDT is mapped using 2MB superpage */ BUILD_BUG_ON(BOOT_FDT_VIRT_START % SZ_2M); =20 - rc =3D map_pages_to_xen(BOOT_FDT_VIRT_START, maddr_to_mfn(base_paddr), - SZ_2M >> PAGE_SHIFT, - PAGE_HYPERVISOR_RO | _PAGE_BLOCK); - if ( rc ) - panic("Unable to map the device-tree.\n"); - + create_boot_mappings(BOOT_FDT_VIRT_START, maddr_to_mfn(base_paddr)); =20 offset =3D fdt_paddr % SECOND_SIZE; fdt_virt =3D (void *)BOOT_FDT_VIRT_START + offset; @@ -410,12 +415,8 @@ void * __init early_fdt_map(paddr_t fdt_paddr) =20 if ( (offset + size) > SZ_2M ) { - rc =3D map_pages_to_xen(BOOT_FDT_VIRT_START + SZ_2M, - maddr_to_mfn(base_paddr + SZ_2M), - SZ_2M >> PAGE_SHIFT, - PAGE_HYPERVISOR_RO | _PAGE_BLOCK); - if ( rc ) - panic("Unable to map the device-tree\n"); + create_boot_mappings(BOOT_FDT_VIRT_START + SZ_2M, + maddr_to_mfn(base_paddr + SZ_2M)); } =20 return fdt_virt; @@ -514,6 +515,12 @@ void __init setup_pagetables(unsigned long boot_phys_o= ffset) pte.pt.table =3D 1; xen_second[second_table_offset(FIXMAP_ADDR(0))] =3D pte; =20 + /* ... DTB */ + pte =3D boot_second[second_table_offset(BOOT_FDT_VIRT_START)]; + xen_second[second_table_offset(BOOT_FDT_VIRT_START)] =3D pte; + pte =3D boot_second[second_table_offset(BOOT_FDT_VIRT_START + SZ_2M)]; + xen_second[second_table_offset(BOOT_FDT_VIRT_START + SZ_2M)] =3D pte; + #ifdef CONFIG_ARM_64 ttbr =3D (uintptr_t) xen_pgtable + phys_offset; #else diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 611c93ad7d..bdfc05bf61 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -910,8 +910,6 @@ void __init start_xen(unsigned long boot_phys_offset, /* Initialize traps early allow us to get backtrace when an error occu= rred */ init_traps(); =20 - setup_pagetables(boot_phys_offset); - smp_clear_cpu_maps(); =20 device_tree_flattened =3D early_fdt_map(fdt_paddr); @@ -938,6 +936,8 @@ void __init start_xen(unsigned long boot_phys_offset, (paddr_t)(uintptr_t)(_end - _start + 1), fals= e); BUG_ON(!xen_bootmodule); =20 + setup_pagetables(boot_phys_offset); + setup_mm(); =20 /* Parse the ACPI tables for possible boot-time configuration */ --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518322; cv=none; d=zohomail.com; s=zohoarc; b=kTQFvZua9QBrXn96M2o3KDCpcgpOgYp0blS/252fWFYmW8UqLB/4MvGlpRIRtg/Hvacg9EC9bzfgrRNKICongLdvFr644y785VwZgpYhxmuixqakJ9M1Kpv60TRIx5OeL2A8NTeld1sv2iQCrQJuhoKh+cvPwx3Ocs7r+x2jmHw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518322; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=R2LVk/C6PM+sPeAhFE7BODzOKtHorLP8l5p0UCGQm08=; b=R0bjhuGL6cb5Hvf73UegmiE9kPJ9F8k1GXSPRBRjIFq4UvQ0p70DUZnPQuYEnZWrJBBNVUtJBCwCUQSiXixUWOzacmT929R8f7QrWiGczmzsOglT887gfoDKHk+/cCUXT9NHtevYlTc0rE7OLB/KTYlpS29GLqpuL7S6XQl6RrI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518322921757.2769672490264; Fri, 26 Aug 2022 05:52:02 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393763.632982 (Exim 4.92) (envelope-from ) id 1oRYoM-00049L-Ag; Fri, 26 Aug 2022 12:51:46 +0000 Received: by outflank-mailman (output) from mailman id 393763.632982; Fri, 26 Aug 2022 12:51:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoM-00048l-5M; Fri, 26 Aug 2022 12:51:46 +0000 Received: by outflank-mailman (input) for mailman id 393763; Fri, 26 Aug 2022 12:51:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoK-00013M-Th for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:44 +0000 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [2a00:1450:4864:20::633]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d5e4340f-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:43 +0200 (CEST) Received: by mail-ej1-x633.google.com with SMTP id w19so2977774ejc.7 for ; Fri, 26 Aug 2022 05:51:42 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:42 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d5e4340f-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=R2LVk/C6PM+sPeAhFE7BODzOKtHorLP8l5p0UCGQm08=; b=emY69uwkApgKgI11ZNEMUZfM+vPFPj8VxmWLvHqsCDAzj7eG5APZ7dBucJ/tr1t/j1 lXsQgOOwIb7jfg/+lT68jTUaX9AS+t3pD6CgvC0osp6Ij7XsOydAto4TNnZAKblA/nSL ZmEDzGQz08dbpLSgSMd/DdpupS2amd+8elAVUE2Iv+qEUxiCYquAAYLoxb7v066zkMIU jl+C0ngg1198wx/o8i2Rs1B4TXYZQ2C+N29zA+U9R5Et/z6aqzeFcH4kIfKR3j4/iV8n 8fQWFMkHpidqp0UoBwPfURfC8C/o3/SGA8d090aVtD/pbpISHGWqKGvgylnlc5ma0SfB 9KFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=R2LVk/C6PM+sPeAhFE7BODzOKtHorLP8l5p0UCGQm08=; b=sZN3EE8VedAqkJh6lF+Yvc7geT2eIIAXcb9yU/DFAbfCbCCL0AwdPGhJUXhX04OSbb hL922qxz6HN8R11kzatm+/DYqny3ZOwPtVptbbYrRreIOeanMzuPgWgEmVkQRElCNl2X kRn0djCYGCe0bTSWW5DUTfQOtCljgP/ESlXBkndpVNQvqU9xyEI6w0C8IilYI5fHEtok huWHsr2Pva8JYfVdl7Jz6AXf8uq8J0IKsUcgU4CWaBaGbbuACmvi5aCG+fRgtiJ/B/iU GxA/gpUV9qJkLLc5q5oQggPU6WWqPOW8INi1KYM+iRzSSYFczu9ZnD1Wsju57DLCKX1O BhBg== X-Gm-Message-State: ACgBeo0de5z8yD3LyelUcUW75Ni2eKtt/z6f2DLx7dhN0LQDsdg6msW9 FKLvPtQPh8tNJAnip5BOwHvHZo5D6lo8aw== X-Google-Smtp-Source: AA6agR7eTkdARncmnhdOdZxvLR4fOboOR7KBRJv/rWA3iHYAfq7JqmX+pifCMOVdeXbxFzx6X68nIw== X-Received: by 2002:a17:906:fd8d:b0:73a:7424:6c0d with SMTP id xa13-20020a170906fd8d00b0073a74246c0dmr5478138ejb.425.1661518302466; Fri, 26 Aug 2022 05:51:42 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato Subject: [PATCH 10/12] Revert "xen/arm: Remove unused BOOT_RELOC_VIRT_START" Date: Fri, 26 Aug 2022 14:51:09 +0200 Message-Id: <20220826125111.152261-11-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518323645100019 Content-Type: text/plain; charset="utf-8" This reverts commit 0c18fb76323bfb13615b6f13c98767face2d8097. Cache coloring support for Xen needs to relocate Xen code and data in a new colored physical space. The BOOT_RELOC_VIRT_START will be used as the virtu= al base address for a temporary mapping to this new space. --- xen/arch/arm/include/asm/config.h | 4 +++- xen/arch/arm/mm.c | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/include/asm/config.h b/xen/arch/arm/include/asm/c= onfig.h index 2fafb9f228..c8c1095d70 100644 --- a/xen/arch/arm/include/asm/config.h +++ b/xen/arch/arm/include/asm/config.h @@ -77,7 +77,8 @@ * 2M - 4M Xen text, data, bss * 4M - 6M Fixmap: special-purpose 4K mapping slots * 6M - 10M Early boot mapping of FDT - * 10M - 12M Livepatch vmap (if compiled in) + * 10M - 12M Early relocation address (used when relocating Xen) + * and later for livepatch vmap (if compiled in) * * ARM32 layout: * 0 - 12M @@ -113,6 +114,7 @@ #define BOOT_FDT_VIRT_START _AT(vaddr_t,0x00600000) #define BOOT_FDT_VIRT_SIZE _AT(vaddr_t, MB(4)) =20 +#define BOOT_RELOC_VIRT_START _AT(vaddr_t,0x00a00000) #ifdef CONFIG_LIVEPATCH #define LIVEPATCH_VMAP_START _AT(vaddr_t,0x00a00000) #define LIVEPATCH_VMAP_SIZE _AT(vaddr_t, MB(2)) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 1afa02b4af..efc0dd75d1 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -154,6 +154,7 @@ static void __init __maybe_unused build_assertions(void) /* 2MB aligned regions */ BUILD_BUG_ON(XEN_VIRT_START & ~SECOND_MASK); BUILD_BUG_ON(FIXMAP_ADDR(0) & ~SECOND_MASK); + BUILD_BUG_ON(BOOT_RELOC_VIRT_START & ~SECOND_MASK); /* 1GB aligned regions */ #ifdef CONFIG_ARM_32 BUILD_BUG_ON(XENHEAP_VIRT_START & ~FIRST_MASK); --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518330; cv=none; d=zohomail.com; s=zohoarc; b=O+uTt8A/tZ/wBmsu7dYw4FS6z7njAJ/7O7B718SW2UzLw+PoUn5kFH73rOPhqPM0QjeBccxKM7iux7nwQ6CGOHnPl+mbNUPPlIpHgenfNU/qCAY/8ALmEtOaTOCn9irR0ZGkb8iamS3bmWlNQonsWKyPIWQwc5al1fmXyRdFzeY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518330; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dZ3qv2o8Z5R8iaRWPpU2jN0ZDcc6sdXYlyjmn+N7fqM=; b=b6uDOCDnTpAh55ZpjzZWiYrdEEFwO7rXBKEietU5ElLeY9iuBPkIks8cVDnLveclVPX7ymlN8nIb27DzmvA5/fU5qLanjF8IaZRUPGhXuAOEo62h7t/QJB8v6pc+B6HUj1iuN6jMKMoZM23WRMoyt1JBMa7RLMj0mcH45BIPWgQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518330281354.15500975385874; Fri, 26 Aug 2022 05:52:10 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393764.632987 (Exim 4.92) (envelope-from ) id 1oRYoN-0004EI-Aw; Fri, 26 Aug 2022 12:51:47 +0000 Received: by outflank-mailman (output) from mailman id 393764.632987; Fri, 26 Aug 2022 12:51:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoM-0004Bj-My; Fri, 26 Aug 2022 12:51:46 +0000 Received: by outflank-mailman (input) for mailman id 393764; Fri, 26 Aug 2022 12:51:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoL-00013M-HW for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:45 +0000 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [2a00:1450:4864:20::62e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d6cc40dc-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:44 +0200 (CEST) Received: by mail-ej1-x62e.google.com with SMTP id u9so2993938ejy.5 for ; Fri, 26 Aug 2022 05:51:44 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:43 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d6cc40dc-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=dZ3qv2o8Z5R8iaRWPpU2jN0ZDcc6sdXYlyjmn+N7fqM=; b=rmeyh6Filnzt+/RKgIchnVdgT0CM2K8b5Y1fMOcCTG7gdtnfy8xPkinC7mDfEpoVJP QFVDPe1HXyOshMooxclN2PDL8GhGUUDt1gGx0+FwCCkoesgPTHNpuEeUyr/tC07blIAP hmGqJ+hAz43s/2tIDsSBNONjNQ86JYz/y4yFT6TelWvwWkAoC+D9Lfs6fn/jYTZzIvDe GkayLPdo1q9PnCuoeUTAvFfpfq3bndlDWg1YKhx9yCUQWfxuqKju4YYHPlo1dDO6Tlup V2Cyza3aWxtMrgGzAJrud/32d57ueRuKnpUSi1EmsI6zx6NAGXC2ANoL9CkqQjLhBM5j /BRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=dZ3qv2o8Z5R8iaRWPpU2jN0ZDcc6sdXYlyjmn+N7fqM=; b=HeJGGFRIUKYaXdKuiUgOHo6LBrTo/j4AyLTOGnYn1E50/UF2Kj0aC6JC04U0v/EKWJ GJbW2W7jkDAJ1TaHsPVPqwWE/GdZGWW3cu+6DQsTeCKMFhHnQ1/7NPE3bnk4iF5J0RSA dqGMJ5g6Zj/TRzdChF3GwZXfrxcisxnPoc4LbKwtR3/xg2IrTP8JmaULFhdt0WhZqWLd idOe3iFfUln47UYCxC+nZTM4kZT7oyN/kSfZ1MQGDnAT0PQ6gTun6A7O9AjD2AGj+FIP C5GDt3tO1REmQsRaQDwrkpRCWEzRuFuaHqsMBQukXefzoJIB7uQX16PrLwBQepIiA+6V 5tsg== X-Gm-Message-State: ACgBeo1et4XhrnmAILGctgE4rmMITI2eQhYvJVzj1yllvqmA0kArhTkT rrWJ18ttqzU1FR7iRI8dFx2S21h9kFvcsw== X-Google-Smtp-Source: AA6agR5OtsLrD0ECCjgz0yMmObcumL1gAUFhYfipVmPy1FhcCjfV6DVPgweww28OtGAQbO4ZhnuyaQ== X-Received: by 2002:a17:907:6d8c:b0:731:6c60:eced with SMTP id sb12-20020a1709076d8c00b007316c60ecedmr5513987ejc.266.1661518303914; Fri, 26 Aug 2022 05:51:43 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 11/12] xen/arm: add Xen cache colors command line parameter Date: Fri, 26 Aug 2022 14:51:10 +0200 Message-Id: <20220826125111.152261-12-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518331701100001 Content-Type: text/plain; charset="utf-8" This commit adds a new command line parameter to configure Xen cache colors. These colors can be dumped with the cache coloring info debug-key. By default, Xen uses the first color. Benchmarking the VM interrupt response time provides an estimation of LLC usage by Xen's most latency-critical runtime task. Results on Arm Cortex-A53 on Xilinx Zynq UltraScale+ XCZU9EG show that one color, which reserves 64 KiB of L2, is enough to attain best responsiveness. More colors are instead very likely to be needed on processors whose L1 cache is physically-indexed and physically-tagged, such as Cortex-A57. In such cases, coloring applies to L1 also, and there typically are two distinct L1-colors. Therefore, reserving only one color for Xen would senselessly partitions a cache memory that is already private, i.e. underutilize it. The default amount of Xen colors is thus set to one. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- docs/misc/arm/cache-coloring.rst | 8 ++++---- docs/misc/xen-command-line.pandoc | 9 +++++++++ xen/arch/arm/coloring.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 4 deletions(-) diff --git a/docs/misc/arm/cache-coloring.rst b/docs/misc/arm/cache-colorin= g.rst index 5f9132e525..f347245093 100644 --- a/docs/misc/arm/cache-coloring.rst +++ b/docs/misc/arm/cache-coloring.rst @@ -16,8 +16,8 @@ In order to enable and use it, few steps are needed. - If needed, change the amount of memory reserved for the buddy allocator = either from the Xen configuration file, via the CONFIG_BUDDY_ALLOCATOR_SIZE val= ue, or with the command line option. See `Colored allocator and buddy alloca= tor`. -- Assign colors to domains using the `Color selection format`_ (see - `Coloring parameters`_ for more documentation pointers). +- Assign colors to each memory pool (Xen, Dom0/DomUs) using the + `Color selection format`_ for `Coloring parameters`_ configuration. =20 Background ********** @@ -110,8 +110,8 @@ Examples: Coloring parameters ******************* =20 -LLC way size (as previously discussed) and Dom0 colors can be set using the -appropriate command line parameters. See the relevant documentation in +LLC way size (as previously discussed), Xen colors and Dom0 colors can be = set +using the appropriate command line parameters. See the relevant documentat= ion in "docs/misc/xen-command-line.pandoc". =20 DomUs colors can be set either in the xl configuration file (relative diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index 4e85c4dfe4..3b4f7d54a7 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2736,6 +2736,15 @@ In the case that x2apic is in use, this option switc= hes between physical and clustered mode. The default, given no hint from the **FADT**, is cluster mode. =20 +### xen-colors (arm64) +> `=3D List of [ | - ]` + +> Default: `0: the lowermost color` + +Specify Xen color configuration. +Two colors are most likely needed on platforms where private caches are +physically indexed, e.g. the L1 instruction cache of the Arm Cortex-A57. + ### xenheap_megabytes (arm32) > `=3D ` =20 diff --git a/xen/arch/arm/coloring.c b/xen/arch/arm/coloring.c index 3fb86043d1..7ae4095dba 100644 --- a/xen/arch/arm/coloring.c +++ b/xen/arch/arm/coloring.c @@ -31,6 +31,10 @@ #include #include =20 +/* By default Xen uses the lowestmost color */ +#define XEN_DEFAULT_COLOR 0 +#define XEN_DEFAULT_NUM_COLORS 1 + /* Size of a LLC way */ static unsigned int llc_way_size; /* Number of colors available in the LLC */ @@ -42,6 +46,9 @@ static uint64_t addr_col_mask; #define addr_set_color(addr, color) (((addr) & ~addr_col_mask) \ | ((color) << PAGE_SHIFT)) =20 +static unsigned int xen_colors[CONFIG_MAX_CACHE_COLORS]; +static unsigned int xen_num_colors; + static unsigned int dom0_colors[CONFIG_MAX_CACHE_COLORS]; static unsigned int dom0_num_colors; =20 @@ -93,6 +100,12 @@ static int parse_color_config(const char *buf, unsigned= int *colors, =20 size_param("llc-way-size", llc_way_size); =20 +static int __init parse_xen_colors(const char *s) +{ + return parse_color_config(s, xen_colors, &xen_num_colors); +} +custom_param("xen-colors", parse_xen_colors); + static int __init parse_dom0_colors(const char *s) { return parse_color_config(s, dom0_colors, &dom0_num_colors); @@ -187,6 +200,8 @@ static void dump_coloring_info(unsigned char key) printk("LLC way size: %u KiB\n", llc_way_size >> 10); printk("Number of LLC colors supported: %u\n", max_colors); printk("Address color mask: 0x%lx\n", addr_col_mask); + printk("Xen colors: "); + print_colors(xen_colors, xen_num_colors); } =20 bool __init coloring_init(void) @@ -204,6 +219,21 @@ bool __init coloring_init(void) ASSERT((max_colors & (max_colors - 1)) =3D=3D 0); addr_col_mask =3D (max_colors - 1) << PAGE_SHIFT; =20 + if ( !xen_num_colors ) + { + printk(XENLOG_WARNING + "Xen color config not found. Using default color: %u\n", + XEN_DEFAULT_COLOR); + xen_colors[0] =3D XEN_DEFAULT_COLOR; + xen_num_colors =3D XEN_DEFAULT_NUM_COLORS; + } + + if ( !check_colors(xen_colors, xen_num_colors) ) + { + printk(XENLOG_ERR "Bad color config for Xen\n"); + return false; + } + if ( !dom0_num_colors ) { printk(XENLOG_WARNING --=20 2.34.1 From nobody Tue May 21 16:10:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1661518788; cv=none; d=zohomail.com; s=zohoarc; b=mB0kCMkJKXMxCUXl9wzrckxc+vhRosgFT8gG2hdQDnsct4oE5oXdY37iSSp8VWWwukBcG02kIZPB8EQFQcRVrs3mKRCjfDGuOrI2Y8fWhoDPG5pw4akkKtnAFJY/K0+bUf8IwsY8VNoeXoAWGZADoNbo3oNIxAA4THH57UrYgAA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661518788; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MAcCYgLuW2JN1CdLnlsHkZM9+Svr9dDqHhzmUix4b28=; b=NFHRTGcfMOZgNIR45JoAydthf3GE3mRDWtbZK5vMWFiz8EC3re2nrydNvR0VRXp14nRCjUHB1j+7t+d7d/aX43GXjHwOkj8s6Qxx8Kj+BtudRfwkBcKkA7rR+69zafuzz+MqX+YbkZZIZ+v8a6MzSIUljN4bzMNUskgm0xzlr4s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1661518788597473.7785410570192; Fri, 26 Aug 2022 05:59:48 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.393805.633015 (Exim 4.92) (envelope-from ) id 1oRYvo-0000LN-3r; Fri, 26 Aug 2022 12:59:28 +0000 Received: by outflank-mailman (output) from mailman id 393805.633015; Fri, 26 Aug 2022 12:59:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYvn-0000LG-W5; Fri, 26 Aug 2022 12:59:27 +0000 Received: by outflank-mailman (input) for mailman id 393805; Fri, 26 Aug 2022 12:59:26 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oRYoO-00013M-Qu for xen-devel@lists.xenproject.org; Fri, 26 Aug 2022 12:51:49 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d897abc7-253d-11ed-9250-1f966e50362f; Fri, 26 Aug 2022 14:51:47 +0200 (CEST) Received: by mail-ed1-x52e.google.com with SMTP id w20so1966146edd.10 for ; Fri, 26 Aug 2022 05:51:47 -0700 (PDT) Received: from carlo-ubuntu.. (hipert-gw1.mat.unimo.it. [155.185.5.1]) by smtp.gmail.com with ESMTPSA id b18-20020a1709063cb200b006ff0b457cdasm888812ejh.53.2022.08.26.05.51.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Aug 2022 05:51:46 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d897abc7-253d-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=MAcCYgLuW2JN1CdLnlsHkZM9+Svr9dDqHhzmUix4b28=; b=aQvtU6aY3GOOMBSLSs3py6Mtgu1qvnSKmxihgWB4BhPWlA2Mc9lOtyFFXBVlKUgHdx 25/+zuVeVsB1w6YpBKcF14YFoVBX2URFz5L7KzJoHEQt+hEhh+0qTBr7E0J4L1RanJOo lbhmEvQPHGFh/+P4jTjggXFn1ctwJJGUmUHBpu4ndUvGLJzU011J+tJdbwwX7LFcxO5C 10QcecvM/M9cavXSLSokG33KsWIHyJ3jX6i0zO26k+198Of79a4SEE+7gReOmYS1F8/T 5p7nqBo3A4Fp29TIq7FW1tnX1vqp8wS1s/66PHsou2f2C7+7Qkq9y+jiLhqYLHN0zv3a hPMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=MAcCYgLuW2JN1CdLnlsHkZM9+Svr9dDqHhzmUix4b28=; b=NASuXMoa+YbVxGgk05iNhMg+A0C1TnJtbNrjwkn0rt7xdjKF5jSsS+DvE8KeW5VmAh 3Fo8O3duh7kZ0Nt85TE0IUhFNaNl+rZbZdV6kT/wFD0wp6ih27NxzFGe++JSAmrBDasR GHLXmjYCilb0cEsGGJG5K8+k3gTslOr3SQ93WgsYV09HYqsYW4ni8EXiHRiKCnofn7jz mAGtwKNXbf27j1bC24Sb36N+iPO0XzKMKnf3OekSmIIkWvfIsT1c/GVJ1KAnF2+twcwl QZ6I5u+aDhM2dyD2ZezFLD1tQ1AVQqUeZNWgONVLAOpesNlr1g3vBFT9TW7hzu7fTC6s wssg== X-Gm-Message-State: ACgBeo1x1JN4jvXwjoL1aYdLZIyP7fD0mcl6mqODsS97gfJ003QDXv3D Xx+dd/Q3o/y4GwN4EfkAzTHhfFbfD8CIog== X-Google-Smtp-Source: AA6agR6D6V0v6j+wVNZ1pvQb0K+SKpKrJSiV8IaWDMo5bmX4t6jn3eMM9/AeOAPNQO/NvW26sf1qgQ== X-Received: by 2002:aa7:cdcc:0:b0:447:bac0:4c20 with SMTP id h12-20020aa7cdcc000000b00447bac04c20mr5659135edw.183.1661518306779; Fri, 26 Aug 2022 05:51:46 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, jbeulich@suse.com, julien@xen.org, stefano.stabellini@amd.com, wl@xen.org, marco.solieri@unimore.it, andrea.bastoni@minervasys.tech, lucmiccio@gmail.com, Carlo Nonato , Marco Solieri Subject: [PATCH 12/12] xen/arm: add cache coloring support for Xen Date: Fri, 26 Aug 2022 14:51:11 +0200 Message-Id: <20220826125111.152261-13-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220826125111.152261-1-carlo.nonato@minervasys.tech> References: <20220826125111.152261-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @minervasys-tech.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1661518789543100001 Content-Type: text/plain; charset="utf-8" This commit adds the cache coloring support for Xen own physical space. It extends the implementation of setup_pagetables to make use of Xen cache coloring configuration. Page tables construction is essentially the same except for the fact that the physical addresses, in case of cache coloring, are taken from the translation of a new, temporary, virtual space that is physically colored. The temporary mapping is also used to relocate Xen to the new physical space starting at the address taken from the old get_xen_paddr() function which is brought back for the occasion. The temporary mapping is finally converted to a mapping of the "old" (meani= ng the original physical space) Xen code, so that the boot CPU can actually address the variables and functions used by secondary CPUs. This happens when the boot CPU needs to bring up other CPUs (psci.c and smpboot.c) and when the TTBR value is passed to them (init_secondary_pagetables). Finally, since the alternative framework needs to remap the Xen text and inittext sections, this operation must be done in a coloring-aware way. The function __vmap_colored is introduced for that. It actually is really similar to the original __vmap, with the exception that it doesn't take for granted that the physical memory is contiguous. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- xen/arch/arm/alternative.c | 5 ++ xen/arch/arm/coloring.c | 16 ++++ xen/arch/arm/include/asm/coloring.h | 12 +++ xen/arch/arm/include/asm/mm.h | 19 ++++- xen/arch/arm/mm.c | 118 +++++++++++++++++++++++++--- xen/arch/arm/psci.c | 4 +- xen/arch/arm/setup.c | 74 +++++++++++++++-- xen/arch/arm/smpboot.c | 3 +- xen/arch/arm/xen.lds.S | 2 +- xen/common/vmap.c | 25 ++++++ xen/include/xen/vmap.h | 4 + 11 files changed, 258 insertions(+), 24 deletions(-) diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c index f03cd943c6..fcecd3fec8 100644 --- a/xen/arch/arm/alternative.c +++ b/xen/arch/arm/alternative.c @@ -220,8 +220,13 @@ void __init apply_alternatives_all(void) * The text and inittext section are read-only. So re-map Xen to * be able to patch the code. */ +#ifdef CONFIG_CACHE_COLORING + xenmap =3D __vmap_colored(&xen_mfn, 1U << xen_order, 1, PAGE_HYPERVISO= R, + VMAP_DEFAULT); +#else xenmap =3D __vmap(&xen_mfn, 1U << xen_order, 1, 1, PAGE_HYPERVISOR, VMAP_DEFAULT); +#endif /* Re-mapping Xen is not expected to fail during boot. */ BUG_ON(!xenmap); =20 diff --git a/xen/arch/arm/coloring.c b/xen/arch/arm/coloring.c index 7ae4095dba..946b21b826 100644 --- a/xen/arch/arm/coloring.c +++ b/xen/arch/arm/coloring.c @@ -340,6 +340,22 @@ unsigned int get_max_colors(void) return max_colors; } =20 +paddr_t next_xen_colored(paddr_t phys) +{ + unsigned int i, color =3D addr_to_color(phys); + + for( i =3D 0; i < xen_num_colors; i++ ) + { + if ( color =3D=3D xen_colors[i] ) + return phys; + else if ( color < xen_colors[i] ) + return addr_set_color(phys, xen_colors[i]); + } + + /* Jump to next color space (llc_way_size bytes) and use the first col= or */ + return addr_set_color(phys + llc_way_size, xen_colors[0]); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/include/asm/coloring.h b/xen/arch/arm/include/asm= /coloring.h index 0982bc9853..374fef9278 100644 --- a/xen/arch/arm/include/asm/coloring.h +++ b/xen/arch/arm/include/asm/coloring.h @@ -25,10 +25,20 @@ #define __ASM_ARM_COLORING_H__ =20 #include +#include #include =20 #include =20 +/* + * Amount of memory that we need to map in order to color Xen. The value + * depends on the maximum number of available colors of the hardware. The + * memory size is pessimistically calculated assuming only one color is us= ed, + * which means that any pages belonging to any other color has to be skipp= ed. + */ +#define XEN_COLOR_MAP_SIZE \ + ROUNDUP((_end - _start) * get_max_colors(), XEN_PADDR_ALIGN) + struct page_info; =20 bool __init coloring_init(void); @@ -45,4 +55,6 @@ unsigned int page_to_color(struct page_info *pg); =20 unsigned int get_max_colors(void); =20 +paddr_t next_xen_colored(paddr_t phys); + #endif /* !__ASM_ARM_COLORING_H__ */ diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index a59fc3791a..00351ee014 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -192,11 +192,15 @@ extern unsigned long total_pages; #define PDX_GROUP_SHIFT SECOND_SHIFT =20 /* Boot-time pagetable setup */ -extern void setup_pagetables(unsigned long boot_phys_offset); +extern void setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_p= addr); /* Map FDT in boot pagetable */ extern void *early_fdt_map(paddr_t fdt_paddr); /* Remove early mappings */ extern void remove_early_mappings(void); +#ifdef CONFIG_CACHE_COLORING +/* Remove early coloring mappings */ +extern void remove_coloring_mappings(void); +#endif /* Allocate and initialise pagetables for a secondary CPU. Sets init_ttbr = to the * new page table */ extern int init_secondary_pagetables(int cpu); @@ -398,6 +402,19 @@ static inline void page_set_xenheap_gfn(struct page_in= fo *p, gfn_t gfn) } while ( (y =3D cmpxchg(&p->u.inuse.type_info, x, nx)) !=3D x ); } =20 +#ifdef CONFIG_CACHE_COLORING +#define virt_boot_xen(virt)\ + (vaddr_t)(virt - XEN_VIRT_START + BOOT_RELOC_VIRT_START) +#define set_value_for_secondary(var, val) \ + *(typeof(var) *)(virt_boot_xen((vaddr_t)&var)) =3D val; \ + clean_dcache(var); +#else +#define virt_boot_xen(virt) virt +#define set_value_for_secondary(var, val) \ + var =3D val; + clean_dcache(var); +#endif + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index efc0dd75d1..951e1796af 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -33,6 +33,9 @@ =20 #include =20 +#ifdef CONFIG_CACHE_COLORING +#include +#endif #include #include =20 @@ -105,6 +108,9 @@ DEFINE_BOOT_PAGE_TABLE(boot_third); static DEFINE_PAGE_TABLE(xen_pgtable); static DEFINE_PAGE_TABLE(xen_first); #define THIS_CPU_PGTABLE xen_pgtable +#ifdef CONFIG_CACHE_COLORING +static DEFINE_PAGE_TABLE(xen_colored_temp); +#endif #else #define HYP_PT_ROOT_LEVEL 1 /* Per-CPU pagetable pages */ @@ -362,13 +368,6 @@ void flush_page_to_ram(unsigned long mfn, bool sync_ic= ache) invalidate_icache(); } =20 -static inline lpae_t pte_of_xenaddr(vaddr_t va) -{ - paddr_t ma =3D va + phys_offset; - - return mfn_to_xen_entry(maddr_to_mfn(ma), MT_NORMAL); -} - static void __init create_boot_mappings(unsigned long virt_offset, mfn_t base_mfn) { @@ -460,9 +459,79 @@ static void clear_table(void *table) clean_and_invalidate_dcache_va_range(table, PAGE_SIZE); } =20 -/* Boot-time pagetable setup. - * Changes here may need matching changes in head.S */ -void __init setup_pagetables(unsigned long boot_phys_offset) +#ifdef CONFIG_CACHE_COLORING +/* + * Translate a Xen (.text) virtual address to the colored physical one + * depending on the hypervisor configuration. + * N.B: this function must be used only when migrating from non colored to + * colored pagetables since it assumes to have the temporary mappings crea= ted + * during setup_pagetables that starts from BOOT_RELOC_VIRT_START. + * After the migration we have to use virt_to_maddr. + */ +static paddr_t virt_to_maddr_colored(vaddr_t virt) +{ + unsigned int va_offset; + + va_offset =3D virt - XEN_VIRT_START; + return __pa(BOOT_RELOC_VIRT_START + va_offset); +} + +static void __init create_coloring_temp_mappings(paddr_t xen_paddr) +{ + lpae_t pte; + unsigned int i; + + for ( i =3D 0; i < (_end - _start) / PAGE_SIZE; i++ ) + { + xen_paddr =3D next_xen_colored(xen_paddr); + pte =3D mfn_to_xen_entry(maddr_to_mfn(xen_paddr), MT_NORMAL); + pte.pt.table =3D 1; /* level 3 mappings always have this bit set */ + xen_colored_temp[i] =3D pte; + xen_paddr +=3D PAGE_SIZE; + } + + pte =3D mfn_to_xen_entry(virt_to_mfn(xen_colored_temp), MT_NORMAL); + pte.pt.table =3D 1; + write_pte(&boot_second[second_table_offset(BOOT_RELOC_VIRT_START)], pt= e); +} + +void __init remove_coloring_mappings(void) +{ + int rc; + + /* destroy the _PAGE_BLOCK mapping */ + rc =3D modify_xen_mappings(BOOT_RELOC_VIRT_START, + BOOT_RELOC_VIRT_START + SZ_2M, + _PAGE_BLOCK); + BUG_ON(rc); +} +#endif /* !CONFIG_CACHE_COLORING */ + +static inline lpae_t pte_of_xenaddr(vaddr_t va) +{ +#ifdef CONFIG_CACHE_COLORING + paddr_t ma =3D virt_to_maddr_colored(va); +#else + paddr_t ma =3D va + phys_offset; +#endif + + return mfn_to_xen_entry(maddr_to_mfn(ma), MT_NORMAL); +} + +/* + * Boot-time pagetable setup with coloring support + * Changes here may need matching changes in head.S + * + * The coloring support consists of: + * - Create a temporary colored mapping that conforms to Xen color selecti= on. + * - pte_of_xenaddr takes care of translating the virtual addresses to the + * new colored physical space and the returns the pte, so that the page = table + * initialization can remain the same. + * - Copy Xen to the new colored physical space by exploiting the temporary + * mapping. + * - Update TTBR0_EL2 with the new root page table address. + */ +void __init setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_p= addr) { uint64_t ttbr; lpae_t pte, *p; @@ -470,6 +539,10 @@ void __init setup_pagetables(unsigned long boot_phys_o= ffset) =20 phys_offset =3D boot_phys_offset; =20 +#ifdef CONFIG_CACHE_COLORING + create_coloring_temp_mappings(xen_paddr); +#endif + #ifdef CONFIG_ARM_64 p =3D (void *) xen_pgtable; p[0] =3D pte_of_xenaddr((uintptr_t)xen_first); @@ -522,7 +595,14 @@ void __init setup_pagetables(unsigned long boot_phys_o= ffset) pte =3D boot_second[second_table_offset(BOOT_FDT_VIRT_START + SZ_2M)]; xen_second[second_table_offset(BOOT_FDT_VIRT_START + SZ_2M)] =3D pte; =20 -#ifdef CONFIG_ARM_64 +#ifdef CONFIG_CACHE_COLORING + /* Copy Xen to the new location */ + memcpy((void *)BOOT_RELOC_VIRT_START, (const void *)XEN_VIRT_START, + (_end - _start)); + clean_dcache_va_range((void *)BOOT_RELOC_VIRT_START, (_end - _start)); + + ttbr =3D virt_to_maddr_colored((vaddr_t)xen_pgtable); +#elif CONFIG_ARM_64 ttbr =3D (uintptr_t) xen_pgtable + phys_offset; #else ttbr =3D (uintptr_t) cpu0_pgtable + phys_offset; @@ -530,6 +610,18 @@ void __init setup_pagetables(unsigned long boot_phys_o= ffset) =20 switch_ttbr(ttbr); =20 +#ifdef CONFIG_CACHE_COLORING + /* + * Keep original Xen memory mapped because secondary CPUs still point = to it + * and a few variables needs to be accessed by the master CPU in order= to + * let them boot. This mapping will also replace the one created at the + * beginning of setup_pagetables. + */ + map_pages_to_xen(BOOT_RELOC_VIRT_START, + maddr_to_mfn(XEN_VIRT_START + phys_offset), + SZ_2M >> PAGE_SHIFT, PAGE_HYPERVISOR_RW | _PAGE_BLOCK= ); +#endif + xen_pt_enforce_wnx(); =20 #ifdef CONFIG_ARM_32 @@ -559,8 +651,8 @@ int init_secondary_pagetables(int cpu) =20 /* Set init_ttbr for this CPU coming up. All CPus share a single setof * pagetables, but rewrite it each time for consistency with 32 bit. */ - init_ttbr =3D (uintptr_t) xen_pgtable + phys_offset; - clean_dcache(init_ttbr); + set_value_for_secondary(init_ttbr, virt_to_maddr(xen_pgtable)); + return 0; } #else diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index 0c90c2305c..d25325d28f 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -49,8 +49,8 @@ int call_psci_cpu_on(int cpu) { struct arm_smccc_res res; =20 - arm_smccc_smc(psci_cpu_on_nr, cpu_logical_map(cpu), __pa(init_secondar= y), - &res); + arm_smccc_smc(psci_cpu_on_nr, cpu_logical_map(cpu), + __pa(virt_boot_xen((vaddr_t)init_secondary)), &res); =20 return PSCI_RET(res); } diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index bdfc05bf61..4917ac84ab 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -453,7 +453,7 @@ static void * __init relocate_fdt(paddr_t dtb_paddr, si= ze_t dtb_size) return fdt; } =20 -#ifdef CONFIG_ARM_32 +#if defined (CONFIG_ARM_32) || (CONFIG_CACHE_COLORING) /* * Returns the end address of the highest region in the range s..e * with required size and alignment that does not conflict with the @@ -581,6 +581,60 @@ static paddr_t __init next_module(paddr_t s, paddr_t *= end) return lowest; } =20 +#ifdef CONFIG_CACHE_COLORING +/** + * get_xen_paddr - get physical address to relocate Xen to + * + * Xen is relocated to as near to the top of RAM as possible and + * aligned to a XEN_PADDR_ALIGN boundary. + */ +static paddr_t __init get_xen_paddr(uint32_t xen_size) +{ + struct meminfo *mi =3D &bootinfo.mem; + paddr_t min_size; + paddr_t paddr =3D 0; + int i; + + min_size =3D (xen_size + (XEN_PADDR_ALIGN-1)) & ~(XEN_PADDR_ALIGN-1); + + /* Find the highest bank with enough space. */ + for ( i =3D 0; i < mi->nr_banks; i++ ) + { + const struct membank *bank =3D &mi->bank[i]; + paddr_t s, e; + + if ( bank->size >=3D min_size ) + { + e =3D consider_modules(bank->start, bank->start + bank->size, + min_size, XEN_PADDR_ALIGN, 0); + if ( !e ) + continue; + +#ifdef CONFIG_ARM_32 + /* Xen must be under 4GB */ + if ( e > 0x100000000ULL ) + e =3D 0x100000000ULL; + if ( e < bank->start ) + continue; +#endif + + s =3D e - min_size; + + if ( s > paddr ) + paddr =3D s; + } + } + + if ( !paddr ) + panic("Not enough memory to relocate Xen\n"); + + printk("Placing Xen at 0x%"PRIpaddr"-0x%"PRIpaddr"\n", + paddr, paddr + min_size); + + return paddr; +} +#endif + static void __init init_pdx(void) { paddr_t bank_start, bank_size, bank_end; @@ -900,6 +954,8 @@ void __init start_xen(unsigned long boot_phys_offset, struct bootmodule *xen_bootmodule; struct domain *d; int rc, i; + paddr_t xen_paddr =3D (paddr_t)(uintptr_t)(_start + boot_phys_offset); + paddr_t xen_size =3D (paddr_t)(uintptr_t)(_end - _start + 1); =20 dcache_line_bytes =3D read_dcache_line_bytes(); =20 @@ -928,15 +984,15 @@ void __init start_xen(unsigned long boot_phys_offset, #ifdef CONFIG_CACHE_COLORING if ( !coloring_init() ) panic("Xen Coloring support: setup failed\n"); + xen_size =3D XEN_COLOR_MAP_SIZE; + xen_paddr =3D get_xen_paddr((uint32_t)xen_size); #endif =20 /* Register Xen's load address as a boot module. */ - xen_bootmodule =3D add_boot_module(BOOTMOD_XEN, - (paddr_t)(uintptr_t)(_start + boot_phys_offse= t), - (paddr_t)(uintptr_t)(_end - _start + 1), fals= e); + xen_bootmodule =3D add_boot_module(BOOTMOD_XEN, xen_paddr, xen_size, f= alse); BUG_ON(!xen_bootmodule); =20 - setup_pagetables(boot_phys_offset); + setup_pagetables(boot_phys_offset, xen_paddr); =20 setup_mm(); =20 @@ -1052,6 +1108,14 @@ void __init start_xen(unsigned long boot_phys_offset, =20 setup_virt_paging(); =20 +#ifdef CONFIG_CACHE_COLORING + /* + * The removal is done earlier than discard_initial_modules beacuse the + * livepatch init uses a virtual address equal to BOOT_RELOC_VIRT_STAR= T. + * Remove coloring mappings to expose a clear state to the livepatch m= odule. + */ + remove_coloring_mappings(); +#endif do_initcalls(); =20 /* diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index f7bda3a18b..e7166ad79b 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -470,8 +470,7 @@ int __cpu_up(unsigned int cpu) init_data.cpuid =3D cpu; =20 /* Open the gate for this CPU */ - smp_up_cpu =3D cpu_logical_map(cpu); - clean_dcache(smp_up_cpu); + set_value_for_secondary(smp_up_cpu, cpu_logical_map(cpu)); =20 rc =3D arch_cpu_up(cpu); =20 diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S index 1e986e211f..4cdead1c4f 100644 --- a/xen/arch/arm/xen.lds.S +++ b/xen/arch/arm/xen.lds.S @@ -203,7 +203,7 @@ SECTIONS . =3D ALIGN(POINTER_ALIGN); __bss_end =3D .; } :text - _end =3D . ; + _end =3D ALIGN(PAGE_SIZE); =20 /* Section for the device tree blob (if any). */ .dtb : { *(.dtb) } :text diff --git a/xen/common/vmap.c b/xen/common/vmap.c index 4fd6b3067e..58818ddfe1 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -8,6 +8,9 @@ #include #include #include +#ifdef CONFIG_CACHE_COLORING +#include +#endif =20 static DEFINE_SPINLOCK(vm_lock); static void *__read_mostly vm_base[VMAP_REGION_NR]; @@ -218,6 +221,28 @@ void *__vmap(const mfn_t *mfn, unsigned int granularit= y, return va; } =20 +#ifdef CONFIG_CACHE_COLORING +void * __vmap_colored(const mfn_t *mfn, unsigned int nr, unsigned int alig= n, + unsigned int flags, enum vmap_region type) +{ + void *va =3D vm_alloc(nr, align, type); + unsigned long cur =3D (unsigned long)va; + paddr_t pa =3D mfn_to_maddr(*mfn); + + for ( ; va && nr-- ; cur +=3D PAGE_SIZE ) + { + pa =3D next_xen_colored(pa); + if ( map_pages_to_xen(cur, maddr_to_mfn(pa), 1, flags) ) + { + vunmap(va); + return NULL; + } + pa +=3D PAGE_SIZE; + } + return va; +} +#endif + void *vmap(const mfn_t *mfn, unsigned int nr) { return __vmap(mfn, 1, nr, 1, PAGE_HYPERVISOR, VMAP_DEFAULT); diff --git a/xen/include/xen/vmap.h b/xen/include/xen/vmap.h index b0f7632e89..9bdc9db70b 100644 --- a/xen/include/xen/vmap.h +++ b/xen/include/xen/vmap.h @@ -14,6 +14,10 @@ void vm_init_type(enum vmap_region type, void *start, vo= id *end); =20 void *__vmap(const mfn_t *mfn, unsigned int granularity, unsigned int nr, unsigned int align, unsigned int flags, enum vmap_region); +#ifdef CONFIG_CACHE_COLORING +void *__vmap_colored(const mfn_t *mfn, unsigned int nr, unsigned int align, + unsigned int flags, enum vmap_region); +#endif void *vmap(const mfn_t *mfn, unsigned int nr); void vunmap(const void *); =20 --=20 2.34.1