From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118661; cv=none; d=zohomail.com; s=zohoarc; b=fG6jamKKGchBG7JIe5Taekeso6mk+QotnQZjso+nfbHfJ+Kh4Hb052yNj0JEG3Gi6y7IDDu0hAR6iA12JNBK7vLHqmX0PduzP1uOESQYHxzMCKvqNDJY922BIP8/KiayFTIozLJrMlhbMx+lnenr0gwDhmnzrO2QwGmhVLPIMbg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118661; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QtM/uNwkuYbgus7NQ6D0a7urDOwmeq1CTr/If2lDLg8=; b=ArfvnUQhC36DDNVonx7IilMwmIG8eje+DQek6zeZ8waj2t41CZF/V55KaHjIBxE/1U6uBUx2j+BDLpuo4qmRy599xwR/p0PFQ7MyCJSXcy0mmn7QO8ci3rU4hPg5iqznvpPYMo0Sl/RCCQDbn6JKPsxWcpJ97vfzupOEpeGI+OI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118661132576.2415217052417; Thu, 27 May 2021 05:31:01 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133082.248140 (Exim 4.92) (envelope-from ) id 1lmFA0-000264-Uq; Thu, 27 May 2021 12:30:48 +0000 Received: by outflank-mailman (output) from mailman id 133082.248140; Thu, 27 May 2021 12:30:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFA0-00025x-Rp; Thu, 27 May 2021 12:30:48 +0000 Received: by outflank-mailman (input) for mailman id 133082; Thu, 27 May 2021 12:30:48 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFA0-00025q-8f for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:30:48 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 76fa961c-7bdd-4e98-88bb-af7727108a6c; Thu, 27 May 2021 12:30:46 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id F071C1FD2E; Thu, 27 May 2021 12:30:45 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id D3B6311A98; Thu, 27 May 2021 12:30:45 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 76fa961c-7bdd-4e98-88bb-af7727108a6c DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118645; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QtM/uNwkuYbgus7NQ6D0a7urDOwmeq1CTr/If2lDLg8=; b=BwJjBdlen2P96tXJpurtSF9hnf4oafFPaJ3hQfyOo4MQsvSQ2jyNgRSoCj6Iah4dBMarmZ 8Zi9Mc1bu/lWclw9Fs80aUVW/FuxFPqyv8IGGafuzN9526Rcq+FDJGjRGzWKhPFZA/3h7h QFj/W0iFWGVH1R3LKwRYgQuCd9DqSPM= Subject: [PATCH v2 01/12] x86: introduce ioremap_wc() From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: <20abac99-609c-f4f6-1242-c79919f4c317@suse.com> Date: Thu, 27 May 2021 14:30:41 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" In order for a to-be-introduced ERMS form of memcpy() to not regress boot performance on certain systems when video output is active, we first need to arrange for avoiding further dependency on firmware setting up MTRRs in a way we can actually further modify. On many systems, due to the continuously growing amounts of installed memory, MTRRs get configured with at least one huge WB range, and with MMIO ranges below 4Gb then forced to UC via overlapping MTRRs. mtrr_add(), as it is today, can't deal with such a setup. Hence on such systems we presently leave the frame buffer mapped UC, leading to significantly reduced performance when using REP STOSB / REP MOVSB. On post-PentiumII hardware (i.e. any that's capable of running 64-bit code), an effective memory type of WC can be achieved without MTRRs, by simply referencing the respective PAT entry from the PTEs. While this will leave the switch to ERMS forms of memset() and memcpy() with largely unchanged performance, the change here on its own improves performance on affected systems quite significantly: Measuring just the individual affected memcpy() invocations yielded a speedup by a factor of over 250 on my initial (Skylake) test system. memset() isn't getting improved by as much there, but still by a factor of about 20. While adding {__,}PAGE_HYPERVISOR_WC, also add {__,}PAGE_HYPERVISOR_WT to, at the very least, make clear what PTE flags this memory type uses. Signed-off-by: Jan Beulich --- v2: Mark ioremap_wc() __init. --- TBD: If the VGA range is WC in the fixed range MTRRs, reusing the low 1st Mb mapping (like ioremap() does) would be an option. --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5881,6 +5881,20 @@ void __iomem *ioremap(paddr_t pa, size_t return (void __force __iomem *)va; } =20 +void __iomem *__init ioremap_wc(paddr_t pa, size_t len) +{ + mfn_t mfn =3D _mfn(PFN_DOWN(pa)); + unsigned int offs =3D pa & (PAGE_SIZE - 1); + unsigned int nr =3D PFN_UP(offs + len); + void *va; + + WARN_ON(page_is_ram_type(mfn_x(mfn), RAM_TYPE_CONVENTIONAL)); + + va =3D __vmap(&mfn, nr, 1, 1, PAGE_HYPERVISOR_WC, VMAP_DEFAULT); + + return (void __force __iomem *)(va + offs); +} + int create_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg) --- a/xen/drivers/video/vesa.c +++ b/xen/drivers/video/vesa.c @@ -9,9 +9,9 @@ #include #include #include +#include #include #include -#include #include "font.h" #include "lfb.h" =20 @@ -103,7 +103,7 @@ void __init vesa_init(void) lfbp.text_columns =3D vlfb_info.width / font->width; lfbp.text_rows =3D vlfb_info.height / font->height; =20 - lfbp.lfb =3D lfb =3D ioremap(lfb_base(), vram_remap); + lfbp.lfb =3D lfb =3D ioremap_wc(lfb_base(), vram_remap); if ( !lfb ) return; =20 @@ -179,8 +179,7 @@ void __init vesa_mtrr_init(void) =20 static void lfb_flush(void) { - if ( vesa_mtrr =3D=3D 3 ) - __asm__ __volatile__ ("sfence" : : : "memory"); + __asm__ __volatile__ ("sfence" : : : "memory"); } =20 void __init vesa_endboot(bool_t keep) --- a/xen/drivers/video/vga.c +++ b/xen/drivers/video/vga.c @@ -79,7 +79,7 @@ void __init video_init(void) { case XEN_VGATYPE_TEXT_MODE_3: if ( page_is_ram_type(paddr_to_pfn(0xB8000), RAM_TYPE_CONVENTIONAL= ) || - ((video =3D ioremap(0xB8000, 0x8000)) =3D=3D NULL) ) + ((video =3D ioremap_wc(0xB8000, 0x8000)) =3D=3D NULL) ) return; outw(0x200a, 0x3d4); /* disable cursor */ columns =3D vga_console_info.u.text_mode_3.columns; @@ -164,7 +164,11 @@ void __init video_endboot(void) { case XEN_VGATYPE_TEXT_MODE_3: if ( !vgacon_keep ) + { memset(video, 0, columns * lines * 2); + iounmap(video); + video =3D ZERO_BLOCK_PTR; + } break; case XEN_VGATYPE_VESA_LFB: case XEN_VGATYPE_EFI_LFB: --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -615,6 +615,8 @@ void destroy_perdomain_mapping(struct do unsigned int nr); void free_perdomain_mappings(struct domain *); =20 +void __iomem *ioremap_wc(paddr_t, size_t); + extern int memory_add(unsigned long spfn, unsigned long epfn, unsigned int= pxm); =20 void domain_set_alloc_bitsize(struct domain *d); --- a/xen/include/asm-x86/page.h +++ b/xen/include/asm-x86/page.h @@ -349,8 +349,10 @@ void efi_update_l4_pgtable(unsigned int #define __PAGE_HYPERVISOR_RX (_PAGE_PRESENT | _PAGE_ACCESSED) #define __PAGE_HYPERVISOR (__PAGE_HYPERVISOR_RX | \ _PAGE_DIRTY | _PAGE_RW) +#define __PAGE_HYPERVISOR_WT (__PAGE_HYPERVISOR | _PAGE_PWT) #define __PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR | _PAGE_PCD) #define __PAGE_HYPERVISOR_UC (__PAGE_HYPERVISOR | _PAGE_PCD | _PAGE_P= WT) +#define __PAGE_HYPERVISOR_WC (__PAGE_HYPERVISOR | _PAGE_PAT) #define __PAGE_HYPERVISOR_SHSTK (__PAGE_HYPERVISOR_RO | _PAGE_DIRTY) =20 #define MAP_SMALL_PAGES _PAGE_AVAIL0 /* don't use superpages mappings */ --- a/xen/include/asm-x86/x86_64/page.h +++ b/xen/include/asm-x86/x86_64/page.h @@ -154,6 +154,10 @@ static inline intpte_t put_pte_flags(uns _PAGE_GLOBAL | _PAGE_NX) #define PAGE_HYPERVISOR_UC (__PAGE_HYPERVISOR_UC | \ _PAGE_GLOBAL | _PAGE_NX) +#define PAGE_HYPERVISOR_WC (__PAGE_HYPERVISOR_WC | \ + _PAGE_GLOBAL | _PAGE_NX) +#define PAGE_HYPERVISOR_WT (__PAGE_HYPERVISOR_WT | \ + _PAGE_GLOBAL | _PAGE_NX) =20 #endif /* __X86_64_PAGE_H__ */ =20 From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118684; cv=none; d=zohomail.com; s=zohoarc; b=gVHZ9SYJmGETsYi2g+L3m/mdip9xuAm5PUmZk9OHA66aUOp2BqlIPAB8eGkejCTOXewsFRddwEkBKtvMIPLDf+RUQi3hviq0TYagJYoFjdhMcWvdaQYLUOGLhzdrnJkU0ANHT5yMGnfgD8xPIX/Do9E7MKiNjk6Q7ULkwr84v9I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118684; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0I4gNzPs7QkUzQaZgNM9qNeznBTvZe1/bKfap9B3u8w=; b=flJnYyOSB8+AVSq1mZwlat0HHNkctm3wxWheLVZTB7VHfnFpw0EmYG0NpngV/YZ5rxEfutztQeaFjK5rVr+u4vNqaVlN8YJ2HGS+kS+7uCvPUTncAOC1axC4/mYQ5pJAT8dI1qLng9p4CkTIEQ5qCuN+TryfexcS0/gdX7DrdyY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118684435148.1240907616941; Thu, 27 May 2021 05:31:24 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133087.248151 (Exim 4.92) (envelope-from ) id 1lmFAN-0002aF-73; Thu, 27 May 2021 12:31:11 +0000 Received: by outflank-mailman (output) from mailman id 133087.248151; Thu, 27 May 2021 12:31:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFAN-0002a8-3R; Thu, 27 May 2021 12:31:11 +0000 Received: by outflank-mailman (input) for mailman id 133087; Thu, 27 May 2021 12:31:10 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFAM-0002a0-7c for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:31:10 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0de3975c-099d-42a7-9c1f-15a18aaeef7d; Thu, 27 May 2021 12:31:09 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6EDFC218DD; Thu, 27 May 2021 12:31:08 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 4435511A98; Thu, 27 May 2021 12:31:08 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0de3975c-099d-42a7-9c1f-15a18aaeef7d DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118668; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0I4gNzPs7QkUzQaZgNM9qNeznBTvZe1/bKfap9B3u8w=; b=j6t6sDTyYXBiRPX9kFPTiT8c3JUIc5bsqpm1+JIVm53FA0U71KrhORmXGelcmk0EqVwdYp nXTtw0qpBqQQLM9Jc3pOPA4F8HKPuJPAiEGiTf2rOJY5wk/JTwP5t90qCucd0YtiMOKLfV bCg7y+ALsbYh4O4yh8IgCiIcty//0Lo= Subject: [PATCH v2 02/12] x86: re-work memset() From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: Date: Thu, 27 May 2021 14:31:08 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Move the function to its own assembly file. Having it in C just for the entire body to be an asm() isn't really helpful. Then have two flavors: A "basic" version using qword steps for the bulk of the operation, and an ERMS version for modern hardware, to be substituted in via alternatives patching. Signed-off-by: Jan Beulich --- We may want to consider branching over the REP STOSQ as well, if the number of qwords turns out to be zero. We may also want to consider using non-REP STOS{L,W,B} for the tail. --- a/xen/arch/x86/Makefile +++ b/xen/arch/x86/Makefile @@ -43,6 +43,7 @@ obj-$(CONFIG_INDIRECT_THUNK) +=3D indirect obj-y +=3D ioport_emulate.o obj-y +=3D irq.o obj-$(CONFIG_KEXEC) +=3D machine_kexec.o +obj-y +=3D memset.o obj-y +=3D mm.o x86_64/mm.o obj-$(CONFIG_HVM) +=3D monitor.o obj-y +=3D mpparse.o --- /dev/null +++ b/xen/arch/x86/memset.S @@ -0,0 +1,31 @@ +#include + +.macro memset + and $7, %edx + shr $3, %rcx + movzbl %sil, %esi + mov $0x0101010101010101, %rax + imul %rsi, %rax + mov %rdi, %rsi + rep stosq + or %edx, %ecx + jz 0f + rep stosb +0: + mov %rsi, %rax + ret +.endm + +.macro memset_erms + mov %esi, %eax + mov %rdi, %rsi + rep stosb + mov %rsi, %rax + ret +.endm + +ENTRY(memset) + mov %rdx, %rcx + ALTERNATIVE memset, memset_erms, X86_FEATURE_ERMS + .type memset, @function + .size memset, . - memset --- a/xen/arch/x86/string.c +++ b/xen/arch/x86/string.c @@ -22,19 +22,6 @@ void *(memcpy)(void *dest, const void *s return dest; } =20 -void *(memset)(void *s, int c, size_t n) -{ - long d0, d1; - - asm volatile ( - "rep stosb" - : "=3D&c" (d0), "=3D&D" (d1) - : "a" (c), "1" (s), "0" (n) - : "memory"); - - return s; -} - void *(memmove)(void *dest, const void *src, size_t n) { long d0, d1, d2; From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118699; cv=none; d=zohomail.com; s=zohoarc; b=PykQL5CAzJid37ipHl7DoxkUTyZTigLtUbNZF7x6oGwwVCdQTG9GW9U4OGtgIdq1viJS5kOpZ6bIyiieZ3PLx/qHR16Fj0NnKtLNqJXfIqSg9EwzKxNanJtDRnU8AW9EuhQe9Vrm9i5VtSMZxXpqSWxAnUalxVN9QVp11NhMMGI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118699; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8iVlQanP290BLy02LDzP89TcI1dKH7MaEcZKdyou000=; b=l9W5SYCVNhFtXVlN5bMO8FnWsF60GdWWE5l/0Cz27r1/QwarrV8JbbrwIjxcjTYwzw28GOjGCzSL0vO0D3QYWl9U8l3XzwvEgaxmZgCblxLVyCwjvP3kNea00qupujKy90enqiwMmdk7W2BzJjGuLujrHOq5O3QDT+Sy89hLNr4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 162211869971967.51131327261214; Thu, 27 May 2021 05:31:39 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133092.248161 (Exim 4.92) (envelope-from ) id 1lmFAf-0003AQ-EC; Thu, 27 May 2021 12:31:29 +0000 Received: by outflank-mailman (output) from mailman id 133092.248161; Thu, 27 May 2021 12:31:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFAf-0003AJ-BH; Thu, 27 May 2021 12:31:29 +0000 Received: by outflank-mailman (input) for mailman id 133092; Thu, 27 May 2021 12:31:28 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFAe-00030D-2g for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:31:28 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id bae8eeac-2622-49e8-a362-505f87bbb378; Thu, 27 May 2021 12:31:26 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7BFA8218DD; Thu, 27 May 2021 12:31:25 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 587CF11A98; Thu, 27 May 2021 12:31:25 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: bae8eeac-2622-49e8-a362-505f87bbb378 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118685; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8iVlQanP290BLy02LDzP89TcI1dKH7MaEcZKdyou000=; b=aOK0zKMm3VR+7w75/vqdiEKsBmWa3gLhkabEA/9yX93rxodzhVfg622fFN8c6avMoqKQG3 MzksE9EVTGJAYdyoxdB1hhdCrZR5tFU5qjyXwcGg0pFkvVstG+jbVh9wC35S42GolIqJ7E matnOXfWokSWyjJDYatwFvPi36yMnyo= Subject: [PATCH v2 03/12] x86: re-work memcpy() From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: Date: Thu, 27 May 2021 14:31:25 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Move the function to its own assembly file. Having it in C just for the entire body to be an asm() isn't really helpful. Then have two flavors: A "basic" version using qword steps for the bulk of the operation, and an ERMS version for modern hardware, to be substituted in via alternatives patching. Alternatives patching, however, requires an extra precaution: It uses memcpy() itself, and hence the function may patch itself. Luckily the patched-in code only replaces the prolog of the original function. Make sure this remains this way. Additionally alternatives patching, while supposedly safe via enforcing a control flow change when modifying already prefetched code, may not really be. Afaict a request is pending to drop the first of the two options in the SDM's "Handling Self- and Cross-Modifying Code" section. Insert a serializing instruction there. To avoid having to introduce a local variable, also switch text_poke() to return void: Neither of its callers cares about the returned value. Signed-off-by: Jan Beulich --- We may want to consider branching over the REP MOVSQ as well, if the number of qwords turns out to be zero. We may also want to consider using non-REP MOVS{L,W,B} for the tail. --- a/xen/arch/x86/Makefile +++ b/xen/arch/x86/Makefile @@ -43,6 +43,7 @@ obj-$(CONFIG_INDIRECT_THUNK) +=3D indirect obj-y +=3D ioport_emulate.o obj-y +=3D irq.o obj-$(CONFIG_KEXEC) +=3D machine_kexec.o +obj-y +=3D memcpy.o obj-y +=3D memset.o obj-y +=3D mm.o x86_64/mm.o obj-$(CONFIG_HVM) +=3D monitor.o --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -164,12 +164,14 @@ void init_or_livepatch add_nops(void *in * executing. * * "noinline" to cause control flow change and thus invalidate I$ and - * cause refetch after modification. + * cause refetch after modification. While the SDM continues to suggest t= his + * is sufficient, it may not be - issue a serializing insn afterwards as w= ell. */ -static void *init_or_livepatch noinline +static void init_or_livepatch noinline text_poke(void *addr, const void *opcode, size_t len) { - return memcpy(addr, opcode, len); + memcpy(addr, opcode, len); + cpuid_eax(0); } =20 /* --- /dev/null +++ b/xen/arch/x86/memcpy.S @@ -0,0 +1,21 @@ +#include + +ENTRY(memcpy) + mov %rdx, %rcx + mov %rdi, %rax + /* + * We need to be careful here: memcpy() is involved in alternatives + * patching, so the code doing the actual copying (i.e. past setti= ng + * up registers) may not be subject to patching (unless further + * precautions were taken). + */ + ALTERNATIVE "and $7, %edx; shr $3, %rcx", \ + "rep movsb; ret", X86_FEATURE_ERMS + rep movsq + or %edx, %ecx + jz 1f + rep movsb +1: + ret + .type memcpy, @function + .size memcpy, . - memcpy --- a/xen/arch/x86/string.c +++ b/xen/arch/x86/string.c @@ -7,21 +7,6 @@ =20 #include =20 -void *(memcpy)(void *dest, const void *src, size_t n) -{ - long d0, d1, d2; - - asm volatile ( - " rep ; movs"__OS" ; " - " mov %k4,%k3 ; " - " rep ; movsb " - : "=3D&c" (d0), "=3D&D" (d1), "=3D&S" (d2) - : "0" (n/BYTES_PER_LONG), "r" (n%BYTES_PER_LONG), "1" (dest), "2" = (src) - : "memory" ); - - return dest; -} - void *(memmove)(void *dest, const void *src, size_t n) { long d0, d1, d2; From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118720; cv=none; d=zohomail.com; s=zohoarc; b=mZ5HTM3N6gxnjFhLMoPC/y9gRcOV+gt1nzQtFJVaWCfI5LXWOx19uOzulZKOqNG1DyYF9mFhnq1YLdMTnFXfTWhn6bNXxu1Bdv6DdcfSN7jCRxfzzDicGHGMXuZtzlESLlm6nNCypQoXBeiuBW0FGhb0qIRjqe+1D4pm5iVZfYs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118720; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QcysIdV+/q7pgDThgMz+fBurt337dZex+nTBiQnp9G4=; b=KAa+v0QRwxSsS5DHOroENSMo4ArpTFnFn29C8th6bAY7qoJsBHD9pcHkNn13WNj+CFp1XnvY5n9vL+HKoP426lwiq8qqxMN7u41v6drN8mt3aLJa/hT9UTsNv/35Vyd/uqh21FrVS03a+NPSwWC7bz3u91xU0UvoJ9CJIjzcsDw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 162211872078827.468007986772477; Thu, 27 May 2021 05:32:00 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133097.248173 (Exim 4.92) (envelope-from ) id 1lmFAy-0003jm-Mc; Thu, 27 May 2021 12:31:48 +0000 Received: by outflank-mailman (output) from mailman id 133097.248173; Thu, 27 May 2021 12:31:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFAy-0003jf-Jc; Thu, 27 May 2021 12:31:48 +0000 Received: by outflank-mailman (input) for mailman id 133097; Thu, 27 May 2021 12:31:47 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFAx-0003iF-DD for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:31:47 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 10a0f0d3-1038-4943-a11f-f1120438c9b8; Thu, 27 May 2021 12:31:46 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 8BEE71FD2F; Thu, 27 May 2021 12:31:45 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 66A9411A98; Thu, 27 May 2021 12:31:45 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 10a0f0d3-1038-4943-a11f-f1120438c9b8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118705; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QcysIdV+/q7pgDThgMz+fBurt337dZex+nTBiQnp9G4=; b=TDq+Sg+A4U8B3V4H5lq5PraXQrPLCy0lNJIXEiHPUKfVPqPRcqZlaFl2KgbmRrx5yFDZPm BBunpHeBZj/w92ikA999jOuWKqW7OeEhsy7CNnLVXSH5FaJZsi8KgeS5G3D3lX/UejSyPZ h6VokZGVyR0VW0XIbjsfL+CwwwFEh0Y= Subject: [PATCH v2 04/12] x86: control memset() and memcpy() inlining From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: <654762fa-a7e2-3121-21d5-b992e8428d0e@suse.com> Date: Thu, 27 May 2021 14:31:44 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Stop the compiler from inlining non-trivial memset() and memcpy() (for memset() see e.g. map_vcpu_info() or kimage_load_segments() for examples). This way we even keep the compiler from using REP STOSQ / REP MOVSQ when we'd prefer REP STOSB / REP MOVSB (when ERMS is available). With gcc10 this yields a modest .text size reduction (release build) of around 2k. Unfortunately these options aren't understood by the clang versions I have readily available for testing with; I'm unaware of equivalents. Note also that using cc-option-add is not an option here, or at least I couldn't make things work with it (in case the option was not supported by the compiler): The embedded comma in the option looks to be getting in the way. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich --- v2: New. --- The boundary values are of course up for discussion - I wasn't really certain whether to use 16 or 32; I'd be less certain about using yet larger values. Similarly whether to permit the compiler to emit REP STOSQ / REP MOVSQ for known size, properly aligned blocks is up for discussion. --- a/xen/arch/x86/arch.mk +++ b/xen/arch/x86/arch.mk @@ -51,6 +51,9 @@ CFLAGS-$(CONFIG_INDIRECT_THUNK) +=3D -fno- $(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary= =3D3) export CFLAGS-stack-boundary =20 +CFLAGS +=3D $(call cc-option,$(CC),-mmemcpy-strategy=3Dunrolled_loop:16:no= align$(comma)libcall:-1:noalign) +CFLAGS +=3D $(call cc-option,$(CC),-mmemset-strategy=3Dunrolled_loop:16:no= align$(comma)libcall:-1:noalign) + ifeq ($(CONFIG_UBSAN),y) # Don't enable alignment sanitisation. x86 has efficient unaligned access= es, # and various things (ACPI tables, hypercall pages, stubs, etc) are wont-f= ix. From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118743; cv=none; d=zohomail.com; s=zohoarc; b=KmICu8iknnaCkCvwQR+7fJW5n9Zex2qU67TWHjTGAq+oHZh/74OYUP9xX+faywCNLo2uXpPuUTOClta93JAXrrUD+DtiQpL9chNxC7GnTn45WMIJq/vbyHutf0RX4Z6OejWekoLdXV1n9IeNwI5OeStViOUGB6kBEi9itNrZNbg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118743; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DqA5J7pXICS+Sn1QhqjKtNkkQJN6ELi2AF/cQu5/0QU=; b=EDmNoh/CoXBmx7OTD4dUoq/OOGST68hUtY024Aw22uIwT/uB9CS4VFUMIo7wA1bx4RSGDI7AQ0d8PqqVGq0gUmihYvBjn86X+ke7egZoFq+iIWNQAJwObbOiNUt7SMLXSmtJGDVgUydvPWl9NJw2PaUG+XMvwazA92M7ZFOH8SY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118743079677.5781988194867; Thu, 27 May 2021 05:32:23 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133101.248184 (Exim 4.92) (envelope-from ) id 1lmFBL-0004Kx-0B; Thu, 27 May 2021 12:32:11 +0000 Received: by outflank-mailman (output) from mailman id 133101.248184; Thu, 27 May 2021 12:32:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFBK-0004Kq-SY; Thu, 27 May 2021 12:32:10 +0000 Received: by outflank-mailman (input) for mailman id 133101; Thu, 27 May 2021 12:32:10 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFBK-0004Ka-9F for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:32:10 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 52219094-a289-4d72-8109-0927f6bf7cb5; Thu, 27 May 2021 12:32:09 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 6C6801FD2E; Thu, 27 May 2021 12:32:08 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 4FDF911A98; Thu, 27 May 2021 12:32:08 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 52219094-a289-4d72-8109-0927f6bf7cb5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118728; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DqA5J7pXICS+Sn1QhqjKtNkkQJN6ELi2AF/cQu5/0QU=; b=F+U4y9zJy9LtFGcmwvmfuiJyCOs9OMaGG9NNoJOAP/eVamVH4GC9ohTVPXDVaPxKbuU4nD F4RlAGdhLjOo1YBWLKhrlxUPHuhKokFRpH9VuOsd6VWo0E2BmyJBEj3l9Tjz3JggZd9Mlk Hcm9yh4ILFLbotmGxP/uPJwB+bIQL+w= Subject: [PATCH v2 05/12] x86: introduce "hot" and "cold" page clearing functions From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: <33d79032-d598-2a7c-f361-6d765fd6a54b@suse.com> Date: Thu, 27 May 2021 14:32:04 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" The present clear_page_sse2() is useful in case a page isn't going to get touched again soon, or if we want to limit churn on the caches. Amend it by alternatively using CLZERO, which has been found to be quite a bit faster on Zen2 hardware at least. Note that to use CLZERO, we need to know the cache line size, and hence a feature dependency on CLFLUSH gets introduced. For cases where latency is the most important aspect, or when it is expected that sufficiently large parts of a page will get accessed again soon after the clearing, introduce a "hot" alternative. Again use alternatives patching to select between a "legacy" and an ERMS variant. Don't switch any callers just yet - this will be the subject of subsequent changes. Signed-off-by: Jan Beulich --- v2: New. --- Note: Ankur indicates that for ~L3-size or larger regions MOVNT/CLZERO is better even latency-wise. --- a/xen/arch/x86/clear_page.S +++ b/xen/arch/x86/clear_page.S @@ -1,8 +1,9 @@ .file __FILE__ =20 -#include +#include +#include =20 -ENTRY(clear_page_sse2) + .macro clear_page_sse2 mov $PAGE_SIZE/32, %ecx xor %eax,%eax =20 @@ -16,3 +17,45 @@ ENTRY(clear_page_sse2) =20 sfence ret + .endm + + .macro clear_page_clzero + mov %rdi, %rax + mov $PAGE_SIZE/64, %ecx + .globl clear_page_clzero_post_count +clear_page_clzero_post_count: + +0: clzero + sub $-64, %rax + .globl clear_page_clzero_post_neg_size +clear_page_clzero_post_neg_size: + sub $1, %ecx + jnz 0b + + sfence + ret + .endm + +ENTRY(clear_page_cold) + ALTERNATIVE clear_page_sse2, clear_page_clzero, X86_FEATURE_CLZERO + .type clear_page_cold, @function + .size clear_page_cold, . - clear_page_cold + + .macro clear_page_stosb + mov $PAGE_SIZE, %ecx + xor %eax,%eax + rep stosb + ret + .endm + + .macro clear_page_stosq + mov $PAGE_SIZE/8, %ecx + xor %eax, %eax + rep stosq + ret + .endm + +ENTRY(clear_page_hot) + ALTERNATIVE clear_page_stosq, clear_page_stosb, X86_FEATURE_ERMS + .type clear_page_hot, @function + .size clear_page_hot, . - clear_page_hot --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -56,6 +56,9 @@ static unsigned int forced_caps[NCAPINTS =20 DEFINE_PER_CPU(bool, full_gdt_loaded); =20 +extern uint32_t clear_page_clzero_post_count[]; +extern int8_t clear_page_clzero_post_neg_size[]; + void __init setup_clear_cpu_cap(unsigned int cap) { const uint32_t *dfs; @@ -331,8 +334,38 @@ void __init early_cpu_init(void) =20 edx &=3D ~cleared_caps[cpufeat_word(X86_FEATURE_FPU)]; ecx &=3D ~cleared_caps[cpufeat_word(X86_FEATURE_SSE3)]; - if (edx & cpufeat_mask(X86_FEATURE_CLFLUSH)) - c->x86_cache_alignment =3D ((ebx >> 8) & 0xff) * 8; + if (edx & cpufeat_mask(X86_FEATURE_CLFLUSH)) { + unsigned int size =3D ((ebx >> 8) & 0xff) * 8; + + c->x86_cache_alignment =3D size; + + /* + * Patch in parameters of clear_page_cold()'s CLZERO + * alternative. Note that for now we cap this at 128 bytes. + * Larger cache line sizes would still be dealt with + * correctly, but would cause redundant work done. + */ + if (size > 128) + size =3D 128; + if (size && !(size & (size - 1))) { + /* + * Need to play some games to keep the compiler from + * recognizing the negative array index as being out + * of bounds. The labels in assembler code really are + * _after_ the locations to be patched, so the + * negative index is intentional. + */ + uint32_t *pcount =3D clear_page_clzero_post_count; + int8_t *neg_size =3D clear_page_clzero_post_neg_size; + + OPTIMIZER_HIDE_VAR(pcount); + OPTIMIZER_HIDE_VAR(neg_size); + pcount[-1] =3D PAGE_SIZE / size; + neg_size[-1] =3D -size; + } + else + setup_clear_cpu_cap(X86_FEATURE_CLZERO); + } /* Leaf 0x1 capabilities filled in early for Xen. */ c->x86_capability[cpufeat_word(X86_FEATURE_FPU)] =3D edx; c->x86_capability[cpufeat_word(X86_FEATURE_SSE3)] =3D ecx; --- a/xen/include/asm-x86/asm-defns.h +++ b/xen/include/asm-x86/asm-defns.h @@ -20,6 +20,10 @@ .byte 0x0f, 0x01, 0xdd .endm =20 +.macro clzero + .byte 0x0f, 0x01, 0xfc +.endm + .macro INDIRECT_BRANCH insn:req arg:req /* * Create an indirect branch. insn is one of call/jmp, arg is a single --- a/xen/include/asm-x86/page.h +++ b/xen/include/asm-x86/page.h @@ -232,10 +232,11 @@ typedef struct { u64 pfn; } pagetable_t; #define pagetable_from_paddr(p) pagetable_from_pfn((p)>>PAGE_SHIFT) #define pagetable_null() pagetable_from_pfn(0) =20 -void clear_page_sse2(void *); +void clear_page_hot(void *); +void clear_page_cold(void *); void copy_page_sse2(void *, const void *); =20 -#define clear_page(_p) clear_page_sse2(_p) +#define clear_page(_p) clear_page_cold(_p) #define copy_page(_t, _f) copy_page_sse2(_t, _f) =20 /* Convert between Xen-heap virtual addresses and machine addresses. */ --- a/xen/tools/gen-cpuid.py +++ b/xen/tools/gen-cpuid.py @@ -182,6 +182,10 @@ def crunch_numbers(state): # the first place. APIC: [X2APIC, TSC_DEADLINE, EXTAPIC], =20 + # The CLZERO insn requires a means to determine the cache line siz= e, + # which is tied to the CLFLUSH insn. + CLFLUSH: [CLZERO], + # AMD built MMXExtentions and 3DNow as extentions to MMX. MMX: [MMXEXT, _3DNOW], =20 From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118780; cv=none; d=zohomail.com; s=zohoarc; b=e4b2WWHtV5+2QT9xB5Wt4fYdK9HBFBTbFPMaqxEwvBUdk0sqhuGkkhVGblkrG5rOJdoWEXP5jgzHNlHNSpuB7faDE5aeM+boFyDNNlQbGrst91HjcPRhk7/hEWie0OFrN0EwcMuj/V+BWDjvJaIDXoffHv+Fv99/LWrwirsndVI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118780; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TZmIcIJiPml4pYGzZ8PXew8/Uidlmz9UaHgSxe2VY6k=; b=A9NeT3UagEjLqfsQxGOFjIobvWykw+jd/d+mllA7GG5k5rqji5EBd2KE0aNisD0ZAnWTya+WRSFBV4IBZPub75MB4gCJMgvAihAeDHo2Jp8txleoj3OqkmBzKiC7Rp0laMMXLKAREA+EnyIXmpzHCIwylimZntR+knAsywBaNIc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118779942947.067804027511; Thu, 27 May 2021 05:32:59 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133111.248195 (Exim 4.92) (envelope-from ) id 1lmFBv-00057U-E3; Thu, 27 May 2021 12:32:47 +0000 Received: by outflank-mailman (output) from mailman id 133111.248195; Thu, 27 May 2021 12:32:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFBv-00057H-AF; Thu, 27 May 2021 12:32:47 +0000 Received: by outflank-mailman (input) for mailman id 133111; Thu, 27 May 2021 12:32:46 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFBu-000573-8W for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:32:46 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id cf02cc0e-f7ce-432b-8bba-6a8fe52d17c5; Thu, 27 May 2021 12:32:45 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A0E4A1FD2E; Thu, 27 May 2021 12:32:44 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 6CF1011A98; Thu, 27 May 2021 12:32:44 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cf02cc0e-f7ce-432b-8bba-6a8fe52d17c5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118764; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZmIcIJiPml4pYGzZ8PXew8/Uidlmz9UaHgSxe2VY6k=; b=BFs8RLNJEVICqaTsdEvhcEPVbK+oospJS0U72GU1SqP5Gb+ztJ048ziABRVXRsl4GbWOqn yRGbmCmQtyNNrsrjxMWiKQTVHlMe0K8KiaGia8I5JfoMmNszbu+qnEQe7KJfZQes19jXDp 8vVcOkfp7kUlHg5hP+ohkjgwFDl/PQI= Subject: [PATCH v2 06/12] page-alloc: make scrub_on_page() static From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: <6f24db03-8c96-3a81-a073-657743e8faee@suse.com> Date: Thu, 27 May 2021 14:32:40 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Before starting to alter its properties, restrict the function's visibility. The only external user is mem-paging, which we can accommodate by different means. Also move the function up in its source file, so we won't need to forward-declare it. Constify its parameter at the same time. Signed-off-by: Jan Beulich --- v2: New. --- a/xen/arch/x86/mm/mem_paging.c +++ b/xen/arch/x86/mm/mem_paging.c @@ -316,9 +316,6 @@ static int evict(struct domain *d, gfn_t ret =3D p2m_set_entry(p2m, gfn, INVALID_MFN, PAGE_ORDER_4K, p2m_ram_paged, a); =20 - /* Clear content before returning the page to Xen */ - scrub_one_page(page); - /* Track number of paged gfns */ atomic_inc(&d->paged_pages); =20 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -136,6 +136,7 @@ #include #include #include +#include #include #include #include @@ -757,6 +758,21 @@ static void page_list_add_scrub(struct p #endif #define SCRUB_BYTE_PATTERN (SCRUB_PATTERN & 0xff) =20 +static void scrub_one_page(const struct page_info *pg) +{ + if ( unlikely(pg->count_info & PGC_broken) ) + return; + +#ifndef NDEBUG + /* Avoid callers relying on allocations returning zeroed pages. */ + unmap_domain_page(memset(__map_domain_page(pg), + SCRUB_BYTE_PATTERN, PAGE_SIZE)); +#else + /* For a production build, clear_page() is the fastest way to scrub. */ + clear_domain_page(_mfn(page_to_mfn(pg))); +#endif +} + static void poison_one_page(struct page_info *pg) { #ifdef CONFIG_SCRUB_DEBUG @@ -2431,10 +2447,12 @@ void free_domheap_pages(struct page_info /* * Normally we expect a domain to clear pages before freeing t= hem, * if it cares about the secrecy of their contents. However, a= fter - * a domain has died we assume responsibility for erasure. We = do - * scrub regardless if option scrub_domheap is set. + * a domain has died or if it has mem-paging enabled we assume + * responsibility for erasure. We do scrub regardless if option + * scrub_domheap is set. */ - scrub =3D d->is_dying || scrub_debug || opt_scrub_domheap; + scrub =3D d->is_dying || mem_paging_enabled(d) || + scrub_debug || opt_scrub_domheap; } else { @@ -2519,21 +2537,6 @@ static __init int pagealloc_keyhandler_i __initcall(pagealloc_keyhandler_init); =20 =20 -void scrub_one_page(struct page_info *pg) -{ - if ( unlikely(pg->count_info & PGC_broken) ) - return; - -#ifndef NDEBUG - /* Avoid callers relying on allocations returning zeroed pages. */ - unmap_domain_page(memset(__map_domain_page(pg), - SCRUB_BYTE_PATTERN, PAGE_SIZE)); -#else - /* For a production build, clear_page() is the fastest way to scrub. */ - clear_domain_page(_mfn(page_to_mfn(pg))); -#endif -} - static void dump_heap(unsigned char key) { s_time_t now =3D NOW(); --- a/xen/include/asm-x86/mem_paging.h +++ b/xen/include/asm-x86/mem_paging.h @@ -24,12 +24,6 @@ =20 int mem_paging_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_paging_op_t) arg); =20 -#ifdef CONFIG_MEM_PAGING -# define mem_paging_enabled(d) vm_event_check_ring((d)->vm_event_paging) -#else -# define mem_paging_enabled(d) false -#endif - #endif /*__ASM_X86_MEM_PAGING_H__ */ =20 /* --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -498,8 +498,6 @@ static inline unsigned int get_order_fro return order; } =20 -void scrub_one_page(struct page_info *); - #ifndef arch_free_heap_page #define arch_free_heap_page(d, pg) \ page_list_del(pg, page_to_list(d, pg)) --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -1117,6 +1117,12 @@ static always_inline bool is_iommu_enabl return evaluate_nospec(d->options & XEN_DOMCTL_CDF_iommu); } =20 +#ifdef CONFIG_MEM_PAGING +# define mem_paging_enabled(d) vm_event_check_ring((d)->vm_event_paging) +#else +# define mem_paging_enabled(d) false +#endif + extern bool sched_smt_power_savings; extern bool sched_disable_smt_switching; =20 From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118823; cv=none; d=zohomail.com; s=zohoarc; b=FA6MFEYsx1SyXgY6fu+28FI/SO+/NA/cuEv6PmSqeD7UiJNIJpzwO5TySN5ZyUVYTQVLnFsrH4y9oL589ZZDGHQqR4jixnIskrwQK07quv4d3JBac1S/5ikOHqh4pyqDWU1unQ0m14PsEnHP+4p7GuaoEBKhKLxpZHL6PftYI/I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118823; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Gn1qu776spR85ydugwRCif9Q5VssQAZffidjn3a689E=; b=NtDJ+cUy5VryqirZCYvyW45urTwmaT/FF7b29dAThGoVM2DuNDRZFUOxduVWOpmnnkM/fX9slaUL5wXCxlnn51g3qopDqogrwbRPihfEO8sStVARqASWXt3MC4Qq+IhIfwEmowDy0C9VnTbGT4BhpMBSIdH01N02zs0Fx/RCIoA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118823064836.4517691836633; Thu, 27 May 2021 05:33:43 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133119.248206 (Exim 4.92) (envelope-from ) id 1lmFCc-0005mn-NB; Thu, 27 May 2021 12:33:30 +0000 Received: by outflank-mailman (output) from mailman id 133119.248206; Thu, 27 May 2021 12:33:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFCc-0005mg-Ju; Thu, 27 May 2021 12:33:30 +0000 Received: by outflank-mailman (input) for mailman id 133119; Thu, 27 May 2021 12:33:29 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFCa-0005mO-Ts for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:33:28 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id ffebd46d-b651-435a-8e82-416844cedd35; Thu, 27 May 2021 12:33:27 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E44A8218DD; Thu, 27 May 2021 12:33:26 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id A6A2911A98; Thu, 27 May 2021 12:33:26 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ffebd46d-b651-435a-8e82-416844cedd35 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118806; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gn1qu776spR85ydugwRCif9Q5VssQAZffidjn3a689E=; b=XgUiD8My1RPv5xCY9pnl60B8sVur91hT2Wl3haNQebANPXCAJnJRn3Fv+H/uCjkbbUubvs lYxNcGFH4UEroTCF97IBD01ncn+K2ehjMaHpio+iBahPNfPSKcv8wdB5YFu2X11hAgyMW6 jaUE+7NDqwikjVTCrNubwdRcLtiyu8g= Subject: [PATCH v2 07/12] mm: allow page scrubbing routine(s) to be arch controlled From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: <49c46d4d-4eaa-16a8-ccc8-c873b0b1d092@suse.com> Date: Thu, 27 May 2021 14:33:23 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Especially when dealing with large amounts of memory, memset() may not be very efficient; this can be bad enough that even for debug builds a custom function is warranted. We additionally want to distinguish "hot" and "cold" cases. Keep the default fallback to clear_page_*() in common code; this may want to be revisited down the road. Signed-off-by: Jan Beulich --- v2: New. --- The choice between hot and cold in scrub_one_page()'s callers is certainly up for discussion / improvement. --- a/xen/arch/x86/Makefile +++ b/xen/arch/x86/Makefile @@ -55,6 +55,7 @@ obj-y +=3D percpu.o obj-y +=3D physdev.o obj-$(CONFIG_COMPAT) +=3D x86_64/physdev.o obj-y +=3D psr.o +obj-bin-$(CONFIG_DEBUG) +=3D scrub_page.o obj-y +=3D setup.o obj-y +=3D shutdown.o obj-y +=3D smp.o --- /dev/null +++ b/xen/arch/x86/scrub_page.S @@ -0,0 +1,41 @@ + .file __FILE__ + +#include +#include +#include + +ENTRY(scrub_page_cold) + mov $PAGE_SIZE/32, %ecx + mov $SCRUB_PATTERN, %rax + +0: movnti %rax, (%rdi) + movnti %rax, 8(%rdi) + movnti %rax, 16(%rdi) + movnti %rax, 24(%rdi) + add $32, %rdi + sub $1, %ecx + jnz 0b + + sfence + ret + .type scrub_page_cold, @function + .size scrub_page_cold, . - scrub_page_cold + + .macro scrub_page_stosb + mov $PAGE_SIZE, %ecx + mov $SCRUB_BYTE_PATTERN, %eax + rep stosb + ret + .endm + + .macro scrub_page_stosq + mov $PAGE_SIZE/8, %ecx + mov $SCRUB_PATTERN, %rax + rep stosq + ret + .endm + +ENTRY(scrub_page_hot) + ALTERNATIVE scrub_page_stosq, scrub_page_stosb, X86_FEATURE_ERMS + .type scrub_page_hot, @function + .size scrub_page_hot, . - scrub_page_hot --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -124,6 +124,7 @@ #include #include #include +#include #include #include #include @@ -750,27 +751,31 @@ static void page_list_add_scrub(struct p page_list_add(pg, &heap(node, zone, order)); } =20 -/* SCRUB_PATTERN needs to be a repeating series of bytes. */ -#ifndef NDEBUG -#define SCRUB_PATTERN 0xc2c2c2c2c2c2c2c2ULL -#else -#define SCRUB_PATTERN 0ULL +/* + * While in debug builds we want callers to avoid relying on allocations + * returning zeroed pages, for a production build, clear_page_*() is the + * fastest way to scrub. + */ +#ifndef CONFIG_DEBUG +# undef scrub_page_hot +# define scrub_page_hot clear_page_hot +# undef scrub_page_cold +# define scrub_page_cold clear_page_cold #endif -#define SCRUB_BYTE_PATTERN (SCRUB_PATTERN & 0xff) =20 -static void scrub_one_page(const struct page_info *pg) +static void scrub_one_page(const struct page_info *pg, bool cold) { + void *ptr; + if ( unlikely(pg->count_info & PGC_broken) ) return; =20 -#ifndef NDEBUG - /* Avoid callers relying on allocations returning zeroed pages. */ - unmap_domain_page(memset(__map_domain_page(pg), - SCRUB_BYTE_PATTERN, PAGE_SIZE)); -#else - /* For a production build, clear_page() is the fastest way to scrub. */ - clear_domain_page(_mfn(page_to_mfn(pg))); -#endif + ptr =3D __map_domain_page(pg); + if ( cold ) + scrub_page_cold(ptr); + else + scrub_page_hot(ptr); + unmap_domain_page(ptr); } =20 static void poison_one_page(struct page_info *pg) @@ -1046,12 +1051,14 @@ static struct page_info *alloc_heap_page if ( first_dirty !=3D INVALID_DIRTY_IDX || (scrub_debug && !(memflags & MEMF_no_scrub)) ) { + bool cold =3D d && d !=3D current->domain; + for ( i =3D 0; i < (1U << order); i++ ) { if ( test_and_clear_bit(_PGC_need_scrub, &pg[i].count_info) ) { if ( !(memflags & MEMF_no_scrub) ) - scrub_one_page(&pg[i]); + scrub_one_page(&pg[i], cold); =20 dirty_cnt++; } @@ -1308,7 +1315,7 @@ bool scrub_free_pages(void) { if ( test_bit(_PGC_need_scrub, &pg[i].count_info) ) { - scrub_one_page(&pg[i]); + scrub_one_page(&pg[i], true); /* * We can modify count_info without holding heap * lock since we effectively locked this buddy by @@ -1947,7 +1954,7 @@ static void __init smp_scrub_heap_pages( if ( !mfn_valid(_mfn(mfn)) || !page_state_is(pg, free) ) continue; =20 - scrub_one_page(pg); + scrub_one_page(pg, true); } } =20 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -135,6 +135,12 @@ extern size_t dcache_line_bytes; =20 #define copy_page(dp, sp) memcpy(dp, sp, PAGE_SIZE) =20 +#define clear_page_hot clear_page +#define clear_page_cold clear_page + +#define scrub_page_hot(page) memset(page, SCRUB_BYTE_PATTERN, PAGE_SIZE) +#define scrub_page_cold scrub_page_hot + static inline size_t read_dcache_line_bytes(void) { register_t ctr; --- a/xen/include/asm-x86/page.h +++ b/xen/include/asm-x86/page.h @@ -239,6 +239,11 @@ void copy_page_sse2(void *, const void * #define clear_page(_p) clear_page_cold(_p) #define copy_page(_t, _f) copy_page_sse2(_t, _f) =20 +#ifdef CONFIG_DEBUG +void scrub_page_hot(void *); +void scrub_page_cold(void *); +#endif + /* Convert between Xen-heap virtual addresses and machine addresses. */ #define __pa(x) (virt_to_maddr(x)) #define __va(x) (maddr_to_virt(x)) --- /dev/null +++ b/xen/include/xen/scrub.h @@ -0,0 +1,24 @@ +#ifndef __XEN_SCRUB_H__ +#define __XEN_SCRUB_H__ + +#include + +/* SCRUB_PATTERN needs to be a repeating series of bytes. */ +#ifdef CONFIG_DEBUG +# define SCRUB_PATTERN _AC(0xc2c2c2c2c2c2c2c2,ULL) +#else +# define SCRUB_PATTERN _AC(0,ULL) +#endif +#define SCRUB_BYTE_PATTERN (SCRUB_PATTERN & 0xff) + +#endif /* __XEN_SCRUB_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118864; cv=none; d=zohomail.com; s=zohoarc; b=Bu+I1Bf4ZSBcDqnFn6gf5O7yOXfS6iQGJ2/4rkDNwZHt50Q9zJCFFNdBZ9un+7SM34OAUkdM0VHi1idBp6FPU/zn4+wOmQgZaYY55e8EWYhM0O8up6iho0uidZZ9sFAQlg9dkmwFtrQU3atBpVgoY5DFV2gFMp9qe4adLn13508= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118864; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=T37zxeTBRZgcSqOKYX7WewAeIo59kPVTD7P0XmcfTcQ=; b=ATA0e1uDlTApJojWZJ7E5mk/cQ4Wz9iJQnKQ0CZQejlBCOWxK24migKhKi3FzmvHnBFhuKqjYqTvJN6kqsxco0PBVN9/lTyISuU9kCrYogsyYvfnSGHjh8ueye49qiTrJxnf/gar0/Egndn7vjMskoyU1v57sa6aIG2UeHgtFVA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118864270418.69691452774373; Thu, 27 May 2021 05:34:24 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133128.248217 (Exim 4.92) (envelope-from ) id 1lmFDE-0006Rv-Vy; Thu, 27 May 2021 12:34:08 +0000 Received: by outflank-mailman (output) from mailman id 133128.248217; Thu, 27 May 2021 12:34:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFDE-0006Ro-Su; Thu, 27 May 2021 12:34:08 +0000 Received: by outflank-mailman (input) for mailman id 133128; Thu, 27 May 2021 12:34:07 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFDD-0006Pc-8Z for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:34:07 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 9815c3a5-9e12-4398-ac35-e8ac1248d20e; Thu, 27 May 2021 12:34:06 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C04991FD2E; Thu, 27 May 2021 12:34:05 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id A413711A98; Thu, 27 May 2021 12:34:05 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9815c3a5-9e12-4398-ac35-e8ac1248d20e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118845; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T37zxeTBRZgcSqOKYX7WewAeIo59kPVTD7P0XmcfTcQ=; b=CEkK6A2AMkUaWy8Wpu9K+D3Mohi0H8TEqiEQ6bDFXg/HHxe6WoKkzElzBjjdtn9P0GsmRT BVnYJt9bcBwt6ByA8kPrJNHevC4rwR291N02RXw7NY+C34GRfp23NlbvNPY2TzBzNstqLy dY4pMCyVCkI0DzGydfYGdIuj/QcAqjM= Subject: [PATCH v2 08/12] x86: move .text.kexec From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: Date: Thu, 27 May 2021 14:34:02 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" The source file requests page alignment - avoid a padding hole by placing it right after .text.entry. On average this yields a .text size reduction of 2k. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper --- v2: New. --- a/xen/arch/x86/xen.lds.S +++ b/xen/arch/x86/xen.lds.S @@ -83,10 +83,11 @@ SECTIONS . =3D ALIGN(PAGE_SIZE); _etextentry =3D .; =20 + *(.text.kexec) /* Page aligned in the object file. */ + *(.text.cold) *(.text.unlikely) *(.fixup) - *(.text.kexec) *(.gnu.warning) _etext =3D .; /* End of text section */ } PHDR(text) =3D 0x9090 From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118892; cv=none; d=zohomail.com; s=zohoarc; b=QCeFoBfsGehR86S8niQHySmHOWy8cNW9Z7G027z5TnMjFn7Zl5vSK4Zzho+s2d1PyHtKonQXdmBZ3+hgarHxNZyitJwE63Heu0M3g4gDpVl4jdutlH2oyhQs4iVYhuzh3XtjdX+ZXCDx40ZU0ys9CtW5CVikp3FjZaPmzUstAco= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118892; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=s14UwhbZNQTt7lOAtjre/DnjC1roVy/CdMyBN/BeKjU=; b=PL2/Hq3iCocXK8Q2AIsxLRtTmvlb+t2GnRs03OZVbi8nXN+uCZYK6Vo1M0VmWkbW1CJIbtHNsARupXY+dtf7WFORYHUQZ6e976tHcP9qXqsh83iOz7gWlWKd4l0gVQ/3LzJP2+HBr7ZkfOdHj9Pve9BcyYn/Z/3Cpf8jTVi7cgg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118892199655.785588561914; Thu, 27 May 2021 05:34:52 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133132.248228 (Exim 4.92) (envelope-from ) id 1lmFDh-000718-8R; Thu, 27 May 2021 12:34:37 +0000 Received: by outflank-mailman (output) from mailman id 133132.248228; Thu, 27 May 2021 12:34:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFDh-000711-5S; Thu, 27 May 2021 12:34:37 +0000 Received: by outflank-mailman (input) for mailman id 133132; Thu, 27 May 2021 12:34:36 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFDg-00070k-Qk for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:34:36 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id b16f1a14-e0ad-4a58-9e46-3a98dbaac28e; Thu, 27 May 2021 12:34:36 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7CB041FD2E; Thu, 27 May 2021 12:34:35 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 627DF11A98; Thu, 27 May 2021 12:34:35 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b16f1a14-e0ad-4a58-9e46-3a98dbaac28e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118875; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s14UwhbZNQTt7lOAtjre/DnjC1roVy/CdMyBN/BeKjU=; b=c3NCNrrsRqryCplBfqnWzNqBgavC9PaVCwK7GfhTDGMxDhSLEwXp6j42Re6gisC5CXdV5G nHHia1Z9bK1qnveiJ5pdPiqgXvsbjZmSebQ3ECl+3f0j5BJ3LLeQ69mpVZiNOVCcpbzDz2 fYRLuwaSIHJCo6qg/1b2fWwvYqgCkY0= Subject: [PATCH v2 09/12] video/vesa: unmap frame buffer when relinquishing console From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: <843f440e-039f-ca0b-6ac1-a4d50559d5bc@suse.com> Date: Thu, 27 May 2021 14:34:29 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" There's no point in keeping the VA space occupied when no further output will occur. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- a/xen/drivers/video/lfb.c +++ b/xen/drivers/video/lfb.c @@ -168,4 +168,5 @@ void lfb_free(void) xfree(lfb.lbuf); xfree(lfb.text_buf); xfree(lfb.line_len); + lfb.lfbp.lfb =3D ZERO_BLOCK_PTR; } --- a/xen/drivers/video/vesa.c +++ b/xen/drivers/video/vesa.c @@ -197,5 +197,7 @@ void __init vesa_endboot(bool_t keep) vlfb_info.width * bpp); lfb_flush(); lfb_free(); + iounmap(lfb); + lfb =3D ZERO_BLOCK_PTR; } } From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118944; cv=none; d=zohomail.com; s=zohoarc; b=O9RonYB5TIdEIQF6ZL9TDv0omcwAOlaQ321Yp+3EASM7D297QIEoeD0QZKZETbqKGNvKvjnkZA7a8u+elKdUezSPocpbqzmaFZteiUp1INUpi9yyljnLA9gyBRE3WkyyPwpZh16IK5dwOEHIMUxQykfTSqr7UogxPJGPxS7PtDM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118944; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Gqg+pjRYzvrnv7dX+llGzQkhdttXtK16oe9Z8AMg9RY=; b=mKuo6QCQnmbHdTE/HFVVEyHxY0n4e228De2YNoPiw74ONJLV+rLC18GUrXdF9GyLWs8j8M+UZRQtkcn809kW7mT9uK8253gaj9L3OCJJiJJJYiHjCs2r+AzrZsHFXsYBQTmIjbunBApC5db3FQfVFnHjDErqyZ7oj5soOqRtiHk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118944197722.8501059442261; Thu, 27 May 2021 05:35:44 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133139.248238 (Exim 4.92) (envelope-from ) id 1lmFEW-0007gr-Hh; Thu, 27 May 2021 12:35:28 +0000 Received: by outflank-mailman (output) from mailman id 133139.248238; Thu, 27 May 2021 12:35:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFEW-0007gk-Em; Thu, 27 May 2021 12:35:28 +0000 Received: by outflank-mailman (input) for mailman id 133139; Thu, 27 May 2021 12:35:27 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFEV-0007ge-Bf for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:35:27 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 832eeafd-88b1-4951-95ea-f636978a74e7; Thu, 27 May 2021 12:35:26 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A29BE218DD; Thu, 27 May 2021 12:35:25 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 72E3B11A98; Thu, 27 May 2021 12:35:25 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 832eeafd-88b1-4951-95ea-f636978a74e7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118925; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gqg+pjRYzvrnv7dX+llGzQkhdttXtK16oe9Z8AMg9RY=; b=taBtAtRvmeS/XJaxTuCZlR6WQZ2Hi+LLSwxGHHoRDfnuZV5ysOnnt3FvDK80R8EO5vgleO y0gQ2oIpA3hpmJVITyX4M7jqy0aBgCCbcopTv8QNuNIA5geExhxMysP/zSu/dyFXn59p1V 3HmNMpi0ABiccNfotuUqqf3Jceedcbo= Subject: [PATCH v2 10/12] video/vesa: drop "vesa-mtrr" command line option From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: <10607087-9216-c5fb-08cd-29090a80e402@suse.com> Date: Thu, 27 May 2021 14:35:21 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Now that we use ioremap_wc() for mapping the frame buffer, there's no need for this option anymore. As noted in the change introducing the use of ioremap_wc(), mtrr_add() didn't work in certain cases anyway. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,9 +6,10 @@ The format is based on [Keep a Changelog =20 ## [unstable UNRELEASED](https://xenbits.xen.org/gitweb/?p=3Dxen.git;a=3Ds= hortlog;h=3Dstaging) - TBD =20 -### Removed +### Removed / support downgraded - XENSTORED_ROOTDIR environment variable from configuartion files and initscripts, due to being unused. + - dropped support for the (x86-only) "vesa-mtrr" command line option =20 ## [4.15.0 UNRELEASED](https://xenbits.xen.org/gitweb/?p=3Dxen.git;a=3Dsho= rtlog;h=3DRELEASE-4.15.0) - TBD =20 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2369,9 +2369,6 @@ cache-warming. 1ms (1000) has been measu ### vesa-map > `=3D ` =20 -### vesa-mtrr -> `=3D ` - ### vesa-ram > `=3D ` =20 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1816,8 +1816,6 @@ void __init noreturn __start_xen(unsigne =20 local_irq_enable(); =20 - vesa_mtrr_init(); - early_msi_init(); =20 iommu_setup(); /* setup iommu if available */ --- a/xen/drivers/video/vesa.c +++ b/xen/drivers/video/vesa.c @@ -145,38 +145,6 @@ void __init vesa_init(void) video_puts =3D lfb_redraw_puts; } =20 -#include - -static unsigned int vesa_mtrr; -integer_param("vesa-mtrr", vesa_mtrr); - -void __init vesa_mtrr_init(void) -{ - static const int mtrr_types[] =3D { - 0, MTRR_TYPE_UNCACHABLE, MTRR_TYPE_WRBACK, - MTRR_TYPE_WRCOMB, MTRR_TYPE_WRTHROUGH }; - unsigned int size_total; - int rc, type; - - if ( !lfb || (vesa_mtrr =3D=3D 0) || (vesa_mtrr >=3D ARRAY_SIZE(mtrr_t= ypes)) ) - return; - - type =3D mtrr_types[vesa_mtrr]; - if ( !type ) - return; - - /* Find the largest power-of-two */ - size_total =3D vram_total; - while ( size_total & (size_total - 1) ) - size_total &=3D size_total - 1; - - /* Try and find a power of two to add */ - do { - rc =3D mtrr_add(lfb_base(), size_total, type, 1); - size_total >>=3D 1; - } while ( (size_total >=3D PAGE_SIZE) && (rc =3D=3D -EINVAL) ); -} - static void lfb_flush(void) { __asm__ __volatile__ ("sfence" : : : "memory"); --- a/xen/include/asm-x86/setup.h +++ b/xen/include/asm-x86/setup.h @@ -25,10 +25,8 @@ void init_IRQ(void); =20 #ifdef CONFIG_VIDEO void vesa_init(void); -void vesa_mtrr_init(void); #else static inline void vesa_init(void) {}; -static inline void vesa_mtrr_init(void) {}; #endif =20 int construct_dom0( From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622118989; cv=none; d=zohomail.com; s=zohoarc; b=ZGrpGKcmWWytUKLQblJdJQ0ZZ0HOV45M83XVYHylBr1Ip2Pa7ez33c06nEtCuBFVxugJctPcG8AV1HQ0h8ZiTnWbngeYXWAEcDoA8soDjNIStju8rlEt0UOo+IcTSkoNo1T68iyC05LaW9nfRi1+twxo1W2ni9aTl4ZEd7LXSUo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622118989; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JCk3FBjjK8M9z1jIUmNcbjwmgv39UQ+Tt+/agq7sJXk=; b=Wc+PwfFq0PfI+bPPH7wtBvY5mm0q0a2NtY3ds6Jv0OyZ8UMupBEIgx5ghvJAcn7vjTN7cm6q575WW69AW2ioVHWaNlhiiFuUhfO3w++d+T21I4/XpmD/9T1E2Q8WCArWnHOWWWOwHq1xkqUv/YQzrFHRC7IaKsQ7GR7e51ZXWQM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622118989318291.5812465921663; Thu, 27 May 2021 05:36:29 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133147.248250 (Exim 4.92) (envelope-from ) id 1lmFF9-0008IV-SJ; Thu, 27 May 2021 12:36:07 +0000 Received: by outflank-mailman (output) from mailman id 133147.248250; Thu, 27 May 2021 12:36:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFF9-0008IO-Oc; Thu, 27 May 2021 12:36:07 +0000 Received: by outflank-mailman (input) for mailman id 133147; Thu, 27 May 2021 12:36:06 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFF8-0008Bu-BM for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:36:06 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id e29f1ad8-fc4f-433a-905e-58fd79203c5f; Thu, 27 May 2021 12:35:55 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 94EED218DD; Thu, 27 May 2021 12:35:54 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 7A26411A98; Thu, 27 May 2021 12:35:54 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e29f1ad8-fc4f-433a-905e-58fd79203c5f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622118954; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JCk3FBjjK8M9z1jIUmNcbjwmgv39UQ+Tt+/agq7sJXk=; b=CwLdHgXTlzL4WPEEbXsEsixG1mmn4UZqKLjoaWkaBGf3DSCk/P8CmLo224FZA/2vi9m6ls TKO+TG4q4vYjIJZnsZCCWvsU3GZ7KKCwlyQCQChSb+INw2xvUcfU4+wCVyxrun6EvuRpIE QZ9vBsNlwEcQYdXTTEgfI0aQHwokkME= Subject: [PATCH v2 11/12] video/vesa: drop "vesa-remap" command line option From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: Date: Thu, 27 May 2021 14:35:50 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" If we get mode dimensions wrong, having the remapping size controllable via command line option isn't going to help much. Drop the option. While adjusting this also - add __initdata to the variable, - use ROUNDUP() instead of open-coding it. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,7 @@ The format is based on [Keep a Changelog ### Removed / support downgraded - XENSTORED_ROOTDIR environment variable from configuartion files and initscripts, due to being unused. - - dropped support for the (x86-only) "vesa-mtrr" command line option + - dropped support for the (x86-only) "vesa-mtrr" and "vesa-remap" command= line options =20 ## [4.15.0 UNRELEASED](https://xenbits.xen.org/gitweb/?p=3Dxen.git;a=3Dsho= rtlog;h=3DRELEASE-4.15.0) - TBD =20 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2366,9 +2366,6 @@ PCPUs when using the credit1 scheduler. of a VCPU between CPUs, and reduces the implicit overheads such as cache-warming. 1ms (1000) has been measured as a good value. =20 -### vesa-map -> `=3D ` - ### vesa-ram > `=3D ` =20 --- a/xen/drivers/video/vesa.c +++ b/xen/drivers/video/vesa.c @@ -26,8 +26,7 @@ static bool_t vga_compat; static unsigned int vram_total; integer_param("vesa-ram", vram_total); =20 -static unsigned int vram_remap; -integer_param("vesa-map", vram_remap); +static unsigned int __initdata vram_remap; =20 static int font_height; static int __init parse_font_height(const char *s) @@ -79,12 +78,8 @@ void __init vesa_early_init(void) * use for vesafb. With modern cards it is no * option to simply use vram_total as that * wastes plenty of kernel address space. */ - vram_remap =3D (vram_remap ? - (vram_remap << 20) : - ((vram_vmode + (1 << L2_PAGETABLE_SHIFT) - 1) & - ~((1 << L2_PAGETABLE_SHIFT) - 1))); - vram_remap =3D max_t(unsigned int, vram_remap, vram_vmode); - vram_remap =3D min_t(unsigned int, vram_remap, vram_total); + vram_remap =3D ROUNDUP(vram_vmode, 1 << L2_PAGETABLE_SHIFT); + vram_remap =3D min(vram_remap, vram_total); } =20 void __init vesa_init(void) From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1622119027; cv=none; d=zohomail.com; s=zohoarc; b=FbXAI15FgsGEINLRPark2mzthVEl8v4uud7j0wdhHDOW0rmuqLhBzlftBCt/kyEhpBBkJFSCQlLR4koGGc3INBnT8/WIiZmctOoKIUtbN1TbBZaor3+a9h1/I8ZJB/Acw5/Sx19sKbirngHbtcUstHUNzftAm27OF6DyLP9iVos= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622119027; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MhpIoP1aaH4G2df5l8aytfLGQDfICiVkm9QqY6PH9WQ=; b=XslgXuiSrC+Cz3Vnnj6s4ueEbf1LZzqpcIEAgg0zxzN+ZTKn5BUrrULDcNKlZ+zEXI2xdJ090UZat/KuJ1I0GkXgMvAmMWzYv3emxI+EhG4c4VdFaesXkY5WX8oncHYC10EQq3tGEMCiZG+W7p6qIAsueY1ROLAJm5qn8lqkBpc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622119027492719.5391545460702; Thu, 27 May 2021 05:37:07 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133157.248261 (Exim 4.92) (envelope-from ) id 1lmFFu-0000Z2-A9; Thu, 27 May 2021 12:36:54 +0000 Received: by outflank-mailman (output) from mailman id 133157.248261; Thu, 27 May 2021 12:36:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFFu-0000Yv-6d; Thu, 27 May 2021 12:36:54 +0000 Received: by outflank-mailman (input) for mailman id 133157; Thu, 27 May 2021 12:36:52 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmFFs-0000Rp-EK for xen-devel@lists.xenproject.org; Thu, 27 May 2021 12:36:52 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e121168b-91ff-47b9-8d50-55f2917047ce; Thu, 27 May 2021 12:36:44 +0000 (UTC) Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 68EB02190B; Thu, 27 May 2021 12:36:43 +0000 (UTC) Received: from director2.suse.de (director2.suse-dmz.suse.de [192.168.254.72]) by imap.suse.de (Postfix) with ESMTPSA id 4E09411A98; Thu, 27 May 2021 12:36:43 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e121168b-91ff-47b9-8d50-55f2917047ce DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1622119003; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MhpIoP1aaH4G2df5l8aytfLGQDfICiVkm9QqY6PH9WQ=; b=JOkSOdwks15QM8gFY5bUcX9p3IPMpCQ2Nr+zN+ZuOWWti45KYkFv/DaKE6PUR+KCLrdJrq 1fZgcETH4CPMQiJPL3ikpklKHnV3JsHwhbfEIh8f6XniuuVTTv0T+ppDpy5Z71nUJ/GMdt pnQNmCZ8H3aqC7xb4uTVuyatTW/Tnmk= Subject: [PATCH v2 12/12] video/vesa: adjust (not just) command line option handling From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Message-ID: Date: Thu, 27 May 2021 14:36:39 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Document the remaining option. Add section annotation to the variable holding the parsed value as well as a few adjacent ones. Adjust the types of font_height and vga_compat. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- v2: Re-base over added earlier patch. --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2369,6 +2369,11 @@ cache-warming. 1ms (1000) has been measu ### vesa-ram > `=3D ` =20 +> Default: `0` + +This allows to override the amount of video RAM, in MiB, determined to be +present. + ### vga > `=3D ( ask | current | text-80x | gfx-xx | m= ode- )[,keep]` =20 --- a/xen/drivers/video/vesa.c +++ b/xen/drivers/video/vesa.c @@ -19,16 +19,16 @@ =20 static void lfb_flush(void); =20 -static unsigned char *lfb; -static const struct font_desc *font; -static bool_t vga_compat; +static unsigned char *__read_mostly lfb; +static const struct font_desc *__initdata font; +static bool __initdata vga_compat; =20 -static unsigned int vram_total; +static unsigned int __initdata vram_total; integer_param("vesa-ram", vram_total); =20 static unsigned int __initdata vram_remap; =20 -static int font_height; +static unsigned int __initdata font_height; static int __init parse_font_height(const char *s) { if ( simple_strtoul(s, &s, 10) =3D=3D 8 && (*s++ =3D=3D 'x') ) From nobody Sat Apr 27 11:26:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=suse.com); dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=2; a=rsa-sha256; t=1645095713; cv=pass; d=zohomail.com; s=zohoarc; b=HaPMfgLHTMEDEM3WXknlR7PO+6hprO8omeenr8Oe8ZkzckTpOr92z+PsqHxKJELn314bOVClL/NIyYHghRm7ucgvQFApMqRAmGm/xGz+qZJCwG7oW3U/y6rUbnKaGttv7bhz066bUoLiRB5nbPbhd/2gpuHWrI2WiprVNbEG9EI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1645095713; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zM5O0NIegQlxsb91H6RIMRG/er+065jMoSojdzl6S6Q=; b=NuNFP9mejUxPJJoKalurXRRsToK49RnYkDB5aY7IbobLcPQHJtkHUh3PZt+g3R4J7m5ICzEMwB5O6h0XzDlc7UIarPO6GdOt4xM5US1hIFSzHsqYH7agAEpyQwWoGgkE1Y4411tg4MO2uj0JK/FVRUlFVp/bGKmk4WPlIC1sKmU= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=suse.com); dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1645095713848751.7872101773557; Thu, 17 Feb 2022 03:01:53 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.274658.470213 (Exim 4.92) (envelope-from ) id 1nKeXF-0006gX-Hy; Thu, 17 Feb 2022 11:01:17 +0000 Received: by outflank-mailman (output) from mailman id 274658.470213; Thu, 17 Feb 2022 11:01:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nKeXF-0006gL-EW; Thu, 17 Feb 2022 11:01:17 +0000 Received: by outflank-mailman (input) for mailman id 274658; Thu, 17 Feb 2022 11:01:16 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nKeXE-0006fR-45 for xen-devel@lists.xenproject.org; Thu, 17 Feb 2022 11:01:16 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ec5e39c7-8fe0-11ec-b215-9bbe72dcb22c; Thu, 17 Feb 2022 12:01:14 +0100 (CET) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2053.outbound.protection.outlook.com [104.47.9.53]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-16-HJYdhSgqOM-rhwrN7qPJqA-1; Thu, 17 Feb 2022 12:01:13 +0100 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) by DB9PR04MB8393.eurprd04.prod.outlook.com (2603:10a6:10:25f::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.16; Thu, 17 Feb 2022 11:01:10 +0000 Received: from VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::d479:b728:345c:bd65]) by VI1PR04MB5600.eurprd04.prod.outlook.com ([fe80::d479:b728:345c:bd65%5]) with mapi id 15.20.4995.016; Thu, 17 Feb 2022 11:01:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ec5e39c7-8fe0-11ec-b215-9bbe72dcb22c DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1645095674; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zM5O0NIegQlxsb91H6RIMRG/er+065jMoSojdzl6S6Q=; b=TYPY4PDZj0qd1jgBfGHocq6yjire/7ThufXSXY/WjuTGtPcBhwk2W3MACD4rro8x2+lSVp eg5zz4pocbRh0EEr3UsEtBKW4VnGEVl9EjADeKJ9glK+tW4cEdv617inBJ+YsP3lzmVW+4 lVwFd7P3232yapMK9stnqJ2sTz8ZUE4= X-MC-Unique: HJYdhSgqOM-rhwrN7qPJqA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KQR5tHm0FmEeH94C5RKwgbdaFuYzGqU32M072OB2hO5fVYIj6kyT4sO28JOfTd/h2f45hoKBwYnVMevw/sBdPTFuNf9CL29QT/ITB5HwoInbnzYt2GtOf0wMD32Sju4RCQYGb47KSAuFi2KsHzJBm31O64+hIQ9wX0eiw+XnL2yzcmZ8MYkquLINNvDXAmYGr0kB9fkpPzedWGuBx5cZgrabYECJBFwUpj0pTOLUTEm3Db+dbyx7RzK75tm+Zakmh9PtPbBZAvkosFZZIECoHDeuqSRycyG+p1emromuqJI1gAXe9q6Q6mEderr1X6+lA9sHN1VMrmoyiCl4UCoICA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zM5O0NIegQlxsb91H6RIMRG/er+065jMoSojdzl6S6Q=; b=K8bV+/t4COAsnTQP+HuaoJvkgYU9T4M93o0s1fU3jAwmBKq+p7PC1x7X1tMf5K2UYIH5d6QhdP6BP+J+6m9TiDyhQ5P/4RuabNdnw8LUmkDrNXcTXAxG5OC1W621n5HDmuQ5AH2A7OaXzrZTKxJVVPTPsKmgDacwVJnedMtaoFELZlp9QjO3G0Ugwtd+dlm2ixlj4HLfUkRce8NbR6Pje+9SgKqtpNuk93noQ/znU3b/y5yRgg+8EkwcxfW/4h27hSlKr0x2BMV17cNVcaIjyOcDojzqI+gZL4X5GqTQ9oCR2PBeLm3vRF1GtzAcr81fB3t2kPb+B2dBXd7FMrcOUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Thu, 17 Feb 2022 12:01:08 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.6.1 From: Jan Beulich Subject: [PATCH RESEND v2] x86: introduce ioremap_wc() To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Language: en-US In-Reply-To: <8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS9PR05CA0056.eurprd05.prod.outlook.com (2603:10a6:20b:489::19) To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9238906f-8bb4-4832-752c-08d9f204ce45 X-MS-TrafficTypeDiagnostic: DB9PR04MB8393:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5AL8La9bJwtxDuvrq8SCQfmpPoKj9GHkadxfxGpYoS20pKJXKjzb6x+wn3wQbTHXHdiSM8wCCZ6APTDCQeFRfFBler7jOtlHiJB76E90AS8r7wbBZo02kEdeNx7y4nGy883rnvyRHCtpZeyzEvsKte6jxeDo0qRRAxVNaYjmXVoZE49dJlVAOlaZclaZKQWzv9W7noxDNkPyy5xUMExYqhGyP6ghs4oCdHPrLfBEZaT5c3Tjlmz/fmNlDXBBZRvSkvFsRgmELkoQIDntXABUNyfxBqvTXsGdDJEUPQloMvaWTebS6D7pEy6RAIPnUF0aFQ9yOHRS/vqr8Lo35ciGbUeIEs42f0Ygu8QN7fw+OdJMF6cfP+QdiTwWdXlya1CHmXi0AqtPnR76CCmYVBe2SkvG+M/1/XrGxa98t1EvtKENuJjN/vloKRgRFqW15b9CXRkQUDSqJku/yaL+679GyCecNSMUdBYlwUzE9SXPwi4Hh12ct0cCzNCGGX+52jhPDKC0Ru3rwC4dTGdte4YAco5OuEfUYRg9jtWPfUnT98d19fvKkoPHVKNiYl7/8f+cZ2/JZ62stylQ8lT+u1VprqoRQ5v9CqnAFD5w9Wk7Ok/g+3o9hQaUFXX/NE+DvbbKBQz5uvq+wozjJ+hgsYVFlgJO0QxEqNTB83uuKvdb36ZYaFB3QCMTf0h2K2Uhy4VK4/W81ZC2ODEFXR4FaD+Ve3C8CTk5jfGNppE8D580Ddu7UQFhKxXwS8Ze8voc/AGZgby1RboRlwcjM/x4iOU91w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(66946007)(54906003)(6506007)(66556008)(31696002)(66476007)(6512007)(2616005)(86362001)(8936002)(38100700002)(6916009)(8676002)(316002)(6486002)(508600001)(4326008)(26005)(83380400001)(31686004)(5660300002)(36756003)(186003)(2906002)(43740500002)(45980500001)(473944003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?STVoamphVFliNGx2cmhuRzRoVm5oazNxTnllYTBYSUhESFMvQjc3T3BaV3Iz?= =?utf-8?B?aDA2UjEzdVZJQzZtaEpMQTFIajVQWVJQdHdoTnp6VjRWZkdSNlZtUHlyWHVp?= =?utf-8?B?RXlsMGxBaklKcitxVnBKamk1RCtYVjN0KzBzcll2ZTFqZWNZbHJiTDJtZ3Zm?= =?utf-8?B?bmR0QzFDMy8xcmxqRWUrS2tGamwzWUtmbnNua0wrdm4zNHJZQWlFbkRzWUZT?= =?utf-8?B?M2VZNWUxaUVLSmlBQUxsbERNemowU1htaURaN0ZIWmxPblN4Um1SUitXOW9G?= =?utf-8?B?TXJBUXdUNGFCbkR6WExYam10dkRqdnY0ckl0NjlwUzFPT2Y1MmpUenRqWEtq?= =?utf-8?B?K054czJXZE5CbHlwd1JlVEI2Tk9rK3huS3BvQ3hDTWFYSDM0Z2pWNzF6Q1Ay?= =?utf-8?B?RGVEaFh6Y1RRNnVHald0OWlycEk1R1JRWXhKR24xVWhBNlNuZ1RodS9aYzc0?= =?utf-8?B?blBWRFR0cEpoS3JKWGxMeGVUalFTYlh6MWR4MXlKSVBmVDhEZUE0c3FCOVRT?= =?utf-8?B?M3FneUMxTk1tVEVST1VVU0lSRThxQlllYms2SnFYNmEybmRPVjlqVUtvZ3cx?= =?utf-8?B?emRPUDY1ZmRNdk9tU0dMUHMycWNyQ1ExVnR1dUUwYUdyK2hyR2tzcDJid0hv?= =?utf-8?B?Tk5zRmdYaEt1QnowK3JaWjNya0tZaFBISGtYazhML3h6Z082Sm5WaVNkOHA5?= =?utf-8?B?L0Y2T01Qc2pNb0FxRUVMZVZLaDBwbUtmdE5xV2RJVjFiSmk1R0dhbkJLemM1?= =?utf-8?B?NURqTjdUTFR1VVVuVWJLRjBNWkF3d1hhQ1JMTU12LzU4RmZiaEdoZ1F3QXhX?= =?utf-8?B?eTZTV1FOWThRZXVHd0I5eGlVZ1kxMGJHcVVTRnlCdWJwM3JWSjRZN1o2QkJ4?= =?utf-8?B?aStoSVNVejFWb2VFemJ1R0pDeHZINFNQN3B5SlgvZ2ZSY0hid0lxRk80WFNF?= =?utf-8?B?MmQ3QU1CM1NFWjVpbDNkYWl5QWlWcnFLczJxN1ZYeThNZUFOYTFubDlET2Ew?= =?utf-8?B?VVU0bXRjQkI3a2xHY1hNV0R3ZVBCVHorcUoyTlJ4OHlNQldZYkpyN2hKNlNx?= =?utf-8?B?ZFpFQUtzV1BKTkszVzV3L1hHMzZIUE9VWmNlMS82ZU9EaHI5TTI2akVrQndM?= =?utf-8?B?T0k4SC84K3NQQ2ZHN2dEeHFEcDRzK2RJSDZuL0lhYlZvcm84bi9WamxWMElj?= =?utf-8?B?OWY3TUc4SnhqaVNXTk1OMU9Ub3NPUnUxc1dhem85cHdyMkhZZ0dHRnlmUjl3?= =?utf-8?B?OEFkOHc0eG9xY1pzWUxQOVlhYkxPSVhRSVRqVytYQXNYZzNINmlReURUNGpR?= =?utf-8?B?SnRrZUFIT2twUHBwbnRCSnFiOEVQT05MTmVTVjI2eGhIdnZPTGhMazZEVFpt?= =?utf-8?B?c2U4UGltZ3hOODVSOUlTQklqRzhpUFZKaE14ODJycE1RS3dqdlpQd29MRXVq?= =?utf-8?B?TEV6S3VoQlVBaUV1cm9NU3NaNFBndE13UGkzTE80N29rMlBpaGR6czNic2Vh?= =?utf-8?B?Z0RJWjZUTFdBMEUxa09NcmErZ0Y3OGNXaW1VeWR0dTlRVEVYZy9pdVUrbXpq?= =?utf-8?B?Wm9uRUE4aFhwZzUyNUd2NHRYRGhqT01pQ2VxM0wxNkpGZXc3RTNBQUoyOUFL?= =?utf-8?B?bGphYUszR3JmRGMxc3BnZTM5UWVlOTVpRVZVMGJkZ3hhOVhzRkRsZGtyeEFJ?= =?utf-8?B?SGQwZzk3eVVXRXNhdi9qRFJvSzlkdUt3Q3JhSFJlUnFjSFRNUFZ0TmpSQUJX?= =?utf-8?B?S05sZWhpSUhyRWxVTEhReW5FWjRuaCtDZGxMQVY2Qlg0aWpucG9wMkZqa21u?= =?utf-8?B?ZVdlbTE3ZnhDNjdZT3VCWUErNTdnejVJOFhhOVJSc2l6T05uVWY0NkliN3N2?= =?utf-8?B?dzhxcEtuNFl6VWFJeWRiNjB1aTFDTjdKSndHRElOVmVLa1FpTXY3eW05Sm9o?= =?utf-8?B?akUzRStrcUk2SXNSTnh5RDJ0Y1FURkp5cTZpeDl3dHg1b1BjT0RXSFd6VHlr?= =?utf-8?B?T0ZVRDB3b0RWR1NXZzJscmxva3M3bzBtR1pLdXVnRTUycFF6cmZURDlxMUU4?= =?utf-8?B?MGtlWXVhZnFMOVVVUlNoTWJseG9zOHRhOGlMV2hUT0ZBaWJSQlNZejk5MFU2?= =?utf-8?B?VGhOa1BJQlR5UDFnN2FsMFRwRnEzcUpzSHAvNkZlNHpwcUUyTm9uc0lEa01m?= =?utf-8?Q?7Dwxkysy3I/nLXV1eI3P3fk=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9238906f-8bb4-4832-752c-08d9f204ce45 X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Feb 2022 11:01:10.5491 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IfMaBLa+D64+yGhqZ8+OL+tbQfUacDFStusHZCaJYJHPWTZoNr84icy6MJN8ecgYXS+jR0QIre990AdbqnIq6A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR04MB8393 X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1645095717608100001 Content-Type: text/plain; charset="utf-8" In order for a to-be-introduced ERMS form of memcpy() to not regress boot performance on certain systems when video output is active, we first need to arrange for avoiding further dependency on firmware setting up MTRRs in a way we can actually further modify. On many systems, due to the continuously growing amounts of installed memory, MTRRs get configured with at least one huge WB range, and with MMIO ranges below 4Gb then forced to UC via overlapping MTRRs. mtrr_add(), as it is today, can't deal with such a setup. Hence on such systems we presently leave the frame buffer mapped UC, leading to significantly reduced performance when using REP STOSB / REP MOVSB. On post-PentiumII hardware (i.e. any that's capable of running 64-bit code), an effective memory type of WC can be achieved without MTRRs, by simply referencing the respective PAT entry from the PTEs. While this will leave the switch to ERMS forms of memset() and memcpy() with largely unchanged performance, the change here on its own improves performance on affected systems quite significantly: Measuring just the individual affected memcpy() invocations yielded a speedup by a factor of over 250 on my initial (Skylake) test system. memset() isn't getting improved by as much there, but still by a factor of about 20. While adding {__,}PAGE_HYPERVISOR_WC, also add {__,}PAGE_HYPERVISOR_WT to, at the very least, make clear what PTE flags this memory type uses. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monn=C3=A9 --- REPOST (in isolation) upon Roger's request. The header location change I don't really consider a "re-base". v2: Mark ioremap_wc() __init. --- TBD: If the VGA range is WC in the fixed range MTRRs, reusing the low 1st Mb mapping (like ioremap() does) would be an option. --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -602,6 +602,8 @@ void destroy_perdomain_mapping(struct do unsigned int nr); void free_perdomain_mappings(struct domain *); =20 +void __iomem *ioremap_wc(paddr_t, size_t); + extern int memory_add(unsigned long spfn, unsigned long epfn, unsigned int= pxm); =20 void domain_set_alloc_bitsize(struct domain *d); --- a/xen/arch/x86/include/asm/page.h +++ b/xen/arch/x86/include/asm/page.h @@ -349,8 +349,10 @@ void efi_update_l4_pgtable(unsigned int #define __PAGE_HYPERVISOR_RX (_PAGE_PRESENT | _PAGE_ACCESSED) #define __PAGE_HYPERVISOR (__PAGE_HYPERVISOR_RX | \ _PAGE_DIRTY | _PAGE_RW) +#define __PAGE_HYPERVISOR_WT (__PAGE_HYPERVISOR | _PAGE_PWT) #define __PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR | _PAGE_PCD) #define __PAGE_HYPERVISOR_UC (__PAGE_HYPERVISOR | _PAGE_PCD | _PAGE_P= WT) +#define __PAGE_HYPERVISOR_WC (__PAGE_HYPERVISOR | _PAGE_PAT) #define __PAGE_HYPERVISOR_SHSTK (__PAGE_HYPERVISOR_RO | _PAGE_DIRTY) =20 #define MAP_SMALL_PAGES _PAGE_AVAIL0 /* don't use superpages mappings */ --- a/xen/arch/x86/include/asm/x86_64/page.h +++ b/xen/arch/x86/include/asm/x86_64/page.h @@ -152,6 +152,10 @@ static inline intpte_t put_pte_flags(uns _PAGE_GLOBAL | _PAGE_NX) #define PAGE_HYPERVISOR_UC (__PAGE_HYPERVISOR_UC | \ _PAGE_GLOBAL | _PAGE_NX) +#define PAGE_HYPERVISOR_WC (__PAGE_HYPERVISOR_WC | \ + _PAGE_GLOBAL | _PAGE_NX) +#define PAGE_HYPERVISOR_WT (__PAGE_HYPERVISOR_WT | \ + _PAGE_GLOBAL | _PAGE_NX) =20 #endif /* __X86_64_PAGE_H__ */ =20 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5895,6 +5895,20 @@ void __iomem *ioremap(paddr_t pa, size_t return (void __force __iomem *)va; } =20 +void __iomem *__init ioremap_wc(paddr_t pa, size_t len) +{ + mfn_t mfn =3D _mfn(PFN_DOWN(pa)); + unsigned int offs =3D pa & (PAGE_SIZE - 1); + unsigned int nr =3D PFN_UP(offs + len); + void *va; + + WARN_ON(page_is_ram_type(mfn_x(mfn), RAM_TYPE_CONVENTIONAL)); + + va =3D __vmap(&mfn, nr, 1, 1, PAGE_HYPERVISOR_WC, VMAP_DEFAULT); + + return (void __force __iomem *)(va + offs); +} + int create_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg) --- a/xen/drivers/video/vesa.c +++ b/xen/drivers/video/vesa.c @@ -9,9 +9,9 @@ #include #include #include +#include #include #include -#include #include "font.h" #include "lfb.h" =20 @@ -103,7 +103,7 @@ void __init vesa_init(void) lfbp.text_columns =3D vlfb_info.width / font->width; lfbp.text_rows =3D vlfb_info.height / font->height; =20 - lfbp.lfb =3D lfb =3D ioremap(lfb_base(), vram_remap); + lfbp.lfb =3D lfb =3D ioremap_wc(lfb_base(), vram_remap); if ( !lfb ) return; =20 @@ -179,8 +179,7 @@ void __init vesa_mtrr_init(void) =20 static void lfb_flush(void) { - if ( vesa_mtrr =3D=3D 3 ) - __asm__ __volatile__ ("sfence" : : : "memory"); + __asm__ __volatile__ ("sfence" : : : "memory"); } =20 void __init vesa_endboot(bool_t keep) --- a/xen/drivers/video/vga.c +++ b/xen/drivers/video/vga.c @@ -79,7 +79,7 @@ void __init video_init(void) { case XEN_VGATYPE_TEXT_MODE_3: if ( page_is_ram_type(paddr_to_pfn(0xB8000), RAM_TYPE_CONVENTIONAL= ) || - ((video =3D ioremap(0xB8000, 0x8000)) =3D=3D NULL) ) + ((video =3D ioremap_wc(0xB8000, 0x8000)) =3D=3D NULL) ) return; outw(0x200a, 0x3d4); /* disable cursor */ columns =3D vga_console_info.u.text_mode_3.columns; @@ -164,7 +164,11 @@ void __init video_endboot(void) { case XEN_VGATYPE_TEXT_MODE_3: if ( !vgacon_keep ) + { memset(video, 0, columns * lines * 2); + iounmap(video); + video =3D ZERO_BLOCK_PTR; + } break; case XEN_VGATYPE_VESA_LFB: case XEN_VGATYPE_EFI_LFB: