From nobody Tue May 14 17:59:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1694605131; cv=none; d=zohomail.com; s=zohoarc; b=GOjNRjq7He5HQTbI2/+Mxskps2qAVhdWp9UZlz9suA8QmFqGlT5ePRQLDdz2YZinqMrxyiFvFrZDiOpvYFjvy3oMsGUXTrc2ME2CNe+ZKtlXqozzBU8yMoXlTmIQjsPWpJPWfRm2VnilH+yN4BMJgB/ztvSYTTJsHEBNNzHbulQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694605131; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dVPKyjPuFRiMX6H05C4yegFljVDNBdmN4FRlJrZXtC8=; b=awZFNn+1lQRmhG2jXNmbT/BQ0zbTKGSzeRROSv55MtJHnBLWaNnodaP192/eG2Uuf62JqoQuer8xMaTgz4IodhNJPZa3cRTqvaYUmqqPuEr1RbIEM7wISeACRLfQ1mDWqnScI8Xqkj0zyK89d5m+Rih2mJIhsnL5KZxBdFJZjSU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1694605131834877.0014728274921; Wed, 13 Sep 2023 04:38:51 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.601226.937193 (Exim 4.92) (envelope-from ) id 1qgOCc-00033q-CR; Wed, 13 Sep 2023 11:38:38 +0000 Received: by outflank-mailman (output) from mailman id 601226.937193; Wed, 13 Sep 2023 11:38:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgOCc-00033j-9Y; Wed, 13 Sep 2023 11:38:38 +0000 Received: by outflank-mailman (input) for mailman id 601226; Wed, 13 Sep 2023 11:38:37 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgOCb-0002ka-2s for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 11:38:37 +0000 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 1378d01e-522a-11ee-8786-cb3800f73035; Wed, 13 Sep 2023 13:38:36 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 50BCD218E3; Wed, 13 Sep 2023 11:38:36 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 223A113582; Wed, 13 Sep 2023 11:38:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0PoHBzyfAWVoCwAAMHmgww (envelope-from ); Wed, 13 Sep 2023 11:38:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1378d01e-522a-11ee-8786-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1694605116; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dVPKyjPuFRiMX6H05C4yegFljVDNBdmN4FRlJrZXtC8=; b=DlBJopdppVqDlPo0820IN+B0NKE4azUrDiJXIdKEpL7OlV6MYr3nHi2QMsbkX4If5GTMSw NvGizRK0K/HmNwxhO3Ur6reeYkbMTx8gOR9zz1u+xaGJmgkuGNayQN6qLQvBTG8SUtXD26 hVytxMzRpTNnAk+YIeojWOkv2IacXs8= From: Juergen Gross To: linux-kernel@vger.kernel.org Cc: Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , xen-devel@lists.xenproject.org Subject: [PATCH 1/3] arm/xen: remove lazy mode related definitions Date: Wed, 13 Sep 2023 13:38:26 +0200 Message-Id: <20230913113828.18421-2-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230913113828.18421-1-jgross@suse.com> References: <20230913113828.18421-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1694605132717100003 Content-Type: text/plain; charset="utf-8" include/xen/arm/hypervisor.h contains definitions related to paravirt lazy mode, which are used nowhere in the code. All paravirt lazy mode related users are in x86 code, so remove the definitions on Arm side. Signed-off-by: Juergen Gross Acked-by: Stefano Stabellini --- include/xen/arm/hypervisor.h | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/include/xen/arm/hypervisor.h b/include/xen/arm/hypervisor.h index 43ef24dd030e..9995695204f5 100644 --- a/include/xen/arm/hypervisor.h +++ b/include/xen/arm/hypervisor.h @@ -7,18 +7,6 @@ extern struct shared_info *HYPERVISOR_shared_info; extern struct start_info *xen_start_info; =20 -/* Lazy mode for batching updates / context switch */ -enum paravirt_lazy_mode { - PARAVIRT_LAZY_NONE, - PARAVIRT_LAZY_MMU, - PARAVIRT_LAZY_CPU, -}; - -static inline enum paravirt_lazy_mode paravirt_get_lazy_mode(void) -{ - return PARAVIRT_LAZY_NONE; -} - #ifdef CONFIG_XEN void __init xen_early_init(void); #else --=20 2.35.3 From nobody Tue May 14 17:59:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1694605141; cv=none; d=zohomail.com; s=zohoarc; b=ZsLAwt26leScpt99vWTZGwpeOHE2CErOsuz6n3lrdG0km6D7q4KiWqKqHpdlUcY7+ZwkmnM9CnWEfJOnlAdf6HmcLIhUMi0BOwniXHim7mgxNHE9nISD0nzzWQ6c7ifBaCLjE7ggHXnEohTVI4dT85zFfCfil4SEbnAK2FlkV5I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694605141; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pys8ZMeihbqu1bWQt/XnMbkpDqMreD1pEdIt/VtSRwk=; b=mCGJe78yWZF40QeJ3p3LTf+F4pnDt4gNrZISnIP5ZRdRm+7+28JYv82JxhX96/hVERdvl00IOlCi/BifCro+HGDo6LD6I4IexxUFXgnrkhy+gQpmsyzMG+8X7bDDiWKw5X3QbynPLTMYK6cWhu3anXHFNq6165TDj+5kDxdWPrY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1694605141418874.9366664669446; Wed, 13 Sep 2023 04:39:01 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.601228.937204 (Exim 4.92) (envelope-from ) id 1qgOCj-0003Qz-MO; Wed, 13 Sep 2023 11:38:45 +0000 Received: by outflank-mailman (output) from mailman id 601228.937204; Wed, 13 Sep 2023 11:38:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgOCj-0003Qo-HW; Wed, 13 Sep 2023 11:38:45 +0000 Received: by outflank-mailman (input) for mailman id 601228; Wed, 13 Sep 2023 11:38:44 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgOCi-0002ka-7i for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 11:38:44 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 17164623-522a-11ee-8786-cb3800f73035; Wed, 13 Sep 2023 13:38:42 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 2C7F81F45B; Wed, 13 Sep 2023 11:38:42 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BB6E513582; Wed, 13 Sep 2023 11:38:41 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id rzyeLEGfAWV8CwAAMHmgww (envelope-from ); Wed, 13 Sep 2023 11:38:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 17164623-522a-11ee-8786-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1694605122; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pys8ZMeihbqu1bWQt/XnMbkpDqMreD1pEdIt/VtSRwk=; b=KbCJatsAJtpEHO9X8iEcCiF/f6loxsNqBoDiqyiM0AFZ4RzaFcQc/ha3w17oXxz1bsUNdh ZxKyIQhwguDSEfDE0PPBW56VU6oh+zzqVzG9E5Ug9oY+uyW3zHiP4GNhNoOgkrAnhNd8SL +/49XOWY03XxzlSOwY3uYIO6t6d8bTY= From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org, virtualization@lists.linux-foundation.org, linux-trace-kernel@vger.kernel.org Cc: Juergen Gross , Ajay Kaher , Alexey Makhalov , VMware PV-Drivers Reviewers , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Boris Ostrovsky , Steven Rostedt , Masami Hiramatsu , xen-devel@lists.xenproject.org Subject: [PATCH 2/3] x86/xen: move paravirt lazy code Date: Wed, 13 Sep 2023 13:38:27 +0200 Message-Id: <20230913113828.18421-3-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230913113828.18421-1-jgross@suse.com> References: <20230913113828.18421-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1694605143457100001 Content-Type: text/plain; charset="utf-8" Only Xen is using the paravirt lazy mode code, so it can be moved to Xen specific sources. This allows to make some of the functions static or to merge them into their only call sites. While at it do a rename from "paravirt" to "xen" for all moved specifiers. No functional change. Signed-off-by: Juergen Gross --- arch/x86/include/asm/paravirt_types.h | 15 ------ arch/x86/include/asm/xen/hypervisor.h | 26 +++++++++++ arch/x86/kernel/paravirt.c | 67 --------------------------- arch/x86/xen/enlighten_pv.c | 39 +++++++++++++--- arch/x86/xen/mmu_pv.c | 55 ++++++++++++++-------- arch/x86/xen/multicalls.h | 4 +- include/trace/events/xen.h | 12 ++--- 7 files changed, 102 insertions(+), 116 deletions(-) diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/p= aravirt_types.h index 4acbcddddc29..772d03487520 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -9,13 +9,6 @@ struct paravirt_patch_site { u8 type; /* type of this instruction */ u8 len; /* length of original instruction */ }; - -/* Lazy mode for batching updates / context switch */ -enum paravirt_lazy_mode { - PARAVIRT_LAZY_NONE, - PARAVIRT_LAZY_MMU, - PARAVIRT_LAZY_CPU, -}; #endif =20 #ifdef CONFIG_PARAVIRT @@ -549,14 +542,6 @@ int paravirt_disable_iospace(void); __PVOP_VCALL(op, PVOP_CALL_ARG1(arg1), PVOP_CALL_ARG2(arg2), \ PVOP_CALL_ARG3(arg3), PVOP_CALL_ARG4(arg4)) =20 -enum paravirt_lazy_mode paravirt_get_lazy_mode(void); -void paravirt_start_context_switch(struct task_struct *prev); -void paravirt_end_context_switch(struct task_struct *next); - -void paravirt_enter_lazy_mmu(void); -void paravirt_leave_lazy_mmu(void); -void paravirt_flush_lazy_mmu(void); - void _paravirt_nop(void); void paravirt_BUG(void); unsigned long paravirt_ret0(void); diff --git a/arch/x86/include/asm/xen/hypervisor.h b/arch/x86/include/asm/x= en/hypervisor.h index 5fc35f889cd1..ed05ce3df5c7 100644 --- a/arch/x86/include/asm/xen/hypervisor.h +++ b/arch/x86/include/asm/xen/hypervisor.h @@ -36,6 +36,7 @@ extern struct shared_info *HYPERVISOR_shared_info; extern struct start_info *xen_start_info; =20 +#include #include =20 #define XEN_SIGNATURE "XenVMMXenVMM" @@ -63,4 +64,29 @@ void __init xen_pvh_init(struct boot_params *boot_params= ); void __init mem_map_via_hcall(struct boot_params *boot_params_p); #endif =20 +/* Lazy mode for batching updates / context switch */ +enum xen_lazy_mode { + XEN_LAZY_NONE, + XEN_LAZY_MMU, + XEN_LAZY_CPU, +}; + +DECLARE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode); + +static inline void enter_lazy(enum xen_lazy_mode mode) +{ + BUG_ON(this_cpu_read(xen_lazy_mode) !=3D XEN_LAZY_NONE); + + this_cpu_write(xen_lazy_mode, mode); +} + +static inline void leave_lazy(enum xen_lazy_mode mode) +{ + BUG_ON(this_cpu_read(xen_lazy_mode) !=3D mode); + + this_cpu_write(xen_lazy_mode, XEN_LAZY_NONE); +} + +enum xen_lazy_mode xen_get_lazy_mode(void); + #endif /* _ASM_X86_XEN_HYPERVISOR_H */ diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 975f98d5eee5..97f1436c1a20 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -143,66 +143,7 @@ int paravirt_disable_iospace(void) return request_resource(&ioport_resource, &reserve_ioports); } =20 -static DEFINE_PER_CPU(enum paravirt_lazy_mode, paravirt_lazy_mode) =3D PAR= AVIRT_LAZY_NONE; - -static inline void enter_lazy(enum paravirt_lazy_mode mode) -{ - BUG_ON(this_cpu_read(paravirt_lazy_mode) !=3D PARAVIRT_LAZY_NONE); - - this_cpu_write(paravirt_lazy_mode, mode); -} - -static void leave_lazy(enum paravirt_lazy_mode mode) -{ - BUG_ON(this_cpu_read(paravirt_lazy_mode) !=3D mode); - - this_cpu_write(paravirt_lazy_mode, PARAVIRT_LAZY_NONE); -} - -void paravirt_enter_lazy_mmu(void) -{ - enter_lazy(PARAVIRT_LAZY_MMU); -} - -void paravirt_leave_lazy_mmu(void) -{ - leave_lazy(PARAVIRT_LAZY_MMU); -} - -void paravirt_flush_lazy_mmu(void) -{ - preempt_disable(); - - if (paravirt_get_lazy_mode() =3D=3D PARAVIRT_LAZY_MMU) { - arch_leave_lazy_mmu_mode(); - arch_enter_lazy_mmu_mode(); - } - - preempt_enable(); -} - #ifdef CONFIG_PARAVIRT_XXL -void paravirt_start_context_switch(struct task_struct *prev) -{ - BUG_ON(preemptible()); - - if (this_cpu_read(paravirt_lazy_mode) =3D=3D PARAVIRT_LAZY_MMU) { - arch_leave_lazy_mmu_mode(); - set_ti_thread_flag(task_thread_info(prev), TIF_LAZY_MMU_UPDATES); - } - enter_lazy(PARAVIRT_LAZY_CPU); -} - -void paravirt_end_context_switch(struct task_struct *next) -{ - BUG_ON(preemptible()); - - leave_lazy(PARAVIRT_LAZY_CPU); - - if (test_and_clear_ti_thread_flag(task_thread_info(next), TIF_LAZY_MMU_UP= DATES)) - arch_enter_lazy_mmu_mode(); -} - static noinstr void pv_native_write_cr2(unsigned long val) { native_write_cr2(val); @@ -229,14 +170,6 @@ static noinstr void pv_native_safe_halt(void) } #endif =20 -enum paravirt_lazy_mode paravirt_get_lazy_mode(void) -{ - if (in_interrupt()) - return PARAVIRT_LAZY_NONE; - - return this_cpu_read(paravirt_lazy_mode); -} - struct pv_info pv_info =3D { .name =3D "bare hardware", #ifdef CONFIG_PARAVIRT_XXL diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index 49352fad7d1d..54b83825c4b6 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -101,6 +101,16 @@ struct tls_descs { struct desc_struct desc[3]; }; =20 +DEFINE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode) =3D XEN_LAZY_NONE; + +enum xen_lazy_mode xen_get_lazy_mode(void) +{ + if (in_interrupt()) + return XEN_LAZY_NONE; + + return this_cpu_read(xen_lazy_mode); +} + /* * Updating the 3 TLS descriptors in the GDT on every task switch is * surprisingly expensive so we avoid updating them if they haven't @@ -362,10 +372,25 @@ static noinstr unsigned long xen_get_debugreg(int reg) return HYPERVISOR_get_debugreg(reg); } =20 +static void xen_start_context_switch(struct task_struct *prev) +{ + BUG_ON(preemptible()); + + if (this_cpu_read(xen_lazy_mode) =3D=3D XEN_LAZY_MMU) { + arch_leave_lazy_mmu_mode(); + set_ti_thread_flag(task_thread_info(prev), TIF_LAZY_MMU_UPDATES); + } + enter_lazy(XEN_LAZY_CPU); +} + static void xen_end_context_switch(struct task_struct *next) { + BUG_ON(preemptible()); + xen_mc_flush(); - paravirt_end_context_switch(next); + leave_lazy(XEN_LAZY_CPU); + if (test_and_clear_ti_thread_flag(task_thread_info(next), TIF_LAZY_MMU_UP= DATES)) + arch_enter_lazy_mmu_mode(); } =20 static unsigned long xen_store_tr(void) @@ -472,7 +497,7 @@ static void xen_set_ldt(const void *addr, unsigned entr= ies) =20 MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); =20 - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); } =20 static void xen_load_gdt(const struct desc_ptr *dtr) @@ -568,7 +593,7 @@ static void xen_load_tls(struct thread_struct *t, unsig= ned int cpu) * exception between the new %fs descriptor being loaded and * %fs being effectively cleared at __switch_to(). */ - if (paravirt_get_lazy_mode() =3D=3D PARAVIRT_LAZY_CPU) + if (xen_get_lazy_mode() =3D=3D XEN_LAZY_CPU) loadsegment(fs, 0); =20 xen_mc_batch(); @@ -577,7 +602,7 @@ static void xen_load_tls(struct thread_struct *t, unsig= ned int cpu) load_TLS_descriptor(t, cpu, 1); load_TLS_descriptor(t, cpu, 2); =20 - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); } =20 static void xen_load_gs_index(unsigned int idx) @@ -909,7 +934,7 @@ static void xen_load_sp0(unsigned long sp0) =20 mcs =3D xen_mc_entry(0); MULTI_stack_switch(mcs.mc, __KERNEL_DS, sp0); - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); this_cpu_write(cpu_tss_rw.x86_tss.sp0, sp0); } =20 @@ -973,7 +998,7 @@ static void xen_write_cr0(unsigned long cr0) =20 MULTI_fpu_taskswitch(mcs.mc, (cr0 & X86_CR0_TS) !=3D 0); =20 - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); } =20 static void xen_write_cr4(unsigned long cr4) @@ -1156,7 +1181,7 @@ static const typeof(pv_ops) xen_cpu_ops __initconst = =3D { #endif .io_delay =3D xen_io_delay, =20 - .start_context_switch =3D paravirt_start_context_switch, + .start_context_switch =3D xen_start_context_switch, .end_context_switch =3D xen_end_context_switch, }, }; diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c index 1652c39e3dfb..b6830554ff69 100644 --- a/arch/x86/xen/mmu_pv.c +++ b/arch/x86/xen/mmu_pv.c @@ -236,7 +236,7 @@ static void xen_set_pmd_hyper(pmd_t *ptr, pmd_t val) u.val =3D pmd_val_ma(val); xen_extend_mmu_update(&u); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 preempt_enable(); } @@ -270,7 +270,7 @@ static bool xen_batched_set_pte(pte_t *ptep, pte_t ptev= al) { struct mmu_update u; =20 - if (paravirt_get_lazy_mode() !=3D PARAVIRT_LAZY_MMU) + if (xen_get_lazy_mode() !=3D XEN_LAZY_MMU) return false; =20 xen_mc_batch(); @@ -279,7 +279,7 @@ static bool xen_batched_set_pte(pte_t *ptep, pte_t ptev= al) u.val =3D pte_val_ma(pteval); xen_extend_mmu_update(&u); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 return true; } @@ -325,7 +325,7 @@ void xen_ptep_modify_prot_commit(struct vm_area_struct = *vma, unsigned long addr, u.val =3D pte_val_ma(pte); xen_extend_mmu_update(&u); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } =20 /* Assume pteval_t is equivalent to all the other *val_t types. */ @@ -419,7 +419,7 @@ static void xen_set_pud_hyper(pud_t *ptr, pud_t val) u.val =3D pud_val_ma(val); xen_extend_mmu_update(&u); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 preempt_enable(); } @@ -499,7 +499,7 @@ static void __init xen_set_p4d_hyper(p4d_t *ptr, p4d_t = val) =20 __xen_set_p4d_hyper(ptr, val); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 preempt_enable(); } @@ -531,7 +531,7 @@ static void xen_set_p4d(p4d_t *ptr, p4d_t val) if (user_ptr) __xen_set_p4d_hyper((p4d_t *)user_ptr, val); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } =20 #if CONFIG_PGTABLE_LEVELS >=3D 5 @@ -1245,7 +1245,7 @@ static noinline void xen_flush_tlb(void) op->cmd =3D MMUEXT_TLB_FLUSH_LOCAL; MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 preempt_enable(); } @@ -1265,7 +1265,7 @@ static void xen_flush_tlb_one_user(unsigned long addr) op->arg1.linear_addr =3D addr & PAGE_MASK; MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 preempt_enable(); } @@ -1302,7 +1302,7 @@ static void xen_flush_tlb_multi(const struct cpumask = *cpus, =20 MULTI_mmuext_op(mcs.mc, &args->op, 1, NULL, DOMID_SELF); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } =20 static unsigned long xen_read_cr3(void) @@ -1361,7 +1361,7 @@ static void xen_write_cr3(unsigned long cr3) else __xen_write_cr3(false, 0); =20 - xen_mc_issue(PARAVIRT_LAZY_CPU); /* interrupts restored */ + xen_mc_issue(XEN_LAZY_CPU); /* interrupts restored */ } =20 /* @@ -1396,7 +1396,7 @@ static void __init xen_write_cr3_init(unsigned long c= r3) =20 __xen_write_cr3(true, cr3); =20 - xen_mc_issue(PARAVIRT_LAZY_CPU); /* interrupts restored */ + xen_mc_issue(XEN_LAZY_CPU); /* interrupts restored */ } =20 static int xen_pgd_alloc(struct mm_struct *mm) @@ -1557,7 +1557,7 @@ static inline void xen_alloc_ptpage(struct mm_struct = *mm, unsigned long pfn, if (level =3D=3D PT_PTE && USE_SPLIT_PTE_PTLOCKS && !pinned) __pin_pagetable_pfn(MMUEXT_PIN_L1_TABLE, pfn); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } } =20 @@ -1587,7 +1587,7 @@ static inline void xen_release_ptpage(unsigned long p= fn, unsigned level) =20 __set_pfn_prot(pfn, PAGE_KERNEL); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 ClearPagePinned(page); } @@ -1804,7 +1804,7 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, un= signed long max_pfn) */ xen_mc_batch(); __xen_write_cr3(true, __pa(init_top_pgt)); - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); =20 /* We can't that easily rip out L3 and L2, as the Xen pagetables are * set out this way: [L4], [L1], [L2], [L3], [L1], [L1] ... for @@ -2083,6 +2083,23 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t= phys, pgprot_t prot) #endif } =20 +static void xen_enter_lazy_mmu(void) +{ + enter_lazy(XEN_LAZY_MMU); +} + +static void xen_flush_lazy_mmu(void) +{ + preempt_disable(); + + if (xen_get_lazy_mode() =3D=3D XEN_LAZY_MMU) { + arch_leave_lazy_mmu_mode(); + arch_enter_lazy_mmu_mode(); + } + + preempt_enable(); +} + static void __init xen_post_allocator_init(void) { pv_ops.mmu.set_pte =3D xen_set_pte; @@ -2107,7 +2124,7 @@ static void xen_leave_lazy_mmu(void) { preempt_disable(); xen_mc_flush(); - paravirt_leave_lazy_mmu(); + leave_lazy(XEN_LAZY_MMU); preempt_enable(); } =20 @@ -2166,9 +2183,9 @@ static const typeof(pv_ops) xen_mmu_ops __initconst = =3D { .exit_mmap =3D xen_exit_mmap, =20 .lazy_mode =3D { - .enter =3D paravirt_enter_lazy_mmu, + .enter =3D xen_enter_lazy_mmu, .leave =3D xen_leave_lazy_mmu, - .flush =3D paravirt_flush_lazy_mmu, + .flush =3D xen_flush_lazy_mmu, }, =20 .set_fixmap =3D xen_set_fixmap, @@ -2385,7 +2402,7 @@ static noinline void xen_flush_tlb_all(void) op->cmd =3D MMUEXT_TLB_FLUSH_ALL; MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); =20 - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); =20 preempt_enable(); } diff --git a/arch/x86/xen/multicalls.h b/arch/x86/xen/multicalls.h index 1c51b2c87f30..c3867b585e0d 100644 --- a/arch/x86/xen/multicalls.h +++ b/arch/x86/xen/multicalls.h @@ -26,7 +26,7 @@ static inline void xen_mc_batch(void) =20 /* need to disable interrupts until this entry is complete */ local_irq_save(flags); - trace_xen_mc_batch(paravirt_get_lazy_mode()); + trace_xen_mc_batch(xen_get_lazy_mode()); __this_cpu_write(xen_mc_irq_flags, flags); } =20 @@ -44,7 +44,7 @@ static inline void xen_mc_issue(unsigned mode) { trace_xen_mc_issue(mode); =20 - if ((paravirt_get_lazy_mode() & mode) =3D=3D 0) + if ((xen_get_lazy_mode() & mode) =3D=3D 0) xen_mc_flush(); =20 /* restore flags saved in xen_mc_batch */ diff --git a/include/trace/events/xen.h b/include/trace/events/xen.h index 44a3f565264d..0577f0cdd231 100644 --- a/include/trace/events/xen.h +++ b/include/trace/events/xen.h @@ -6,26 +6,26 @@ #define _TRACE_XEN_H =20 #include -#include +#include #include =20 struct multicall_entry; =20 /* Multicalls */ DECLARE_EVENT_CLASS(xen_mc__batch, - TP_PROTO(enum paravirt_lazy_mode mode), + TP_PROTO(enum xen_lazy_mode mode), TP_ARGS(mode), TP_STRUCT__entry( - __field(enum paravirt_lazy_mode, mode) + __field(enum xen_lazy_mode, mode) ), TP_fast_assign(__entry->mode =3D mode), TP_printk("start batch LAZY_%s", - (__entry->mode =3D=3D PARAVIRT_LAZY_MMU) ? "MMU" : - (__entry->mode =3D=3D PARAVIRT_LAZY_CPU) ? "CPU" : "NONE") + (__entry->mode =3D=3D XEN_LAZY_MMU) ? "MMU" : + (__entry->mode =3D=3D XEN_LAZY_CPU) ? "CPU" : "NONE") ); #define DEFINE_XEN_MC_BATCH(name) \ DEFINE_EVENT(xen_mc__batch, name, \ - TP_PROTO(enum paravirt_lazy_mode mode), \ + TP_PROTO(enum xen_lazy_mode mode), \ TP_ARGS(mode)) =20 DEFINE_XEN_MC_BATCH(xen_mc_batch); --=20 2.35.3 From nobody Tue May 14 17:59:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1694605144; cv=none; d=zohomail.com; s=zohoarc; b=dBZI1s/DHRO+jsDIHYE/59qi+bdCbNdn/27LldlP3+HcaWR4hQ6DnAdHzSQAOzmMqNjMavP/30bIpFlYb3C2ApSno78aCKYPygZTCeWaEurvTGtAYFlIdyaFIHRDwB40Xk/nCG6hSa42sRkOJ566k/gOuvzA2fBQAeQ+AuHFV3w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694605144; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=++Gf4nZevYukLnMCd8J7gk+s802IZpzJx/5E73wtImY=; b=HKBdVkrgbZsbtO7ppv6NtZWCCobT05dkL0eD92JNAaCqTw4V4MZdDmAnBdAaq6oet5ULwRH5wA5RS05++uV9zNwcIywoWB+NW3ormu67krfNU4IGNdkgiUvIsd21hJRItYkUuyIV4nldtnnFfZnljZ/BaU0/zKxC5HFO9P4dNsE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1694605144636357.8368863724937; Wed, 13 Sep 2023 04:39:04 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.601229.937214 (Exim 4.92) (envelope-from ) id 1qgOCp-0003rq-S7; Wed, 13 Sep 2023 11:38:51 +0000 Received: by outflank-mailman (output) from mailman id 601229.937214; Wed, 13 Sep 2023 11:38:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgOCp-0003rO-P7; Wed, 13 Sep 2023 11:38:51 +0000 Received: by outflank-mailman (input) for mailman id 601229; Wed, 13 Sep 2023 11:38:50 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgOCo-0003ob-H6 for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 11:38:50 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1a5c0bf3-522a-11ee-9b0d-b553b5be7939; Wed, 13 Sep 2023 13:38:48 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DE4EF1F461; Wed, 13 Sep 2023 11:38:47 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9401D13582; Wed, 13 Sep 2023 11:38:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id NCztIkefAWWVCwAAMHmgww (envelope-from ); Wed, 13 Sep 2023 11:38:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1a5c0bf3-522a-11ee-9b0d-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1694605127; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=++Gf4nZevYukLnMCd8J7gk+s802IZpzJx/5E73wtImY=; b=SHILpcKUB+qttiLi6k4sYhNzmXEFE767/y3kD9zbG2bluvZ7ghrloO9UGyNAhPubecIaKP NGNWWh5CDcoRq9haIK8xFSRVnZ0ag62gtllZXtXM6vGJ8WcYBytmyjAszMfOa3Kyr1S2Ur 7HXYBqGaBGWrqpTPsAL3X8ci5xu+xRw= From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Boris Ostrovsky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , xen-devel@lists.xenproject.org Subject: [PATCH 3/3] x86/xen: allow nesting of same lazy mode Date: Wed, 13 Sep 2023 13:38:28 +0200 Message-Id: <20230913113828.18421-4-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230913113828.18421-1-jgross@suse.com> References: <20230913113828.18421-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1694605145283100003 Content-Type: text/plain; charset="utf-8" When running as a paravirtualized guest under Xen, Linux is using "lazy mode" for issuing hypercalls which don't need to take immediate effect in order to improve performance (examples are e.g. multiple PTE changes). There are two different lazy modes defined: MMU and CPU lazy mode. Today it is not possible to nest multiple lazy mode sections, even if they are of the same kind. A recent change in memory management added nesting of MMU lazy mode sections, resulting in a regression when running as Xen PV guest. Technically there is no reason why nesting of multiple sections of the same kind of lazy mode shouldn't be allowed. So add support for that for fixing the regression. Fixes: bcc6cc832573 ("mm: add default definition of set_ptes()") Signed-off-by: Juergen Gross Reviewed-by: Boris Ostrovsky --- arch/x86/include/asm/xen/hypervisor.h | 15 +++++++++++++-- arch/x86/xen/enlighten_pv.c | 1 + 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/xen/hypervisor.h b/arch/x86/include/asm/x= en/hypervisor.h index ed05ce3df5c7..7048dfacc04b 100644 --- a/arch/x86/include/asm/xen/hypervisor.h +++ b/arch/x86/include/asm/xen/hypervisor.h @@ -72,10 +72,18 @@ enum xen_lazy_mode { }; =20 DECLARE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode); +DECLARE_PER_CPU(unsigned int, xen_lazy_nesting); =20 static inline void enter_lazy(enum xen_lazy_mode mode) { - BUG_ON(this_cpu_read(xen_lazy_mode) !=3D XEN_LAZY_NONE); + enum xen_lazy_mode old_mode =3D this_cpu_read(xen_lazy_mode); + + if (mode =3D=3D old_mode) { + this_cpu_inc(xen_lazy_nesting); + return; + } + + BUG_ON(old_mode !=3D XEN_LAZY_NONE); =20 this_cpu_write(xen_lazy_mode, mode); } @@ -84,7 +92,10 @@ static inline void leave_lazy(enum xen_lazy_mode mode) { BUG_ON(this_cpu_read(xen_lazy_mode) !=3D mode); =20 - this_cpu_write(xen_lazy_mode, XEN_LAZY_NONE); + if (this_cpu_read(xen_lazy_nesting) =3D=3D 0) + this_cpu_write(xen_lazy_mode, XEN_LAZY_NONE); + else + this_cpu_dec(xen_lazy_nesting); } =20 enum xen_lazy_mode xen_get_lazy_mode(void); diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index 54b83825c4b6..bbbfdd495ebd 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -102,6 +102,7 @@ struct tls_descs { }; =20 DEFINE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode) =3D XEN_LAZY_NONE; +DEFINE_PER_CPU(unsigned int, xen_lazy_nesting); =20 enum xen_lazy_mode xen_get_lazy_mode(void) { --=20 2.35.3