From nobody Mon Feb 9 20:35:07 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1611920749; cv=none; d=zohomail.com; s=zohoarc; b=QfezjDl/qJapJbFP4BdQLpx+TRUcjEn0WhpP8gRlGDDkGOCNHDxbhY6K2hCiG2HUX65ftG7Do3QuvX+tiSVHkLyRqDnSstXo9kh6qqqES4rFz+v/K0OV//kxbM7TJvGl9sYq18cnTw1v8O19gFz/5y+87B0TuBLVuIsU8b3PuIo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611920749; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UhomGSR9LRIN8C+S5vDrRybEDyG9mEI4O6vDeSTiQgc=; b=V6efSwPKcmQBKeJIY0CCTUUrp2ZVD3PPfmpmo3Qt5n6xhcnNozr0PsL9as7be83OpGbr+8cgoq0Sh3QPbu8giO8y1ccvfhcp68uqwn4Huz/BRL7cvK/Ey4aEFYGvx7UrL7ynA6mJOp9OP3S5weRKCwNp1u7tAFu++a3D6A0pdXA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611920749144900.7571486228809; Fri, 29 Jan 2021 03:45:49 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.78051.141818 (Exim 4.92) (envelope-from ) id 1l5SDW-0008Pk-5O; Fri, 29 Jan 2021 11:45:34 +0000 Received: by outflank-mailman (output) from mailman id 78051.141818; Fri, 29 Jan 2021 11:45:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l5SDW-0008Pd-1Q; Fri, 29 Jan 2021 11:45:34 +0000 Received: by outflank-mailman (input) for mailman id 78051; Fri, 29 Jan 2021 11:45:32 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l5SDU-0008PD-KB for xen-devel@lists.xenproject.org; Fri, 29 Jan 2021 11:45:32 +0000 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 9ac9aadc-6cfa-4114-a821-99c1510a134d; Fri, 29 Jan 2021 11:45:32 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9ac9aadc-6cfa-4114-a821-99c1510a134d DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1611920731; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rTeaVOpSmujxBmF8hy4sR35dRGwZYTO/o0Z6xBtNMK8=; b=ek/kSpdAsNNAWBu6BRWPJedv5LzzqGbxJFHT3n0KqDgfZGWz094a812o oAZZYgNBRsFA12rllZtP78wY34JHvBXKHVGiI9I3N9l6ZLsA+FY9BWPS9 UlmDLHKKFYZuEnXBRCSrmNnjH30h+XT72O7T6bhws7ee7oIfQKjhGFhFF I=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: 0M+mj86SFvJHGnfxlk5KUKwjRuAgoaln6P7EGlA/P4WTycX3vD4aWBC7D3H7K4SYFptnItGLNa LHx19Jo8t0QZyePVwD/KcSy5YEuBME9c6d/LkfxCEI8rZHcASJHH/Ei38AlKmHKzD9CLS84RAE zR4dggOth/47oyK/yTc96AoqJrNZJXzwJlT3yo96rTpFjjapADzCB68ykH/pQHvl4PWAiAL2g0 lav/SabNNJ5XX7wXkd2s1CQdny3GHL3DsTMKfdGDwxNJkqXxBDqTgjWsWUqH+WKDW7aXbEEzox SJA= X-SBRS: 5.1 X-MesageID: 37468581 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.79,385,1602561600"; d="scan'208";a="37468581" From: Andrew Cooper To: Xen-devel CC: =?UTF-8?q?Norbert=20Kami=C5=84ski?= , "Marek Kasiewicz" , Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu , Michal Zygowski , Piotr Krol , Krystian Hebel , "Daniel P . Smith" , Rich Persaud , Christopher Clark Subject: [PATCH v2 3/3] x86: Support booting under Secure Startup via SKINIT Date: Fri, 29 Jan 2021 11:45:10 +0000 Message-ID: <20210129114510.30966-1-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210115231046.31785-1-andrew.cooper3@citrix.com> References: <20210115231046.31785-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) From: Norbert Kami=C5=84ski For now, this is simply enough logic to let Xen come up after the bootloader has executed an SKINIT instruction to begin a Secure Startup. During a Secure Startup, the BSP operates with the GIF clear (blocks all external interrupts, even SMI/NMI), and INIT_REDIRECTION active (converts I= NIT IPIs to #SX exceptions, if e.g. the platform needs to scrub secrets before resetting). To afford APs the same Secure Startup protections as the BSP, = the INIT IPI must be skipped, and SIPI must be the first interrupt seen. Full details are available in AMD APM Vol2 15.27 "Secure Startup with SKINI= T" Introduce skinit_enable_intr() and call it from cpu_init(), next to the enable_nmis() which performs a related function for tboot startups. Also introduce ap_boot_method to control the sequence of actions for AP boo= t. Signed-off-by: Marek Kasiewicz Signed-off-by: Norbert Kami=C5=84ski Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Wei Liu CC: Marek Kasiewicz CC: Norbert Kami=C5=84ski CC: Michal Zygowski CC: Piotr Krol CC: Krystian Hebel CC: Daniel P. Smith CC: Rich Persaud CC: Christopher Clark v2: * Rebase over 'stgi' cleanup * Don't use wrmsr_safe(). --- xen/arch/x86/cpu/common.c | 32 ++++++++++++++++++++++++++++++++ xen/arch/x86/smpboot.c | 12 +++++++++++- xen/include/asm-x86/cpufeature.h | 1 + xen/include/asm-x86/msr-index.h | 1 + xen/include/asm-x86/processor.h | 6 ++++++ 5 files changed, 51 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c index a684519a20..cca97e4e03 100644 --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -834,6 +834,29 @@ void load_system_tables(void) BUG_ON(system_state !=3D SYS_STATE_early_boot && (stack_bottom & 0xf)); } =20 +static void skinit_enable_intr(void) +{ + uint64_t val; + + /* + * If the platform is performing a Secure Launch via SKINIT + * INIT_REDIRECTION flag will be active. + */ + if ( !cpu_has_skinit || rdmsr_safe(MSR_K8_VM_CR, val) || + !(val & VM_CR_INIT_REDIRECTION) ) + return; + + ap_boot_method =3D AP_BOOT_SKINIT; + + /* + * We don't yet handle #SX. Disable INIT_REDIRECTION first, before + * enabling GIF, so a pending INIT resets us, rather than causing a + * panic due to an unknown exception. + */ + wrmsr(MSR_K8_VM_CR, val & ~VM_CR_INIT_REDIRECTION); + asm volatile ( "stgi" ::: "memory" ); +} + /* * cpu_init() initializes state that is per-CPU. Some data is already * initialized (naturally) in the bootstrap process, such as the GDT @@ -865,6 +888,15 @@ void cpu_init(void) write_debugreg(6, X86_DR6_DEFAULT); write_debugreg(7, X86_DR7_DEFAULT); =20 + /* + * If the platform is performing a Secure Launch via SKINIT, GIF is + * clear to prevent external interrupts interfering with Secure + * Startup. Re-enable all interrupts now that we are suitably set up. + * + * Refer to AMD APM Vol2 15.27 "Secure Startup with SKINIT". + */ + skinit_enable_intr(); + /* Enable NMIs. Our loader (e.g. Tboot) may have left them disabled. */ enable_nmis(); } diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 61ce923189..82c1012e89 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -49,6 +49,7 @@ #include =20 unsigned long __read_mostly trampoline_phys; +enum ap_boot_method __read_mostly ap_boot_method =3D AP_BOOT_NORMAL; =20 /* representing HT siblings of each logical CPU */ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask); @@ -424,7 +425,16 @@ static int wakeup_secondary_cpu(int phys_apicid, unsig= ned long start_eip) { unsigned long send_status =3D 0, accept_status =3D 0; int maxlvt, timeout, i; - bool send_INIT =3D true; + + /* + * Normal AP startup uses an INIT-SIPI-SIPI sequence. + * + * When using SKINIT for Secure Startup, the INIT IPI must be skipped,= so + * that SIPI is the first interrupt the AP sees. + * + * Refer to AMD APM Vol2 15.27 "Secure Startup with SKINIT". + */ + bool send_INIT =3D ap_boot_method !=3D AP_BOOT_SKINIT; =20 /* * Some versions of tboot might be able to handle the entire wake sequ= ence diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeat= ure.h index ad3d84bdde..f62e526a96 100644 --- a/xen/include/asm-x86/cpufeature.h +++ b/xen/include/asm-x86/cpufeature.h @@ -76,6 +76,7 @@ #define cpu_has_svm boot_cpu_has(X86_FEATURE_SVM) #define cpu_has_sse4a boot_cpu_has(X86_FEATURE_SSE4A) #define cpu_has_xop boot_cpu_has(X86_FEATURE_XOP) +#define cpu_has_skinit boot_cpu_has(X86_FEATURE_SKINIT) #define cpu_has_fma4 boot_cpu_has(X86_FEATURE_FMA4) #define cpu_has_tbm boot_cpu_has(X86_FEATURE_TBM) =20 diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-inde= x.h index ff583cf0ed..1f5a5d0e38 100644 --- a/xen/include/asm-x86/msr-index.h +++ b/xen/include/asm-x86/msr-index.h @@ -117,6 +117,7 @@ #define PASID_VALID (_AC(1, ULL) << 31) =20 #define MSR_K8_VM_CR 0xc0010114 +#define VM_CR_INIT_REDIRECTION (_AC(1, ULL) << 1) #define VM_CR_SVM_DISABLE (_AC(1, ULL) << 4) =20 /* diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processo= r.h index 9acb80fdcd..d5f467d245 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -631,6 +631,12 @@ static inline uint8_t get_cpu_family(uint32_t raw, uin= t8_t *model, extern int8_t opt_tsx, cpu_has_tsx_ctrl; void tsx_init(void); =20 +enum ap_boot_method { + AP_BOOT_NORMAL, + AP_BOOT_SKINIT, +}; +extern enum ap_boot_method ap_boot_method; + #endif /* !__ASSEMBLY__ */ =20 #endif /* __ASM_X86_PROCESSOR_H */ --=20 2.11.0