From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370195; cv=none; d=zohomail.com; s=zohoarc; b=VmWpfa+0QBiYW6gzxWl80h4w+bUQCbMoT02cTsDSyrzefu+cyOTU4gdpT86Gm/K5ddqR08vYFP791OMZ2Dwqk/XtmC6OxqPJ7Xt/FImywBO/tb4xrLKpQL9TMbJ3zEUcPoR7pzwmIounOZmIgdG6+5FJNFT0N3zJuH3Xh3biSdE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370195; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=4YZT122JRc5fL7nppkrV93A7BiGANzdSfpXbbRGCgmE=; b=Q204MCoRAsdqn807CiN1TDm2ZNZ9T959OW19Ji9ykWW9NHdRkJWjRxDIFHyVczzMZcNK393SDW+A1uQ0T+RTRoBOqcy8x5IIoCwhK/53F0VMQ5mzLnxsWx9axRTNwNbeattBsfLqY+LszdGUtvKLkwnoytp24tKwgDDGcy1W1TY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1745370195957913.280905641345; Tue, 22 Apr 2025 18:03:15 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963780.1354743 (Exim 4.92) (envelope-from ) id 1u7OVp-0001kG-8x; Wed, 23 Apr 2025 01:02:53 +0000 Received: by outflank-mailman (output) from mailman id 963780.1354743; Wed, 23 Apr 2025 01:02:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVp-0001k2-39; Wed, 23 Apr 2025 01:02:53 +0000 Received: by outflank-mailman (input) for mailman id 963780; Wed, 23 Apr 2025 01:02:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVn-0000oH-2p for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:51 +0000 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [2a00:1450:4864:20::432]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id aa88e0c7-1fde-11f0-9eb0-5ba50f476ded; Wed, 23 Apr 2025 03:02:45 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-39c266c1389so4317299f8f.1 for ; Tue, 22 Apr 2025 18:02:45 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:43 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aa88e0c7-1fde-11f0-9eb0-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370165; x=1745974965; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4YZT122JRc5fL7nppkrV93A7BiGANzdSfpXbbRGCgmE=; b=nWczUQqGxDdDNCCmYnk9bryi/tXnXU6PjKOGgZhD5YohoeqKpt7WFAAn6h4xZJzqcB +r0Gco0nmxfctyp2WAq5ibHI+oACc977sfNWU+0/mZIRbRW1Ij0PC2wSS79MuWh4nuqN AFbsjKhFCDine4OA+Y+h7Kqea8Z1Wv5j8BsEE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370165; x=1745974965; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4YZT122JRc5fL7nppkrV93A7BiGANzdSfpXbbRGCgmE=; b=gIMBh5euoWy5jDKiGsRJeuAocPK8dlCmXWXv3h2f4xqSJ1DoOBCuCdC+adz86Ebzih n4xPDQrT7ndJYdElRBCZhzAfyARu8jFbnJCXl86jbOCQPeQmO8bBlIuET2EGRIC6CPi/ ZZyj//NNSf+NH40SneQPI5+tRGeNcZvgbQyZw/n1ID9pkjFf5K9l0FSDgiV0Duny2NoU IHhSwSENoskJKxwj4qaAkgP+BlCr0xsPmeo+n2fVTY1aByH6Amc9700giMJDCwzYRtbp yYw23SLaFQBQs+aUOGEyqfvPxBx7cY0kKGtGqtSR0IqAywMkjJKvW97uc7M9wq6AGFrv zpGA== X-Gm-Message-State: AOJu0YxBVI06aH+Mv3wh1dNXQNTzy4PN09bjMuwpUHluBsYxORMPO/UN Bcku2o8XyVlRWuydQ2u0QPtySVv4PJQz8USJJnSsm8e6kzLfjPtdqhuJdoMasd7riOO5mCdBNFE 9 X-Gm-Gg: ASbGncvW5ANDwyGRszGZfKp22qUArTq4veSmeSvCrAaFY/UnTT1rRheFXTYaP34ZaFS vMks9CEhjKjYtlxR23JuSKjRrV0/AcupE3piSmxCaeaVUcjeKoMkKDhyhUr/hH6UqXQx5rhOtSE rb5TgNBiv6bsrLPwlzfEYu3toRn38srb7HKj5Mj2DJTDH9P/1YqO8cUrDhdG5hsH+gmDxNqRLT/ S+wQtqjj9DVQVioswFf/F1CiYD+ycThQjiiFl6YzAqa+GbjfWmaKPeTvUo1c/3m7dpJMc0gzD3w Wz9/QQz6bhtO+KkJROxY4yGreUR3Wj7e2AA0YJ3YalmLCi8YRoEN2a72oJGLRQ== X-Google-Smtp-Source: AGHT+IEeZpTt1UmspNBku1CKlp8q4FyAU25r/v9pESJP2ptVpg/buA3gtThD0Ucrbd2+mu4iWWbp9w== X-Received: by 2002:a5d:5f87:0:b0:391:29f:4f87 with SMTP id ffacd0b85a97d-39efbaee69fmr12718487f8f.49.1745370164518; Tue, 22 Apr 2025 18:02:44 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Stefano Stabellini , "consulting @ bugseng . com" , Nicola Vetrini Subject: [PATCH 1/8] x86/altcall: Split alternative-call.h out of alternative.h Date: Wed, 23 Apr 2025 02:02:30 +0100 Message-Id: <20250423010237.1528582-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370197697019100 ... in preparation for changing how they're implemented. Update the MISRA deviations with the new path. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Anthony PERARD CC: Michal Orzel CC: Jan Beulich CC: Julien Grall CC: Roger Pau Monn=C3=A9 CC: Stefano Stabellini CC: consulting@bugseng.com CC: Nicola Vetrini In terms of naming, while tailcalls can technically be jumps, they're still usually reasoned about as being calls. It appears that everywhere else which needs alternative_{v,}call() gets it transitively through hvm.h --- .../eclair_analysis/ECLAIR/deviations.ecl | 4 +- .../asm/{alternative.h =3D> alternative-call.h} | 171 +----------- xen/arch/x86/include/asm/alternative.h | 262 ------------------ xen/arch/x86/include/asm/hvm/hvm.h | 2 +- xen/common/core_parking.c | 4 +- xen/include/xen/alternative-call.h | 10 +- 6 files changed, 16 insertions(+), 437 deletions(-) copy xen/arch/x86/include/asm/{alternative.h =3D> alternative-call.h} (64%) diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/= eclair_analysis/ECLAIR/deviations.ecl index 2c8fb9271391..9c67358d4663 100644 --- a/automation/eclair_analysis/ECLAIR/deviations.ecl +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl @@ -414,8 +414,8 @@ of the short-circuit evaluation strategy of such logica= l operators." -doc_end =20 -doc_begin=3D"Macros alternative_v?call[0-9] use sizeof and typeof to chec= k that the argument types match the corresponding parameter ones." --config=3DMC3A2.R13.6,reports+=3D{deliberate,"any_area(any_loc(any_exp(mac= ro(^alternative_vcall[0-9]$))&&file(^xen/arch/x86/include/asm/alternative\\= .h*$)))"} --config=3DB.UNEVALEFF,reports+=3D{deliberate,"any_area(any_loc(any_exp(mac= ro(^alternative_v?call[0-9]$))&&file(^xen/arch/x86/include/asm/alterantive\= \.h*$)))"} +-config=3DMC3A2.R13.6,reports+=3D{deliberate,"any_area(any_loc(any_exp(mac= ro(^alternative_vcall[0-9]$))&&file(^xen/arch/x86/include/asm/alternative-c= all\\.h*$)))"} +-config=3DB.UNEVALEFF,reports+=3D{deliberate,"any_area(any_loc(any_exp(mac= ro(^alternative_v?call[0-9]$))&&file(^xen/arch/x86/include/asm/alterantive-= call\\.h*$)))"} -doc_end =20 -doc_begin=3D"Anything, no matter how complicated, inside the BUILD_BUG_ON= macro is subject to a compile-time evaluation without relevant side effect= s." diff --git a/xen/arch/x86/include/asm/alternative.h b/xen/arch/x86/include/= asm/alternative-call.h similarity index 64% copy from xen/arch/x86/include/asm/alternative.h copy to xen/arch/x86/include/asm/alternative-call.h index 7326ad942836..828ea32a9625 100644 --- a/xen/arch/x86/include/asm/alternative.h +++ b/xen/arch/x86/include/asm/alternative-call.h @@ -1,165 +1,8 @@ -#ifndef __X86_ALTERNATIVE_H__ -#define __X86_ALTERNATIVE_H__ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef X86_ALTERNATIVE_CALL_H +#define X86_ALTERNATIVE_CALL_H =20 -#ifdef __ASSEMBLY__ -#include -#else - -#include -#include - -#include -#include - -struct __packed alt_instr { - int32_t orig_offset; /* original instruction */ - int32_t repl_offset; /* offset to replacement instruction */ - uint16_t cpuid; /* cpuid bit set for replacement */ - uint8_t orig_len; /* length of original instruction */ - uint8_t repl_len; /* length of new instruction */ - uint8_t pad_len; /* length of build-time padding */ - uint8_t priv; /* Private, for use by apply_alternatives() */ -}; - -#define __ALT_PTR(a,f) ((uint8_t *)((void *)&(a)->f + (a)->f)) -#define ALT_ORIG_PTR(a) __ALT_PTR(a, orig_offset) -#define ALT_REPL_PTR(a) __ALT_PTR(a, repl_offset) - -extern void add_nops(void *insns, unsigned int len); -/* Similar to alternative_instructions except it can be run with IRQs enab= led. */ -extern int apply_alternatives(struct alt_instr *start, struct alt_instr *e= nd); -extern void alternative_instructions(void); -extern void alternative_branches(void); - -#define alt_orig_len "(.LXEN%=3D_orig_e - .LXEN%=3D_orig_s)" -#define alt_pad_len "(.LXEN%=3D_orig_p - .LXEN%=3D_orig_e)" -#define alt_total_len "(.LXEN%=3D_orig_p - .LXEN%=3D_orig_s)" -#define alt_repl_s(num) ".LXEN%=3D_repl_s"#num -#define alt_repl_e(num) ".LXEN%=3D_repl_e"#num -#define alt_repl_len(num) "(" alt_repl_e(num) " - " alt_repl_s(num) ")" - -/* - * GAS's idea of true is sometimes 1 and sometimes -1, while Clang's idea - * was consistently 1 up to 6.x (it matches GAS's now). Transform it to - * uniformly 1. - */ -#define AS_TRUE(x) "((" x ") & 1)" - -#define as_max(a, b) "(("a") ^ ((("a") ^ ("b")) & -("AS_TRUE("("a") < ("b"= )")")))" - -#define OLDINSTR(oldinstr, padding) \ - ".LXEN%=3D_orig_s:\n\t" oldinstr "\n .LXEN%=3D_orig_e:\n\t" \ - ".LXEN%=3D_diff =3D " padding "\n\t" \ - "mknops ("AS_TRUE(".LXEN%=3D_diff > 0")" * .LXEN%=3D_diff)\n\t" \ - ".LXEN%=3D_orig_p:\n\t" - -#define OLDINSTR_1(oldinstr, n1) \ - OLDINSTR(oldinstr, alt_repl_len(n1) "-" alt_orig_len) - -#define OLDINSTR_2(oldinstr, n1, n2) \ - OLDINSTR(oldinstr, \ - as_max(alt_repl_len(n1), \ - alt_repl_len(n2)) "-" alt_orig_len) - -#define ALTINSTR_ENTRY(feature, num) \ - " .if " STR(feature) " >=3D " STR(NCAPINTS * 32) "\n" \ - " .error \"alternative feature outside of featureset range\"\n" \ - " .endif\n" \ - " .long .LXEN%=3D_orig_s - .\n" /* label */ \ - " .long " alt_repl_s(num)" - .\n" /* new instruction */ \ - " .word " STR(feature) "\n" /* feature bit */ \ - " .byte " alt_orig_len "\n" /* source len */ \ - " .byte " alt_repl_len(num) "\n" /* replacement len */ \ - " .byte " alt_pad_len "\n" /* padding len */ \ - " .byte 0\n" /* priv */ - -#define DISCARD_ENTRY(num) /* repl <=3D total */ \ - " .byte 0xff + (" alt_repl_len(num) ") - (" alt_total_len ")\n" - -#define ALTINSTR_REPLACEMENT(newinstr, num) /* replacement */ \ - alt_repl_s(num)":\n\t" newinstr "\n" alt_repl_e(num) ":\n\t" - -/* alternative assembly primitive: */ -#define ALTERNATIVE(oldinstr, newinstr, feature) \ - OLDINSTR_1(oldinstr, 1) \ - ".pushsection .altinstructions, \"a\", @progbits\n" \ - ALTINSTR_ENTRY(feature, 1) \ - ".section .discard, \"a\", @progbits\n" \ - ".byte " alt_total_len "\n" /* total_len <=3D 255 */ \ - DISCARD_ENTRY(1) \ - ".section .altinstr_replacement, \"ax\", @progbits\n" \ - ALTINSTR_REPLACEMENT(newinstr, 1) \ - ".popsection\n" - -#define ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2) \ - OLDINSTR_2(oldinstr, 1, 2) \ - ".pushsection .altinstructions, \"a\", @progbits\n" \ - ALTINSTR_ENTRY(feature1, 1) \ - ALTINSTR_ENTRY(feature2, 2) \ - ".section .discard, \"a\", @progbits\n" \ - ".byte " alt_total_len "\n" /* total_len <=3D 255 */ \ - DISCARD_ENTRY(1) \ - DISCARD_ENTRY(2) \ - ".section .altinstr_replacement, \"ax\", @progbits\n" \ - ALTINSTR_REPLACEMENT(newinstr1, 1) \ - ALTINSTR_REPLACEMENT(newinstr2, 2) \ - ".popsection\n" - -/* - * Alternative instructions for different CPU types or capabilities. - * - * This allows to use optimized instructions even on generic binary - * kernels. - * - * length of oldinstr must be longer or equal the length of newinstr - * It can be padded with nops as needed. - * - * For non barrier like inlines please define new variants - * without volatile and memory clobber. - */ -#define alternative(oldinstr, newinstr, feature) \ - asm volatile (ALTERNATIVE(oldinstr, newinstr, feature) : : : "memo= ry") - -#define alternative_2(oldinstr, newinstr1, feature1, newinstr2, feature2) \ - asm volatile (ALTERNATIVE_2(oldinstr, newinstr1, feature1, \ - newinstr2, feature2) \ - : : : "memory") - -/* - * Alternative inline assembly with input. - * - * Pecularities: - * No memory clobber here. - * Argument numbers start with 1. - * Best is to use constraints that are fixed size (like (%1) ... "r") - * If you use variable sized constraints like "m" or "g" in the - * replacement make sure to pad to the worst case length. - */ -#define alternative_input(oldinstr, newinstr, feature, input...) \ - asm volatile (ALTERNATIVE(oldinstr, newinstr, feature) \ - : : input) - -/* Like alternative_input, but with a single output argument */ -#define alternative_io(oldinstr, newinstr, feature, output, input...) \ - asm volatile (ALTERNATIVE(oldinstr, newinstr, feature) \ - : output : input) - -/* - * This is similar to alternative_io. But it has two features and - * respective instructions. - * - * If CPU has feature2, newinstr2 is used. - * Otherwise, if CPU has feature1, newinstr1 is used. - * Otherwise, oldinstr is used. - */ -#define alternative_io_2(oldinstr, newinstr1, feature1, newinstr2, \ - feature2, output, input...) \ - asm volatile(ALTERNATIVE_2(oldinstr, newinstr1, feature1, \ - newinstr2, feature2) \ - : output : input) - -/* Use this macro(s) if you need more than one output parameter. */ -#define ASM_OUTPUT2(a...) a +#include =20 /* * Machinery to allow converting indirect to direct calls, when the called @@ -303,7 +146,7 @@ extern void alternative_branches(void); }) =20 #define alternative_call3(func, arg1, arg2, arg3) ({ \ - typeof(arg1) v1_ =3D (arg1); \ + typeof(arg1) v1_ =3D (arg1); \ typeof(arg2) v2_ =3D (arg2); \ typeof(arg3) v3_ =3D (arg3); \ ALT_CALL_ARG(v1_, 1); \ @@ -423,6 +266,4 @@ extern void alternative_branches(void); #define alternative_call(func, args...) \ alternative_call_(count_args(args))(func, ## args) =20 -#endif /* !__ASSEMBLY__ */ - -#endif /* __X86_ALTERNATIVE_H__ */ +#endif /* X86_ALTERNATIVE_CALL_H */ diff --git a/xen/arch/x86/include/asm/alternative.h b/xen/arch/x86/include/= asm/alternative.h index 7326ad942836..2d2ace97f794 100644 --- a/xen/arch/x86/include/asm/alternative.h +++ b/xen/arch/x86/include/asm/alternative.h @@ -161,268 +161,6 @@ extern void alternative_branches(void); /* Use this macro(s) if you need more than one output parameter. */ #define ASM_OUTPUT2(a...) a =20 -/* - * Machinery to allow converting indirect to direct calls, when the called - * function is determined once at boot and later never changed. - */ - -#define ALT_CALL_arg1 "rdi" -#define ALT_CALL_arg2 "rsi" -#define ALT_CALL_arg3 "rdx" -#define ALT_CALL_arg4 "rcx" -#define ALT_CALL_arg5 "r8" -#define ALT_CALL_arg6 "r9" - -#ifdef CONFIG_CC_IS_CLANG -/* - * Clang doesn't follow the psABI and doesn't truncate parameter values at= the - * callee. This can lead to bad code being generated when using alternati= ve - * calls. - * - * Workaround it by using a temporary intermediate variable that's zeroed - * before being assigned the parameter value, as that forces clang to zero= the - * register at the caller. - * - * This has been reported upstream: - * https://github.com/llvm/llvm-project/issues/12579 - * https://github.com/llvm/llvm-project/issues/82598 - */ -#define ALT_CALL_ARG(arg, n) \ - register unsigned long a ## n ## _ asm ( ALT_CALL_arg ## n ) =3D ({ \ - unsigned long tmp =3D 0; \ - BUILD_BUG_ON(sizeof(arg) > sizeof(unsigned long)); \ - *(typeof(arg) *)&tmp =3D (arg); \ - tmp; \ - }) -#else -#define ALT_CALL_ARG(arg, n) \ - register typeof(arg) a ## n ## _ asm ( ALT_CALL_arg ## n ) =3D \ - ({ BUILD_BUG_ON(sizeof(arg) > sizeof(void *)); (arg); }) -#endif -#define ALT_CALL_NO_ARG(n) \ - register unsigned long a ## n ## _ asm ( ALT_CALL_arg ## n ) - -#define ALT_CALL_NO_ARG6 ALT_CALL_NO_ARG(6) -#define ALT_CALL_NO_ARG5 ALT_CALL_NO_ARG(5); ALT_CALL_NO_ARG6 -#define ALT_CALL_NO_ARG4 ALT_CALL_NO_ARG(4); ALT_CALL_NO_ARG5 -#define ALT_CALL_NO_ARG3 ALT_CALL_NO_ARG(3); ALT_CALL_NO_ARG4 -#define ALT_CALL_NO_ARG2 ALT_CALL_NO_ARG(2); ALT_CALL_NO_ARG3 -#define ALT_CALL_NO_ARG1 ALT_CALL_NO_ARG(1); ALT_CALL_NO_ARG2 - -/* - * Unfortunately ALT_CALL_NO_ARG() above can't use a fake initializer (to - * suppress "uninitialized variable" warnings), as various versions of gcc - * older than 8.1 fall on the nose in various ways with that (always becau= se - * of some other construct elsewhere in the same function needing to use t= he - * same hard register). Otherwise the asm() below could uniformly use "+r" - * output constraints, making unnecessary all these ALT_CALL_OUT macros. - */ -#define ALT_CALL0_OUT "=3Dr" (a1_), "=3Dr" (a2_), "=3Dr" (a3_), \ - "=3Dr" (a4_), "=3Dr" (a5_), "=3Dr" (a6_) -#define ALT_CALL1_OUT "+r" (a1_), "=3Dr" (a2_), "=3Dr" (a3_), \ - "=3Dr" (a4_), "=3Dr" (a5_), "=3Dr" (a6_) -#define ALT_CALL2_OUT "+r" (a1_), "+r" (a2_), "=3Dr" (a3_), \ - "=3Dr" (a4_), "=3Dr" (a5_), "=3Dr" (a6_) -#define ALT_CALL3_OUT "+r" (a1_), "+r" (a2_), "+r" (a3_), \ - "=3Dr" (a4_), "=3Dr" (a5_), "=3Dr" (a6_) -#define ALT_CALL4_OUT "+r" (a1_), "+r" (a2_), "+r" (a3_), \ - "+r" (a4_), "=3Dr" (a5_), "=3Dr" (a6_) -#define ALT_CALL5_OUT "+r" (a1_), "+r" (a2_), "+r" (a3_), \ - "+r" (a4_), "+r" (a5_), "=3Dr" (a6_) -#define ALT_CALL6_OUT "+r" (a1_), "+r" (a2_), "+r" (a3_), \ - "+r" (a4_), "+r" (a5_), "+r" (a6_) - -#define alternative_callN(n, rettype, func) ({ \ - rettype ret_; \ - register unsigned long r10_ asm("r10"); \ - register unsigned long r11_ asm("r11"); \ - asm volatile (ALTERNATIVE("call *%c[addr](%%rip)", "call .", \ - X86_FEATURE_ALWAYS) \ - : ALT_CALL ## n ## _OUT, "=3Da" (ret_), \ - "=3Dr" (r10_), "=3Dr" (r11_) ASM_CALL_CONSTRAINT \ - : [addr] "i" (&(func)), "g" (func) \ - : "memory" ); \ - ret_; \ -}) - -#define alternative_vcall0(func) ({ \ - ALT_CALL_NO_ARG1; \ - (void)sizeof(func()); \ - (void)alternative_callN(0, int, func); \ -}) - -#define alternative_call0(func) ({ \ - ALT_CALL_NO_ARG1; \ - alternative_callN(0, typeof(func()), func); \ -}) - -#define alternative_vcall1(func, arg) ({ \ - typeof(arg) v1_ =3D (arg); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_NO_ARG2; \ - (void)sizeof(func(arg)); \ - (void)alternative_callN(1, int, func); \ -}) - -#define alternative_call1(func, arg) ({ \ - typeof(arg) v1_ =3D (arg); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_NO_ARG2; \ - alternative_callN(1, typeof(func(arg)), func); \ -}) - -#define alternative_vcall2(func, arg1, arg2) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_NO_ARG3; \ - (void)sizeof(func(arg1, arg2)); \ - (void)alternative_callN(2, int, func); \ -}) - -#define alternative_call2(func, arg1, arg2) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_NO_ARG3; \ - alternative_callN(2, typeof(func(arg1, arg2)), func); \ -}) - -#define alternative_vcall3(func, arg1, arg2, arg3) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_NO_ARG4; \ - (void)sizeof(func(arg1, arg2, arg3)); \ - (void)alternative_callN(3, int, func); \ -}) - -#define alternative_call3(func, arg1, arg2, arg3) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_NO_ARG4; \ - alternative_callN(3, typeof(func(arg1, arg2, arg3)), \ - func); \ -}) - -#define alternative_vcall4(func, arg1, arg2, arg3, arg4) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - typeof(arg4) v4_ =3D (arg4); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_ARG(v4_, 4); \ - ALT_CALL_NO_ARG5; \ - (void)sizeof(func(arg1, arg2, arg3, arg4)); \ - (void)alternative_callN(4, int, func); \ -}) - -#define alternative_call4(func, arg1, arg2, arg3, arg4) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - typeof(arg4) v4_ =3D (arg4); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_ARG(v4_, 4); \ - ALT_CALL_NO_ARG5; \ - alternative_callN(4, typeof(func(arg1, arg2, \ - arg3, arg4)), \ - func); \ -}) - -#define alternative_vcall5(func, arg1, arg2, arg3, arg4, arg5) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - typeof(arg4) v4_ =3D (arg4); \ - typeof(arg5) v5_ =3D (arg5); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_ARG(v4_, 4); \ - ALT_CALL_ARG(v5_, 5); \ - ALT_CALL_NO_ARG6; \ - (void)sizeof(func(arg1, arg2, arg3, arg4, arg5)); \ - (void)alternative_callN(5, int, func); \ -}) - -#define alternative_call5(func, arg1, arg2, arg3, arg4, arg5) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - typeof(arg4) v4_ =3D (arg4); \ - typeof(arg5) v5_ =3D (arg5); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_ARG(v4_, 4); \ - ALT_CALL_ARG(v5_, 5); \ - ALT_CALL_NO_ARG6; \ - alternative_callN(5, typeof(func(arg1, arg2, arg3, \ - arg4, arg5)), \ - func); \ -}) - -#define alternative_vcall6(func, arg1, arg2, arg3, arg4, arg5, arg6) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - typeof(arg4) v4_ =3D (arg4); \ - typeof(arg5) v5_ =3D (arg5); \ - typeof(arg6) v6_ =3D (arg6); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_ARG(v4_, 4); \ - ALT_CALL_ARG(v5_, 5); \ - ALT_CALL_ARG(v6_, 6); \ - (void)sizeof(func(arg1, arg2, arg3, arg4, arg5, arg6)); \ - (void)alternative_callN(6, int, func); \ -}) - -#define alternative_call6(func, arg1, arg2, arg3, arg4, arg5, arg6) ({ \ - typeof(arg1) v1_ =3D (arg1); \ - typeof(arg2) v2_ =3D (arg2); \ - typeof(arg3) v3_ =3D (arg3); \ - typeof(arg4) v4_ =3D (arg4); \ - typeof(arg5) v5_ =3D (arg5); \ - typeof(arg6) v6_ =3D (arg6); \ - ALT_CALL_ARG(v1_, 1); \ - ALT_CALL_ARG(v2_, 2); \ - ALT_CALL_ARG(v3_, 3); \ - ALT_CALL_ARG(v4_, 4); \ - ALT_CALL_ARG(v5_, 5); \ - ALT_CALL_ARG(v6_, 6); \ - alternative_callN(6, typeof(func(arg1, arg2, arg3, \ - arg4, arg5, arg6)), \ - func); \ -}) - -#define alternative_vcall__(nr) alternative_vcall ## nr -#define alternative_call__(nr) alternative_call ## nr - -#define alternative_vcall_(nr) alternative_vcall__(nr) -#define alternative_call_(nr) alternative_call__(nr) - -#define alternative_vcall(func, args...) \ - alternative_vcall_(count_args(args))(func, ## args) - -#define alternative_call(func, args...) \ - alternative_call_(count_args(args))(func, ## args) - #endif /* !__ASSEMBLY__ */ =20 #endif /* __X86_ALTERNATIVE_H__ */ diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/= hvm/hvm.h index 963e8201130a..bf8bc2e100bd 100644 --- a/xen/arch/x86/include/asm/hvm/hvm.h +++ b/xen/arch/x86/include/asm/hvm/hvm.h @@ -9,9 +9,9 @@ #ifndef __ASM_X86_HVM_HVM_H__ #define __ASM_X86_HVM_HVM_H__ =20 +#include #include =20 -#include #include #include #include diff --git a/xen/common/core_parking.c b/xen/common/core_parking.c index a970ffeab8c3..7d6a18cdcf4c 100644 --- a/xen/common/core_parking.c +++ b/xen/common/core_parking.c @@ -15,10 +15,10 @@ * General Public License for more details. */ =20 -#include +#include #include -#include #include +#include #include =20 #include diff --git a/xen/include/xen/alternative-call.h b/xen/include/xen/alternati= ve-call.h index 62672b732431..39339c3f0f76 100644 --- a/xen/include/xen/alternative-call.h +++ b/xen/include/xen/alternative-call.h @@ -13,10 +13,10 @@ * * For architectures to support: * - * - Implement alternative_{,v}call() in asm/alternative.h. Code generati= on - * requirements are to emit a function pointer call at build time, and s= tash - * enough metadata to simplify the call at boot once the implementation = has - * been resolved. + * - Implement alternative_{,v}call() in asm/alternative-call.h. Code + * generation requirements are to emit a function pointer call at build + * time, and stash enough metadata to simplify the call at boot once the + * implementation has been resolved. * - Select ALTERNATIVE_CALL in Kconfig. * * To use: @@ -48,7 +48,7 @@ =20 #ifdef CONFIG_ALTERNATIVE_CALL =20 -#include +#include =20 #ifdef CONFIG_LIVEPATCH /* Must keep for livepatches to resolve alternative calls. */ --=20 2.39.5 From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370188; cv=none; d=zohomail.com; s=zohoarc; b=kl4Dwe5zbPRMZ0D6peUwyK4j9cFSlbaO2xw06SzIUuSaOqpWYahMSZeDl6OkMOHYc3hz800Y82sdKfD0m7M/IQodfeICXIEGY13zr9ltX270Gj4f9LmSVslss6sXxiRu2HhOVbDsh4t7/KcfBE8k9681hcJSfrfNJkZiW4FT3Bs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370188; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QBk2Bkks19S53TDx+/MMHfe9z/s9WQXhsbPDXWcJ6Gk=; b=cid8LxYOdEb9JDWsm8gGtEyH0a5r4viOhgMdHGffEThF6XLTBwruhPxD9DkmALyNNZO8DEY3rug7bL7utvckwSTQ4PrhdL0nU+0QWr69ZlUta5PGb+2AmQ4d14JEUmLaDruLmL3/whikjbI0gQ07reUNKfSpCjZyTNP0Jy4CWaw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1745370188308500.86113046642606; Tue, 22 Apr 2025 18:03:08 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963777.1354713 (Exim 4.92) (envelope-from ) id 1u7OVl-000129-CD; Wed, 23 Apr 2025 01:02:49 +0000 Received: by outflank-mailman (output) from mailman id 963777.1354713; Wed, 23 Apr 2025 01:02:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVl-000122-9Z; Wed, 23 Apr 2025 01:02:49 +0000 Received: by outflank-mailman (input) for mailman id 963777; Wed, 23 Apr 2025 01:02:48 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVk-0000oH-Dg for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:48 +0000 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [2a00:1450:4864:20::336]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id abab4d9f-1fde-11f0-9eb0-5ba50f476ded; Wed, 23 Apr 2025 03:02:47 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-43edecbfb94so60201785e9.1 for ; Tue, 22 Apr 2025 18:02:47 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:45 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: abab4d9f-1fde-11f0-9eb0-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370166; x=1745974966; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QBk2Bkks19S53TDx+/MMHfe9z/s9WQXhsbPDXWcJ6Gk=; b=UMwEaKw68htl2NZxRZCqHqyz5XU1JLqQmCWbhQGsW/fDSibnA9Gp/cIls2vDMLEz2c 72s+8aewG3PmAHdZCffpfkE110N7d6vOfyY77Kys5mXMlRKD4v85oXPg6SzutPfr5SBG IcJM1G8woGvnuRzPNEi0LgE9SblLHWV+u4/gI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370166; x=1745974966; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QBk2Bkks19S53TDx+/MMHfe9z/s9WQXhsbPDXWcJ6Gk=; b=iqoUWaOPgdLUNlDLyNAs+QGv5xmQqtTgaCux70X902Jt3mYsIIhskuG423nE6dNYvR kggoNBNIrotSnv4b2VObVlxZvRGYT0S3RY90ujYXIHu0UmauWug6J8I6vGLW8KrxKNoZ +hzZ+qGu2fBYna+TgKwakyJj366iwM1DTJKIHVLJITpK7wrGdNwu9rMqoey/WqaDUqEq +wNhMIZvdjBLmYHY3fGOpU9IMxwa8GEzBu1fPW6yfY43PSi+n+UiwTlBNjFaXcOqjCfE NvDbeFocVYvjvX6WKxUrkW4ZHzmGDjZQttoUc46+p+UzSX7dYRIrLHK28MiJqr24T/TQ OQCw== X-Gm-Message-State: AOJu0YyTjmQcR7nSjtLso+uDnrcshs6LzahY700bwoBCqchWTWf1LtSz xE4Qq699G+4Gdy880vaTiREUOa1G/2KRjmmQ/kTuzQR6BjOM21kXPfpqcvR6mwTLVzQ218NZdYc 1 X-Gm-Gg: ASbGncsvj/q3Pe1mPpgSGqHC0ky9jarxXiWmQRzcaXtCyG+NGzQ+XOp4pbXuWPgpgF5 b3g2uR5xsAb6jZ9QvQsbZn2zeNljmjWjUGdwKulENoWOwXrioJoqt7/qykqTFlNiHtGWRdqaJyi 6SUb1FcGmRM0Nz0d2xo57D+xiFOvp4i50HvkDdC7fWrPjjJflpW39DY8e/PuxHJxBPRWNE8xbcj c7PfgYRzgp9wzSbMuj8h7+Oi13WYTb5mVt9nJ5N9GujdzcG4c/Zl86Ch7Nk2QnH55FfnzhusvOR 2TN5JTCRNh4wyEHiEa0Jh2ZRJLQ6AqFWYSATlBgKHmTiNzpmdloCMBf9DiTmoA== X-Google-Smtp-Source: AGHT+IECRcenPSj7qe5KMmpxDDkvujUjR3cZAXi+zIeQHDXnn5RCoe4eX1JvNdayYhnQ/L4lk0wKIQ== X-Received: by 2002:a05:600c:cc4:b0:43b:ce3c:19d0 with SMTP id 5b1f17b1804b1-4406ac176ecmr146769205e9.29.1745370166523; Tue, 22 Apr 2025 18:02:46 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 2/8] x86/altcall: Rename alternative_branches() to boot_apply_alt_calls() Date: Wed, 23 Apr 2025 02:02:31 +0100 Message-Id: <20250423010237.1528582-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370191746019100 The alternatives APIs are not great; rename alternative_branches() to be mo= re precise. Centralise the declaration in xen/alternative-call.h, in the expectation that x86 won't be the only user in the long term. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 2 +- xen/arch/x86/include/asm/alternative.h | 1 - xen/arch/x86/setup.c | 3 ++- xen/include/xen/alternative-call.h | 10 +++++++++- 4 files changed, 12 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 1ba35cb9ede9..d1a3b7ea7ca6 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -493,7 +493,7 @@ void __init alternative_instructions(void) _alternative_instructions(false); } =20 -void __init alternative_branches(void) +void __init boot_apply_alt_calls(void) { local_irq_disable(); _alternative_instructions(true); diff --git a/xen/arch/x86/include/asm/alternative.h b/xen/arch/x86/include/= asm/alternative.h index 2d2ace97f794..29c3d724b07f 100644 --- a/xen/arch/x86/include/asm/alternative.h +++ b/xen/arch/x86/include/asm/alternative.h @@ -29,7 +29,6 @@ extern void add_nops(void *insns, unsigned int len); /* Similar to alternative_instructions except it can be run with IRQs enab= led. */ extern int apply_alternatives(struct alt_instr *start, struct alt_instr *e= nd); extern void alternative_instructions(void); -extern void alternative_branches(void); =20 #define alt_orig_len "(.LXEN%=3D_orig_e - .LXEN%=3D_orig_s)" #define alt_pad_len "(.LXEN%=3D_orig_p - .LXEN%=3D_orig_e)" diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index cf1ea040dd90..25189541244d 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -2082,7 +2083,7 @@ void asmlinkage __init noreturn __start_xen(void) =20 do_presmp_initcalls(); =20 - alternative_branches(); + boot_apply_alt_calls(); =20 /* * NB: when running as a PV shim VCPUOP_up/down is wired to the shim diff --git a/xen/include/xen/alternative-call.h b/xen/include/xen/alternati= ve-call.h index 39339c3f0f76..3c855bfa44f5 100644 --- a/xen/include/xen/alternative-call.h +++ b/xen/include/xen/alternative-call.h @@ -17,6 +17,8 @@ * generation requirements are to emit a function pointer call at build * time, and stash enough metadata to simplify the call at boot once the * implementation has been resolved. + * - Implement boot_apply_alt_calls() to convert the function pointer calls + * into direct calls on boot. * - Select ALTERNATIVE_CALL in Kconfig. * * To use: @@ -57,7 +59,13 @@ # define __alt_call_maybe_initdata __initdata #endif =20 -#else +/* + * Devirtualise the alternative_{,v}call()'s on boot. Convert still-NULL + * function pointers into traps. + */ +void boot_apply_alt_calls(void); + +#else /* CONFIG_ALTERNATIVE_CALL */ =20 #define alternative_call(func, args...) (func)(args) #define alternative_vcall(func, args...) (func)(args) --=20 2.39.5 From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370189; cv=none; d=zohomail.com; s=zohoarc; b=n61QBGFsiA+VdLgMDSjtDD5XP2vTiBDLYv3B2DTCdvYn1Btw0sdM9UxAP/sXzQjZ34N7Gvn7ZxKKHibbiPAhhD41eQ5Gh6XInDBtLrTxyhbxXntEc8VtMCSiyJ6nUpJDiKk2uurh0bkBK18nVcHbQ7dwhRfc4UQgFcjzK5vBn5I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370189; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Hixb/ZwuxNqQ0rmjpvM4G3/QpMDCCkyGKcIaO2DF2yg=; b=Frhn80Q5L9mbnL2+NxHPMUXbfMTWOoA61dEfqKxsKhVQMBAgQ9+vonuILyBglBv6v+9H9DLS7kaJLlbT1lWFi2doWnwgda2Xz8oeVhBszPMCOmeLk39ICwKKBb0zlLwDCmjlo6xS9MO7ohsdGsjIToQcBM2SaJEXCsMr6qc1tpU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 174537018911882.85365393200789; Tue, 22 Apr 2025 18:03:09 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963778.1354722 (Exim 4.92) (envelope-from ) id 1u7OVm-0001Fr-IX; Wed, 23 Apr 2025 01:02:50 +0000 Received: by outflank-mailman (output) from mailman id 963778.1354722; Wed, 23 Apr 2025 01:02:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVm-0001Fk-Fo; Wed, 23 Apr 2025 01:02:50 +0000 Received: by outflank-mailman (input) for mailman id 963778; Wed, 23 Apr 2025 01:02:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVl-0000oH-2l for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:49 +0000 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [2a00:1450:4864:20::329]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ac78f74a-1fde-11f0-9eb0-5ba50f476ded; Wed, 23 Apr 2025 03:02:48 +0200 (CEST) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-43cf0d787eeso67688295e9.3 for ; Tue, 22 Apr 2025 18:02:48 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:47 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ac78f74a-1fde-11f0-9eb0-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370168; x=1745974968; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hixb/ZwuxNqQ0rmjpvM4G3/QpMDCCkyGKcIaO2DF2yg=; b=RjlGaz3BAFrynbTC29EjpQtESSz2LoFW7nPfAzd5GddUurdmcIekNRaFH21URheUqh 9cMYY2b8R9EJLUOqqd7egQn9Pwond3y6PMmIqX6tenjniIO9mwrkXUHCleEXZ0FInckg nU7DnJ9TVLbmBAVLyNJd3YspSgs1E7lgkO328= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370168; x=1745974968; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hixb/ZwuxNqQ0rmjpvM4G3/QpMDCCkyGKcIaO2DF2yg=; b=t4J3nQt5YzXekkLwjvrlNz8GKOVywz8C8hKK29lQZPzXHMC1ezc7n8VJ/V2z2TcaOK 9ICeE0KusW3QoWv+cxLDCSc3lxwN08gGfy282KqB1O8yVBIlS3XiKliQWJInr5mBRqSB kMUzVbhkGBxD9H9sjqOz25VSLK8TWSOcX9Gd1VH9YPDuhg4cC78d5csC8s/RokAd2wLp 2lGlgU/7dvxzONALTZ3M4ANRRqqoFDgvw+ShoLIn6VoUpYjbraR+FHI9Q8ZhQrvGOaPF EUHqKSN11/Q5HHSAi7ODJ7p6icq/uJzak2oJf5LsqWSNgStPJsE59bE4Ix2ejznBjNSv gEtg== X-Gm-Message-State: AOJu0YwfnPxIfY0Qv4FLYGUxHqCrO5c3+eIbARphyCtMUG6/8x8wxJnp BT0tIHxknSMlqFhzssKx0AeHSuku4nRg0KqjKPFnrvsX9gjOHtNYUHW1IdFuGWLAzH0LPY6ArLx s X-Gm-Gg: ASbGnctZixydjEScJokOXtL3SGS1Tjg/eZBDO+m1Y+ChyQn+ijbtG9dqtPkzXov3peE CcFR0Ii1w/aJJXEXBQzSnpVq/hXNJhat89Ftu5iu4lb5yQgnHkgm8NnmjxybncVLBRwsSe8JcBB kgVqPxif1weX2GvvigpRz4+EYhgM4+089vDwg7JrbTUNHQBS5TlXaEdKtVPJINzhzNIK/cHOabN NJls92OibasyzfM7LDSUiJxh8uXvCkHHCF+fSvwfPP/GQKeTiqGSrt6COy2rUsdm0m2qHG/oxGx n42QrkcZaKdDXuYbcedYs7ifZG7BJrNv9pXJ/J0MLWBgoRSvQY8XYhna40HmGg== X-Google-Smtp-Source: AGHT+IGG65bPPpjp6zyCd08HRoORDUf6/LHaa7fgyod4T8zNUbNxIP+hfw/DjUz88TWXAFtevruM/w== X-Received: by 2002:a05:600c:3d96:b0:43c:f895:cb4e with SMTP id 5b1f17b1804b1-4406aba6809mr162887685e9.17.1745370167910; Tue, 22 Apr 2025 18:02:47 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 3/8] x86/alternatives: Rework information passing into nmi_apply_alternatives() Date: Wed, 23 Apr 2025 02:02:32 +0100 Message-Id: <20250423010237.1528582-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370191703019100 nmi_apply_alternatives() is soon going to need to dispatch to multiple functions, and a force parameter is not a good way of passing information. Introduce ALT_INSNS and ALT_CALLS to pass in at the top level to select the operation(s) desired. They represent what will happen when we've separated the altcalls out of the general alternative instructions infrastructure, although in the short term we still need to synthesise the force parameter = for _apply_alternatives(). Move two externs to reduce their scope a little. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index d1a3b7ea7ca6..9aa591b364a4 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -19,8 +19,6 @@ =20 #define MAX_PATCH_LEN (255-1) =20 -extern struct alt_instr __alt_instructions[], __alt_instructions_end[]; - #ifdef K8_NOP1 static const unsigned char k8nops[] init_or_livepatch_const =3D { K8_NOP1, @@ -387,9 +385,13 @@ int apply_alternatives(struct alt_instr *start, struct= alt_instr *end) } #endif =20 +#define ALT_INSNS (1U << 0) +#define ALT_CALLS (1U << 1) static unsigned int __initdata alt_todo; static unsigned int __initdata alt_done; =20 +extern struct alt_instr __alt_instructions[], __alt_instructions_end[]; + /* * At boot time, we patch alternatives in NMI context. This means that the * active NMI-shadow will defer any further NMIs, removing the slim race @@ -419,7 +421,7 @@ static int __init cf_check nmi_apply_alternatives( flush_local(FLUSH_TLB_GLOBAL); =20 rc =3D _apply_alternatives(__alt_instructions, __alt_instructions_= end, - alt_done); + alt_todo =3D=3D ALT_CALLS); if ( rc ) panic("Unable to apply alternatives: %d\n", rc); =20 @@ -442,7 +444,7 @@ static int __init cf_check nmi_apply_alternatives( * This routine is called with local interrupt disabled and used during * bootup. */ -static void __init _alternative_instructions(bool force) +static void __init _alternative_instructions(unsigned int what) { unsigned int i; nmi_callback_t *saved_nmi_callback; @@ -460,7 +462,7 @@ static void __init _alternative_instructions(bool force) ASSERT(!local_irq_is_enabled()); =20 /* Set what operation to perform /before/ setting the callback. */ - alt_todo =3D 1u << force; + alt_todo =3D what; barrier(); =20 /* @@ -490,12 +492,12 @@ static void __init _alternative_instructions(bool for= ce) void __init alternative_instructions(void) { arch_init_ideal_nops(); - _alternative_instructions(false); + _alternative_instructions(ALT_INSNS); } =20 void __init boot_apply_alt_calls(void) { local_irq_disable(); - _alternative_instructions(true); + _alternative_instructions(ALT_CALLS); local_irq_enable(); } --=20 2.39.5 From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370191; cv=none; d=zohomail.com; s=zohoarc; b=AxLyZ7jfBQ+tGI/Db5H8O4D6igEMipx0jz/r+BMudIwyc4MJmRTZ8s3htikhJUjIm43Wbv20uvnRrNige5tnZjysPK8wGE4nam5YYoIJ25hJF04+mATVPsAaBGAN6/H3AV4xYJqx0uKIzMTIbsNqh8BE7AiCFSPXFPTHY7cCuog= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370191; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=exMk98fnpxUzFTmE09OArkpIowUpunQwoLTBGbFt2Tg=; b=WJNer3CxCxkyVEnBkEPP891H6df43+tQSgMPK8B02VpZ+i5hg6Wal+d8X85VkkE1RanSVWdNA1RdWgaoY87IboyyjGZR5F90pU9btL3vDPMhzyYa7TEKz1iLXSP3jXH3w8QbOEX65LjMYtL4oESsLC5jrpSNeaKhJtoru1p1a/c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1745370191127447.7634534030441; Tue, 22 Apr 2025 18:03:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963779.1354733 (Exim 4.92) (envelope-from ) id 1u7OVn-0001U7-RL; Wed, 23 Apr 2025 01:02:51 +0000 Received: by outflank-mailman (output) from mailman id 963779.1354733; Wed, 23 Apr 2025 01:02:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVn-0001U0-OG; Wed, 23 Apr 2025 01:02:51 +0000 Received: by outflank-mailman (input) for mailman id 963779; Wed, 23 Apr 2025 01:02:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVm-0000oH-K2 for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:50 +0000 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [2a00:1450:4864:20::32c]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ad6af910-1fde-11f0-9eb0-5ba50f476ded; Wed, 23 Apr 2025 03:02:50 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-43edecbfb94so60202035e9.1 for ; Tue, 22 Apr 2025 18:02:50 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:48 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ad6af910-1fde-11f0-9eb0-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370169; x=1745974969; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=exMk98fnpxUzFTmE09OArkpIowUpunQwoLTBGbFt2Tg=; b=ceyCFnjKtT1PhgwhpZj6iqeRYcOdoBqgcbd7EpAA0KGdKmR4K4nXtXzeiCM6Ia8Hfk BfPKljDpKq3BdShTU97s+b+cOU3YIYf3IhPiW+hVj+4Mw9TT+yvvlsoqKrCClLsZFVM5 sY8otVNTKOk0Oaei7ie1GAj7MYTKpURPQPF3c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370169; x=1745974969; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=exMk98fnpxUzFTmE09OArkpIowUpunQwoLTBGbFt2Tg=; b=m33eOJmY8SNV6GBskPgY6YRRf9Z1HMu+jsaSbsyrYirUkSuJx8YPdZDOiODDa/glsJ 2WjyAwpdlFnzQHqLq96u7JCYy0LgAmPUtCi/xKGAHey0GYZpkk7QL4+D1t9Zzk53vRHe 4PsNUL1yh7C/4vLOXfvZp7KXRM+56kzsklBDOSAJxH1sck5HkR7nMkqBbsQ3uKJWYag+ KWT6X/EliVbGI4Xu1OhwJQOJ+bj3FDwDmfsz09ouzVTqhfY+ZiATPZCfOL2oD0iy7MyJ /r0z5qSoxsqmZDAsp8DB5sBtGwre1DVRMW5T1E9p8uPFgS6hidNOzadA6ZurZVkfOW8h j6Yw== X-Gm-Message-State: AOJu0YzI9JTUhrPPJPBRRiyPF7Kn7gd9AojghenHAN7bWMhPQ+AB0PHz Xz5ENCJeXK/Z5KdkLszEW0CWRpfSjJkTik/CrxZU8hmKkfqLtWjMxWFEVb7scFmXwfiNo3DeuQN 7 X-Gm-Gg: ASbGncsn9X40Vle4chA6B2czeLdV4Wcs1FM8bebTVpcGFrSvFFp1I42BvcReo3adckc eP7Y8XNxoC3e9v1t5W4w3yaTqRW2qJc8CcLRJVRVd4krievCNmqcpPYOgefy6L/1OWhs0jdLd6G ekeTq6mTVZmNc6ZxyHqKAN7Vh/SvTMZlzQt9GSuKmKIipzwBlWWgOp78df9/ghRaj/9lK9LqqZV TsTyiNgibDxNK3o6PMWZyjee1HEi5aJK+sKUVyKV5QDd0PTk6K1E/tsypZb10uFkxeh6iRpO99b 0sILBPP0rcF08zZb3CQbqzGPlUSPzbxuqSDevTsIRAdWaI0kKcpqWG6eej8law== X-Google-Smtp-Source: AGHT+IH4X0icwZPWLJVXoteY83CJDqNeTRyDblvh+veuV/K4x3dPMcKehsphiBwyufugxNA23Nt/ig== X-Received: by 2002:a05:600c:3d09:b0:43d:47b7:b32d with SMTP id 5b1f17b1804b1-4406abfad84mr136167215e9.25.1745370169420; Tue, 22 Apr 2025 18:02:49 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 4/8] x86/alternatives: Factor seal_endbr64() out of _apply_alternatives() Date: Wed, 23 Apr 2025 02:02:33 +0100 Message-Id: <20250423010237.1528582-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370191777019000 We are going to need to reposition the call in a change with several moving parts. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 70 ++++++++++++++++++++++---------------- 1 file changed, 40 insertions(+), 30 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 9aa591b364a4..4b9f8d860153 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -162,6 +162,44 @@ text_poke(void *addr, const void *opcode, size_t len) extern void *const __initdata_cf_clobber_start[]; extern void *const __initdata_cf_clobber_end[]; =20 +/* + * In CET-IBT enabled builds, clobber endbr64 instructions after altcall h= as + * finished optimising all indirect branches to direct ones. + */ +static void __init seal_endbr64(void) +{ + void *const *val; + unsigned int clobbered =3D 0; + + if ( !cpu_has_xen_ibt ) + return; + + /* + * This is some minor structure (ab)use. We walk the entire contents + * of .init.{ro,}data.cf_clobber as if it were an array of pointers. + * + * If the pointer points into .text, and at an endbr64 instruction, + * nop out the endbr64. This causes the pointer to no longer be a + * legal indirect branch target under CET-IBT. This is a + * defence-in-depth measure, to reduce the options available to an + * adversary who has managed to hijack a function pointer. + */ + for ( val =3D __initdata_cf_clobber_start; + val < __initdata_cf_clobber_end; + val++ ) + { + void *ptr =3D *val; + + if ( !is_kernel_text(ptr) || !is_endbr64(ptr) ) + continue; + + place_endbr64_poison(ptr); + clobbered++; + } + + printk("altcall: Optimised away %u endbr64 instructions\n", clobbered); +} + /* * Replace instructions with better alternatives for this CPU type. * This runs before SMP is initialized to avoid SMP problems with @@ -344,36 +382,8 @@ static int init_or_livepatch _apply_alternatives(struc= t alt_instr *start, * Clobber endbr64 instructions now that altcall has finished optimisi= ng * all indirect branches to direct ones. */ - if ( force && cpu_has_xen_ibt && system_state < SYS_STATE_active ) - { - void *const *val; - unsigned int clobbered =3D 0; - - /* - * This is some minor structure (ab)use. We walk the entire conte= nts - * of .init.{ro,}data.cf_clobber as if it were an array of pointer= s. - * - * If the pointer points into .text, and at an endbr64 instruction, - * nop out the endbr64. This causes the pointer to no longer be a - * legal indirect branch target under CET-IBT. This is a - * defence-in-depth measure, to reduce the options available to an - * adversary who has managed to hijack a function pointer. - */ - for ( val =3D __initdata_cf_clobber_start; - val < __initdata_cf_clobber_end; - val++ ) - { - void *ptr =3D *val; - - if ( !is_kernel_text(ptr) || !is_endbr64(ptr) ) - continue; - - place_endbr64_poison(ptr); - clobbered++; - } - - printk("altcall: Optimised away %u endbr64 instructions\n", clobbe= red); - } + if ( force && system_state < SYS_STATE_active ) + seal_endbr64(); =20 return 0; } --=20 2.39.5 From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370190; cv=none; d=zohomail.com; s=zohoarc; b=iQjuuX3InEjDNnj1dOxDJKk8gmS0EElYpUsbr+rxCsvH913eE7v+GT1J5g1HFS6Dlm/K88EuBRf6KpFDmdRiGUluA/yiwc/5knHf3XEI3+zk3ZwDyLyG1Grj6bU8gK2eU5pw3YK5vsK8BdZHqK+5Tn2CfxG0ERlQE6biCHS9vq0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370190; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Az5L3AwTen2X3hdYSvHBPLsXM8erZfsxm7tPT2WuzcM=; b=N89R7OiRAnkaKkLUQfZUJ9Bp5NSHyHd3CoiF/nRyK26xPg74bKplDgHREDbfVehqpPn3QFtD82h3lRYdZmclrVOWCd4kBajHkZdiHwxmNNPMOjpsvnR/EoV/0CiJ2032KP1trysAdXTr1FeiWv9vJKmx/QiwZvn4iOgLbJDxSLI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1745370190803341.36594725659006; Tue, 22 Apr 2025 18:03:10 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963781.1354753 (Exim 4.92) (envelope-from ) id 1u7OVq-0001zp-Kh; Wed, 23 Apr 2025 01:02:54 +0000 Received: by outflank-mailman (output) from mailman id 963781.1354753; Wed, 23 Apr 2025 01:02:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVq-0001zi-H1; Wed, 23 Apr 2025 01:02:54 +0000 Received: by outflank-mailman (input) for mailman id 963781; Wed, 23 Apr 2025 01:02:53 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVp-0000o7-Go for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:53 +0000 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [2a00:1450:4864:20::331]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id adfa9e1d-1fde-11f0-9ffb-bf95429c2676; Wed, 23 Apr 2025 03:02:51 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-43edecbfb94so60202185e9.1 for ; Tue, 22 Apr 2025 18:02:51 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:50 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: adfa9e1d-1fde-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370171; x=1745974971; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Az5L3AwTen2X3hdYSvHBPLsXM8erZfsxm7tPT2WuzcM=; b=MCOTSkLhYaRq+hIqlwtxj3afDYyjkDl/ZhkgXZGXQl1rgK9fer32P7F3rFFGEfvZUG XAWXyPf+ZAqxrM8nUmUzSxs+gIHzC5/gkDNlBswHnlCq/YulV4P7BVGjKs2jilI5jC6R EzFnSA6eZc4WGRYwY5tyv/NQO/Kuwb5UG2pq8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370171; x=1745974971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Az5L3AwTen2X3hdYSvHBPLsXM8erZfsxm7tPT2WuzcM=; b=kPrxHcRvufn8ysk3L1UHxAxaleiE+M/Uq8moPCiTol3ik8gxTCK1sPJy2yESgj7mNp bB6KCxBiRRe+fjvDxCuMCkYe8Ir7xcGBdjkbP/xkE3BW50OUoKJ3MisR3xWQ0absDaYG KOM3/YIYruQWzl0CS9XP/I+4sD73+aL+HoNkVExT0q3pfPHFog7PhppKS79xhfcsRark cGw9t8b6kCyRWJrGEaEVkqtBWUGtzkk7r+NZ5g1vfHpDjAJ7H9NIIdIONoGAZWyVgC4V qU8z22Vz7CKi+NAvd7u+muC8tSZZVxeEHDKAPL4p3ToPWFBLEVu/DXJ4CaW0idJLO1Bc 2+XA== X-Gm-Message-State: AOJu0Yz1ck47lV6wk7OoXlSHcBIytwXeX+OA1n7wMsD992Hh7CptNDOt EO5EmFGYAFx9aOfK451351gpgDXV9iOVFl8ohpfTbKJ98oBd1DBoZZDfJhfrSHPdOvhDQxoNsQC + X-Gm-Gg: ASbGncs9MKmwSl7CdJgMPZ/7FtIS0Xp7qNpPSXTeJiZRBVpCsDCK/AMLHhnrCc5kj8J UJxJvPHtY2vhC7iVJIg8WrDmOdLwryPZf0J1L8JPe7wqvRmYSed1HnlJ3PMv9M7No5htLHQUokh k8snJH7BdJGtqrEsGEdzs921A002NnZPnIeYo0H9sGD3bwc0Ji1LE9TY4S9bfjURWd6T8DYXbby 7Xh3qkpj8SaVC7fi18FbyR25Fow5iUL/qaVAFWnyHFSRRS6kRyxeYqo4dUyGybtOx9H3QYJ24Wa A4+n6w2zaM6loteIEsiyznSofanPejRkMUMgPh6BHvv2cdUs9+RfKY9rM210UQ== X-Google-Smtp-Source: AGHT+IEcxiy7BCMC8dQM+wTEIGNnjB9Ygk8LgBhJdkXFw8myBmCXysxUCn4C1uwaGlXg9SIIKR4tqA== X-Received: by 2002:a05:600c:1c9d:b0:43c:fb95:c76f with SMTP id 5b1f17b1804b1-4406ab93151mr166134125e9.9.1745370170847; Tue, 22 Apr 2025 18:02:50 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 5/8] x86/altcall: Introduce new simpler scheme Date: Wed, 23 Apr 2025 02:02:34 +0100 Message-Id: <20250423010237.1528582-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370191947019000 Encoding altcalls as regular alternatives leads to an unreasonable amount of complexity in _apply_alternatives(). Introduce apply_alt_calls(), and an .alt_call_sites section which simply tracks the source address (relative, to save on space). That's literally a= ll that is needed in order to devirtualise the function pointers. apply_alt_calls() is mostly as per _apply_alternatives(), except the size is known to be 6 bytes. Drop the logic for JMP *RIPREL, as there's no support for tailcall optimisations, nor a feasbile plan on how to introduce support. Pad with a redundant prefix to avoid needing a separate NOP on the end. Wire it up in nmi_apply_alternatives(), although the section is empty at th= is juncture so nothing happens in practice. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 Finding a 6-byte UD instruction that is distinct from ud2 turns out to be quite challengning. The easy way involves a length changing prefix, which = is best avoided. Suggestions for alternative patterns welcome. --- xen/arch/x86/alternative.c | 94 +++++++++++++++++++++ xen/arch/x86/include/asm/alternative-call.h | 7 ++ xen/arch/x86/xen.lds.S | 4 + 3 files changed, 105 insertions(+) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 4b9f8d860153..f6594e21a14c 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -388,6 +388,92 @@ static int init_or_livepatch _apply_alternatives(struc= t alt_instr *start, return 0; } =20 +/* + * At build time, alternative calls are emitted as: + * ff 15 xx xx xx xx =3D> call *disp32(%rip) + * + * During boot, we devirtualise by editing to: + * 2e e8 xx xx xx xx =3D> cs call disp32 + * + * or, if the function pointer is still NULL, poison to: + * 0f 0b 0f 0b 0f 0b =3D> ud2a (x3) + */ +static int init_or_livepatch apply_alt_calls( + const struct alt_call *start, const struct alt_call *end) +{ + const struct alt_call *a; + + for ( a =3D start; a < end; a++ ) + { + const uint8_t *dest; + uint8_t buf[6], *orig =3D ALT_CALL_PTR(a); + long disp; + + /* It's likely that this won't change, but check just to be safe. = */ + BUILD_BUG_ON(ALT_CALL_LEN(a) !=3D 6); + + if ( orig[0] !=3D 0xff || orig[1] !=3D 0x15 ) + { + printk(XENLOG_ERR + "Altcall for %ps [%6ph] not CALL *RIPREL\n", + orig, orig); + return -EINVAL; + } + + disp =3D *(int32_t *)(orig + 2); + dest =3D *(const void **)(orig + 6 + disp); + + if ( dest ) + { + /* + * When building for CET-IBT, all function pointer targets + * should have an endbr64 instruction. + * + * If this is not the case, leave a warning because + * something is probably wrong with the build. A CET-IBT + * enabled system might have exploded already. + * + * Otherwise, skip the endbr64 instruction. This is a + * marginal perf improvement which saves on instruction + * decode bandwidth. + */ + if ( IS_ENABLED(CONFIG_XEN_IBT) ) + { + if ( is_endbr64(dest) ) + dest +=3D ENDBR64_LEN; + else + printk(XENLOG_WARNING + "Altcall %ps dest %ps has no endbr64\n", + orig, dest); + } + + disp =3D dest - (orig + 6); + ASSERT(disp =3D=3D (int32_t)disp); + + buf[0] =3D 0x2e; + buf[1] =3D 0xe8; + *(int32_t *)(buf + 2) =3D disp; + } + else + { + /* + * The function pointer is still NULL. Seal the whole call, as + * it's not used. + */ + buf[0] =3D 0x0f; + buf[1] =3D 0x0b; + buf[2] =3D 0x0f; + buf[3] =3D 0x0b; + buf[4] =3D 0x0f; + buf[5] =3D 0x0b; + } + + text_poke(orig, buf, sizeof(buf)); + } + + return 0; +} + #ifdef CONFIG_LIVEPATCH int apply_alternatives(struct alt_instr *start, struct alt_instr *end) { @@ -401,6 +487,7 @@ static unsigned int __initdata alt_todo; static unsigned int __initdata alt_done; =20 extern struct alt_instr __alt_instructions[], __alt_instructions_end[]; +extern struct alt_call __alt_call_sites_start[], __alt_call_sites_end[]; =20 /* * At boot time, we patch alternatives in NMI context. This means that the @@ -435,6 +522,13 @@ static int __init cf_check nmi_apply_alternatives( if ( rc ) panic("Unable to apply alternatives: %d\n", rc); =20 + if ( alt_todo & ALT_CALLS ) + { + rc =3D apply_alt_calls(__alt_call_sites_start, __alt_call_site= s_end); + if ( rc ) + panic("Unable to apply alternative calls: %d\n", rc); + } + /* * Reinstate perms on .text to be RX. This also cleans out the di= rty * bits, which matters when CET Shstk is active. diff --git a/xen/arch/x86/include/asm/alternative-call.h b/xen/arch/x86/inc= lude/asm/alternative-call.h index 828ea32a9625..49a04a7cc45b 100644 --- a/xen/arch/x86/include/asm/alternative-call.h +++ b/xen/arch/x86/include/asm/alternative-call.h @@ -4,6 +4,13 @@ =20 #include =20 +/* Simply the relative position of the source call. */ +struct alt_call { + int32_t offset; +}; +#define ALT_CALL_PTR(a) ((void *)&(a)->offset + (a)->offset) +#define ALT_CALL_LEN(a) (6) + /* * Machinery to allow converting indirect to direct calls, when the called * function is determined once at boot and later never changed. diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S index d4dd6434c466..53bafc98a536 100644 --- a/xen/arch/x86/xen.lds.S +++ b/xen/arch/x86/xen.lds.S @@ -260,6 +260,10 @@ SECTIONS __alt_instructions =3D .; *(.altinstructions) __alt_instructions_end =3D .; + . =3D ALIGN(4); + __alt_call_sites_start =3D .; + *(.alt_call_sites) + __alt_call_sites_end =3D .; =20 LOCK_PROFILE_DATA =20 --=20 2.39.5 From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370194; cv=none; d=zohomail.com; s=zohoarc; b=k/zuPAe0SzKmcRp1LblR2qd2QyaaBSW6orjU1RcfJ4cysJ0OKdSe89F2KfKSzWwAxY26auFg6uclXemWu8Fk+ThiMY1f/ftX7R+vjTd91GCWLENPvVadBkHjD9ef1WG4QqlHV7D9W2YdjUKoJWZqbGjBM+1nDtaSr/3atXzJI4M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370194; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Car784Mg7m/YLG0YJpvUcaWm7zKGiPIHLzmw/ReGrpU=; b=IKH1FY/4qrfCKe5Z0ieg3V2gB1bmsNBRPm/0/rJoOEe7PadJwEatDP8YTZnaOJaatxe0Gfpv0tOvEqUMKrk4avPl2K8NYk6yNFh3vqR6LDGbe4IYsP0B+9RWbduePeKlicjZvjYJ86Ctsb17M7yfLugVdO2go3xbkSwvKHMxxAo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1745370194797806.763058843065; Tue, 22 Apr 2025 18:03:14 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963782.1354763 (Exim 4.92) (envelope-from ) id 1u7OVr-0002GJ-Td; Wed, 23 Apr 2025 01:02:55 +0000 Received: by outflank-mailman (output) from mailman id 963782.1354763; Wed, 23 Apr 2025 01:02:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVr-0002FT-Pd; Wed, 23 Apr 2025 01:02:55 +0000 Received: by outflank-mailman (input) for mailman id 963782; Wed, 23 Apr 2025 01:02:54 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVq-0000o7-Nd for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:54 +0000 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [2a00:1450:4864:20::335]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id aeb9fdbb-1fde-11f0-9ffb-bf95429c2676; Wed, 23 Apr 2025 03:02:52 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-43edecbfb94so60202285e9.1 for ; Tue, 22 Apr 2025 18:02:53 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:51 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aeb9fdbb-1fde-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370172; x=1745974972; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Car784Mg7m/YLG0YJpvUcaWm7zKGiPIHLzmw/ReGrpU=; b=CchcjBVMooWXmEQy5dnwCxDF+wmyKeycAw1nqkWwRgBpHFIrilg2DV26YWhiaUcz9x RyiT6wFA1ia9K8n3nRqHSw5ab0CjApJUCJ6UOPFLGzm07aUcwNi7a4MYltR2AYIYT4Te zlYE0qk3ubAx6lA6GvC+nv6nShUj1dU8un2n4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370172; x=1745974972; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Car784Mg7m/YLG0YJpvUcaWm7zKGiPIHLzmw/ReGrpU=; b=IWlIsdV2ZA9vC54S7wKFpnh0EOg9ige8JoWE/4zrLXBBunyLOWxZxJFXLHuBx7U09F N7+KwQk4JjKR3XTSiK493yz5kl+8yfMq/wL+Ac5eohmwBelJeHGXLh0lNn6aRm9lUy3g 6nCuwQomD6KpLcmRGzyOrevVRJj8qzbDW5KwGPXNDrkP4bQOtI3W25fO0Cf+9rx3pwVA tbUPczEpjqbPWzgRW1V148suNctEi6o3fboWQnQ4GOckV4SoNxHpy28E0wCqvTJw2Ya0 HMJIelRMcV0ONuNSXypacuj2p4DJJriMJ67IKAPidrKtZpywSsYj6bnVDF7Y14X9rsAA sBrA== X-Gm-Message-State: AOJu0YzHtA6qcuZWK/GoASWXZ94QpUysqzqUAEeW7wVG8xKO+xYl6g8u N9QXevOOjc0y2/WtQP5Q7V7udc4Sqvyo+Kh0OBjQCY1ATZucqNox8Y06DRV8msGcKhACBKHZ28M E X-Gm-Gg: ASbGncuLUJS7CDpOfRIHujcEBHt1QivF46JqZdWqGtFtTAV/o47/3lvFg8BC30sXQBM B+cgXKK0f/9F9kjqhgUgKxhN2Ce7bStqNEXpnqJyWbT4h2i+wdGvr8RWG3E6qE4eDfE2PkZrb1v VaQ5tWNSI4pAH4vF33YG7cC8pH0SdaBALxEPxgh7dh/z+RmLrtjLC2vGqE0gcP9dBKM86sLJvJs tfcTklBzb6NOM485X1WB3MM5LvCW5bJy+MgFJqdfIFN047GVuboLBzNgwShgXGmh+aEhWhi20vI RH1Y24GJ/Uiny86SgfcGUzM5ewmWbbBpMnOqcmXClnmaTkRKXkhAJ+8y7HjuzA== X-Google-Smtp-Source: AGHT+IHCwz6HUV+NnETadrfzXBEL0LGCaI1Bo4OwY8mvAP0xChdNkPdYskXTtgnH8JOWTKFH6CTRLw== X-Received: by 2002:a05:600c:1c28:b0:440:6a37:be30 with SMTP id 5b1f17b1804b1-4406aba5c25mr143876665e9.16.1745370172253; Tue, 22 Apr 2025 18:02:52 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Ross Lagerwall Subject: [PATCH 6/8] xen/livepatch: Support new altcall scheme Date: Wed, 23 Apr 2025 02:02:35 +0100 Message-Id: <20250423010237.1528582-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370195597019000 The new altcall scheme uses an .alt_call_sites section. Wire this up in ve= ry much the same way as the .altinstructions section, although there is less sanity checking necessary. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Ross Lagerwall --- xen/arch/x86/alternative.c | 6 ++++ xen/common/livepatch.c | 58 ++++++++++++++++++++++++++++++ xen/include/xen/alternative-call.h | 8 +++-- 3 files changed, 70 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index f6594e21a14c..22af224f08f7 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -479,6 +479,12 @@ int apply_alternatives(struct alt_instr *start, struct= alt_instr *end) { return _apply_alternatives(start, end, true); } + +int livepatch_apply_alt_calls(const struct alt_call *start, + const struct alt_call *end) +{ + return apply_alt_calls(start, end); +} #endif =20 #define ALT_INSNS (1U << 0) diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c index 6ce77bf021b7..be9b7e367553 100644 --- a/xen/common/livepatch.c +++ b/xen/common/livepatch.c @@ -905,6 +905,64 @@ static int prepare_payload(struct payload *payload, #endif } =20 + sec =3D livepatch_elf_sec_by_name(elf, ".alt_call_sites"); + if ( sec ) + { +#ifdef CONFIG_ALTERNATIVE_CALL + const struct alt_call *a, *start, *end; + + if ( !section_ok(elf, sec, sizeof(*a)) ) + return -EINVAL; + + /* Tolerate an empty .alt_call_sites section... */ + if ( sec->sec->sh_size =3D=3D 0 ) + goto alt_call_done; + + /* ... but otherwise, there needs to be something to alter... */ + if ( payload->text_size =3D=3D 0 ) + { + printk(XENLOG_ERR LIVEPATCH "%s Alternative calls provided, bu= t no .text\n", + elf->name); + return -EINVAL; + } + + start =3D sec->addr; + end =3D sec->addr + sec->sec->sh_size; + + for ( a =3D start; a < end; a++ ) + { + const void *orig =3D ALT_CALL_PTR(a); + size_t len =3D ALT_CALL_LEN(a); + + /* orig must be fully within .text. */ + if ( orig < payload->text_addr || + len > payload->text_size || + orig + len > payload->text_addr + payload->text_size ) + { + printk(XENLOG_ERR LIVEPATCH + "%s: Alternative call %p+%#zx outside payload text = %p+%#zx\n", + elf->name, orig, len, + payload->text_addr, payload->text_size); + return -EINVAL; + } + } + + rc =3D livepatch_apply_alt_calls(start, end); + if ( rc ) + { + printk(XENLOG_ERR LIVEPATCH "%s: Applying alternative calls fa= iled: %d\n", + elf->name, rc); + return rc; + } + + alt_call_done:; +#else /* CONFIG_ALTERNATIVE_CALL */ + printk(XENLOG_ERR LIVEPATCH "%s: Alternative calls not supported\n= ", + elf->name); + return -EOPNOTSUPP; +#endif /* !CONFIG_ALTERNATIVE_CALL */ + } + sec =3D livepatch_elf_sec_by_name(elf, ".ex_table"); if ( sec ) { diff --git a/xen/include/xen/alternative-call.h b/xen/include/xen/alternati= ve-call.h index 3c855bfa44f5..767c2149bce7 100644 --- a/xen/include/xen/alternative-call.h +++ b/xen/include/xen/alternative-call.h @@ -17,8 +17,8 @@ * generation requirements are to emit a function pointer call at build * time, and stash enough metadata to simplify the call at boot once the * implementation has been resolved. - * - Implement boot_apply_alt_calls() to convert the function pointer calls - * into direct calls on boot. + * - Implement {boot,livepatch}_apply_alt_calls() to convert the function + * pointer calls into direct calls on boot/livepatch. * - Select ALTERNATIVE_CALL in Kconfig. * * To use: @@ -65,6 +65,10 @@ */ void boot_apply_alt_calls(void); =20 +/* As per boot_apply_alt_calls() but for a livepatch. */ +int livepatch_apply_alt_calls(const struct alt_call *start, + const struct alt_call *end); + #else /* CONFIG_ALTERNATIVE_CALL */ =20 #define alternative_call(func, args...) (func)(args) --=20 2.39.5 From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370189; cv=none; d=zohomail.com; s=zohoarc; b=EzVhazeouCD5qgIf8U3Irj1Ssk2SVwBYPNPdcWOUKLpQddDF2ljbjWvjzaAEVXnNgJ3MixbZgVRSo0DS3KE2sBp0XeokAtAT8HCL+ISdM2aVM89zIWAW/L81r5Wl+ErpJQL6qqERGjLP/9w45Aj/9kn/SYDj13rO8Z6HUQpaSYY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370189; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LoWKMqHtqBPDLmEPFfxaswFqTNu87oQfZQ9HmGyS6OI=; b=a85pETcugbpHMkHtqzLLFffdaL4LCRgpmYSP2PtDBfKyuEhD/Ga/pAloL3l5D81qtRCOvDGRd+LbIN7Wii448IDdrhOfNVcfeOyNACqZbQTDBVj1I124rS7ousFnFXBaA6v4pPaGiaJkzRbF5z03rX5uQ59K1cDaNwjte9df68k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1745370189433282.1622340828353; Tue, 22 Apr 2025 18:03:09 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963783.1354773 (Exim 4.92) (envelope-from ) id 1u7OVt-0002XL-8z; Wed, 23 Apr 2025 01:02:57 +0000 Received: by outflank-mailman (output) from mailman id 963783.1354773; Wed, 23 Apr 2025 01:02:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVt-0002WI-3z; Wed, 23 Apr 2025 01:02:57 +0000 Received: by outflank-mailman (input) for mailman id 963783; Wed, 23 Apr 2025 01:02:55 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVr-0000o7-QY for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:55 +0000 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [2a00:1450:4864:20::334]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id af6e1111-1fde-11f0-9ffb-bf95429c2676; Wed, 23 Apr 2025 03:02:53 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-43d04dc73b7so53142435e9.3 for ; Tue, 22 Apr 2025 18:02:54 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:52 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: af6e1111-1fde-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370173; x=1745974973; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LoWKMqHtqBPDLmEPFfxaswFqTNu87oQfZQ9HmGyS6OI=; b=mOg5wIgTIonFrTfHloaskb6MhkA1TkBb4XO66TKPJI/wApelk4ONDdGXVWpCMvreDL +XL3BvtdEwfrwjc7iqjwfrbe+6zdIZ6cXTGJT9KZkqUNhN1iDn2BfEGL5E2sYOapIu1B cuwEDGwvyOmXEDZDB8p/NzfEQvYCFSEIUg+T0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370173; x=1745974973; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LoWKMqHtqBPDLmEPFfxaswFqTNu87oQfZQ9HmGyS6OI=; b=rWGTj8d5MmMtMmIb8yDLy36lQ2xtAFwAC2lOY/PyVTvSyr+xoKGD47bFeD4Zk49+lP +M5jKLvKM3JxaWXwrIBOzQvj3E/xRAVrlXE1+AvH/eUC5aw75DpjUxiVZARU7Sfe8ynX AoLDSLauxWktIKBIOcB/xRdg2guGs+Mh6GrD1i/AhD4zOEdxMQCTsKfERbSG1ORCpREB ps5lowLu/cN7NnQfYeu7/m4c/89vgodtrLH36v3/pQ5claB7Nn5dydAwPSEWYbRmC7vv zYxqlAci1frDHpbhPQ38x4BS+Mw7Ryamq0AulMrocjQCTZqamDCLzTHozu+StwTKOvb8 0yRw== X-Gm-Message-State: AOJu0Yxc4xK0/imT1ECML5QNZ6ZnWWIL2mCB3OWv4pn35vzGtSBQiv5I CAWi+Y3wYxNkWKiWQ9HCJMEtscm+0yYyPf/7+nbUr/4IFdDSS8oBiwMwWbPbrKoEi7dEZiWlOuv / X-Gm-Gg: ASbGncvC/j8oJW6JUn2/IOsbhfzmrCqxpPuo4RGiSo1vGMIL9ad8EX/FyB30NrD4c0V e4iMNFso2jnDSvuUwrbyts1Ha9DuC7bqeiPFNazsr2gtQ1zUXJEyTCwYrcEnk1AVKFnKKXja+w1 e3dHEcU88WNwAw+fj90PwSTSQfDpIqU85xoE6258O6UFonGpIs5aQCa4hdsjgqWvR+aZ8o6W3Lk Rszpjc60v2LVAPsHUpIRMKiM2kaUm5qls6NPFPVnbJm5nWat69lilGQfdpKgaDGRLQy4b8WpfCo SFnosVIzMk6lF+wBnC+uIKfLpai08Vzp3jXrkTztkZoKu/csqjA0T2g5CnDofA== X-Google-Smtp-Source: AGHT+IHzp4hNlxBfSm5YRXWG9V+oQpZjMac/1ALmOGUgU+tXdqLeroMdU4JaXHiFRcvBaTqZatJL9A== X-Received: by 2002:a05:600c:5110:b0:43c:e70d:44f0 with SMTP id 5b1f17b1804b1-4406aba758cmr134644925e9.19.1745370173253; Tue, 22 Apr 2025 18:02:53 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 7/8] x86/altcall: Switch to simpler scheme Date: Wed, 23 Apr 2025 02:02:36 +0100 Message-Id: <20250423010237.1528582-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370191778019000 With all the infrastructure in place, switch from using ALTERNATIVE() to simply populating .alt_call_sites. Before, _apply_alternatives() would devirtualise in two passes; the first being opportunistic, and the second (signified by the force parameter) seal= ing any call with a still-NULL function pointer. Now, all devirtualising is performed together, at the point in time of the second pass previously. The call to seal_endbr64() needs delaying until af= ter apply_alt_calls() is complete, or we have a narrow window with real indirect branches and no ENDBR64 instructions. Under the hood, the following changes are happening: Section Old size New size Change (%) .alt_call_sites 0 0x00730 +0x0730 .altinstructions 0x1350a 0x11fe0 -0x152a (-7%) .altinstr_replacement 0x015f2 0x00e35 -0x07bd (-23%) The changes aren't quite equal because inlining is affected by the smaller asm() block. Nevertheless, the metadata is held in 1/3 of the space, and there are no CALL instructions held in the replacement section any more. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 9 ++------- xen/arch/x86/include/asm/alternative-call.h | 9 ++++++--- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 22af224f08f7..047bfc6e424b 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -378,13 +378,6 @@ static int init_or_livepatch _apply_alternatives(struc= t alt_instr *start, text_poke(orig, buf, total_len); } =20 - /* - * Clobber endbr64 instructions now that altcall has finished optimisi= ng - * all indirect branches to direct ones. - */ - if ( force && system_state < SYS_STATE_active ) - seal_endbr64(); - return 0; } =20 @@ -533,6 +526,8 @@ static int __init cf_check nmi_apply_alternatives( rc =3D apply_alt_calls(__alt_call_sites_start, __alt_call_site= s_end); if ( rc ) panic("Unable to apply alternative calls: %d\n", rc); + + seal_endbr64(); } =20 /* diff --git a/xen/arch/x86/include/asm/alternative-call.h b/xen/arch/x86/inc= lude/asm/alternative-call.h index 49a04a7cc45b..bbc49a5274d9 100644 --- a/xen/arch/x86/include/asm/alternative-call.h +++ b/xen/arch/x86/include/asm/alternative-call.h @@ -2,7 +2,8 @@ #ifndef X86_ALTERNATIVE_CALL_H #define X86_ALTERNATIVE_CALL_H =20 -#include +#include +#include =20 /* Simply the relative position of the source call. */ struct alt_call { @@ -86,8 +87,10 @@ struct alt_call { rettype ret_; \ register unsigned long r10_ asm("r10"); \ register unsigned long r11_ asm("r11"); \ - asm volatile (ALTERNATIVE("call *%c[addr](%%rip)", "call .", \ - X86_FEATURE_ALWAYS) \ + asm volatile ("1: call *%c[addr](%%rip)\n\t" \ + ".pushsection .alt_call_sites, \"a\", @progbits\n\t" \ + ".long 1b - .\n\t" \ + ".popsection" \ : ALT_CALL ## n ## _OUT, "=3Da" (ret_), \ "=3Dr" (r10_), "=3Dr" (r11_) ASM_CALL_CONSTRAINT \ : [addr] "i" (&(func)), "g" (func) \ --=20 2.39.5 From nobody Fri Dec 19 17:25:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1745370199; cv=none; d=zohomail.com; s=zohoarc; b=m9PnCIKhXw+aweJ3ElzmmxbxKOjR5C/8c2LMVmP6VrWWES48piqdR/F0+hOyhTLsNUsigwN33frYLrF6oHn0U8tltJGLziLtvHGAzLBNG04f+dgsHFOCsGhsrGFfPiiYbHCqo6HK1aBo1D/1QDrGY7tqdWRvvZMyNRrOSP06WGQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745370199; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WM99xQkfEqKEfaoeOHiMOk1AMVGIEVl2DOVVorR/k20=; b=btNZgrZMRumAp0CvSsZDFNR1Qj2NvFNZsUDvLceVwqkHGl4U0lGOisYEFLZeRq71LNuqLC5Vnsvmmbd5R7HBPidPJttCnAB375/xCSB1LO6MjRdePer6WrirMeEZKP3PZ039ctf5G+QussTudKzHdXDmv1sRc2VGb1GE9suxBxE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1745370199851966.2195090477793; Tue, 22 Apr 2025 18:03:19 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.963784.1354783 (Exim 4.92) (envelope-from ) id 1u7OVu-0002pR-Kp; Wed, 23 Apr 2025 01:02:58 +0000 Received: by outflank-mailman (output) from mailman id 963784.1354783; Wed, 23 Apr 2025 01:02:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVu-0002oc-Ef; Wed, 23 Apr 2025 01:02:58 +0000 Received: by outflank-mailman (input) for mailman id 963784; Wed, 23 Apr 2025 01:02:56 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1u7OVs-0000o7-Rt for xen-devel@lists.xenproject.org; Wed, 23 Apr 2025 01:02:56 +0000 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [2a00:1450:4864:20::331]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id b003983b-1fde-11f0-9ffb-bf95429c2676; Wed, 23 Apr 2025 03:02:54 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-43d04ea9d9aso27507465e9.3 for ; Tue, 22 Apr 2025 18:02:55 -0700 (PDT) Received: from localhost.localdomain ([83.104.178.215]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2326dsm6707635e9.15.2025.04.22.18.02.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 18:02:53 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b003983b-1fde-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1745370174; x=1745974974; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WM99xQkfEqKEfaoeOHiMOk1AMVGIEVl2DOVVorR/k20=; b=VppJHc+7D/gKKgSwwvk+7RDIx46S06vNNcBcAuaTFHFmm0EdFrD8CtqpxHNDWpYD89 E4AnyiRonMlcp9qGlFOlMlm3GUp4ZLCaLnf4MwYgquYPQeVIgE3wQh/lhSl0NpMrejNi TBrgD9HMKSrFgFzNVLFjK0LaKucs4l7ZXKYjg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745370174; x=1745974974; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WM99xQkfEqKEfaoeOHiMOk1AMVGIEVl2DOVVorR/k20=; b=aqPB7su9SuuOwIlDu0nNAazsTcrIYQUkqIeittdvrAdXBhsvoc8ScxIA9+v6AIn7Uh 9cMXnibpFFP4Pzg2PlkIthmhqfPZyPyHwZlyO0TjOyoDnmJL4PTnRz4m0o7cJQP3tpN2 EnxGaKEjfiNqbfXbRkdtkvpQnCCtkm6JSWOlLXXp+lAdjP7zSzgt/eJkX+qM0iF56Hk4 hLqypc+2Frsg7p4U1B0Lz1rbnqZL44AQtZ9cxLtVUe9Z5O9i0r5ogn23Wpks6RUmvt1a ahyQT0Rx7jf684n94hO4D8D2q8kTj+uUaIiuWEY0vbHLCVd1UUR5jeuNiZHDLBzwhaNP 5Hug== X-Gm-Message-State: AOJu0YxqXJOXd2Ey3OCBC/22oqozdpYU/bPEuUbJxD1/BRKUTza5CQdY /jZvFFmaBBzlJKCol9bNcNjRBX+ADcPftiohs25RCSq4m1bCJVKdBdHHo/MEXVSnoIINgYb5ssh R X-Gm-Gg: ASbGncufDOUeZgobZ7NfLmyzc77nDjd0WASrtCp3ZoOvwk9ZTz2LywQH53Ny253exVW X3dU9TQ7kwRaSmH0Ht2CjNdCKozym9ADqekRmOieJdRmBhjfMAPgGrez1XQsf69pQSwmFjUq77q tYlytmNXvPo527snM/QCpdLfF+EqGgX8Pq24LrQAxEN2anXhaWDAvZFuGSE4Qh7ro/OM2MWwSkZ 6fGjxSpVpkj8Q4gvHVVFRkxK7i9tQ8qPukCgt2MTQc7KBMSzCmGlKMBHPxNuWGi4kJ8PXK5IKv7 5kOvznwVl5EgsCAlo98XGirT6ygdcHoECAPTRgVeUSAHCU42Y4+1q9DvJHo8oA== X-Google-Smtp-Source: AGHT+IFn46uQ6sYjyAjEfhBOcPUByRdj/eRnE8QJcm+B9LeRRzCCKUSJ9COZ57h4R/MfMJ4dQ+GmlA== X-Received: by 2002:a05:600c:a13:b0:43d:46de:b0eb with SMTP id 5b1f17b1804b1-4408ab25c8amr54880075e9.12.1745370174407; Tue, 22 Apr 2025 18:02:54 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 8/8] x86/alternatives: Simplify _apply_alternatives() now altcall is separate Date: Wed, 23 Apr 2025 02:02:37 +0100 Message-Id: <20250423010237.1528582-9-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250423010237.1528582-1-andrew.cooper3@citrix.com> References: <20250423010237.1528582-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1745370202324019100 With altcall handled separately, the special case in _apply_alternatives() = is unused and can be dropped. The force parameter (used to signify the seal pass) can be removed too. In turn, nmi_apply_alternatives() no longer needs to call _apply_alternatives() on the second pass. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 94 ++++---------------------------------- 1 file changed, 10 insertions(+), 84 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 047bfc6e424b..43b009888c02 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -206,14 +206,9 @@ static void __init seal_endbr64(void) * self modifying code. This implies that asymmetric systems where * APs have less capabilities than the boot processor are not handled. * Tough. Make sure you disable such features by hand. - * - * The caller will set the "force" argument to true for the final - * invocation, such that no CALLs/JMPs to NULL pointers will be left - * around. See also the further comment below. */ static int init_or_livepatch _apply_alternatives(struct alt_instr *start, - struct alt_instr *end, - bool force) + struct alt_instr *end) { struct alt_instr *a, *base; =20 @@ -274,10 +269,7 @@ static int init_or_livepatch _apply_alternatives(struc= t alt_instr *start, =20 /* Skip patch sites already handled during the first pass. */ if ( a->priv ) - { - ASSERT(force); continue; - } =20 /* If there is no replacement to make, see about optimising the no= ps. */ if ( !boot_cpu_has(a->cpuid) ) @@ -301,76 +293,7 @@ static int init_or_livepatch _apply_alternatives(struc= t alt_instr *start, =20 /* 0xe8/0xe9 are relative branches; fix the offset. */ if ( a->repl_len >=3D 5 && (*buf & 0xfe) =3D=3D 0xe8 ) - { - /* - * Detect the special case of indirect-to-direct branch patchi= ng: - * - replacement is a direct CALL/JMP (opcodes 0xE8/0xE9; alre= ady - * checked above), - * - replacement's displacement is -5 (pointing back at the ve= ry - * insn, which makes no sense in a real replacement insn), - * - original is an indirect CALL/JMP (opcodes 0xFF/2 or 0xFF/= 4) - * using RIP-relative addressing. - * Some branch destinations may still be NULL when we come here - * the first time. Defer patching of those until the post-pres= mp- - * initcalls re-invocation (with force set to true). If at that - * point the branch destination is still NULL, insert "UD2; UD= 0" - * (for ease of recognition) instead of CALL/JMP. - */ - if ( a->cpuid =3D=3D X86_FEATURE_ALWAYS && - *(int32_t *)(buf + 1) =3D=3D -5 && - a->orig_len >=3D 6 && - orig[0] =3D=3D 0xff && - orig[1] =3D=3D (*buf & 1 ? 0x25 : 0x15) ) - { - long disp =3D *(int32_t *)(orig + 2); - const uint8_t *dest =3D *(void **)(orig + 6 + disp); - - if ( dest ) - { - /* - * When building for CET-IBT, all function pointer tar= gets - * should have an endbr64 instruction. - * - * If this is not the case, leave a warning because - * something is probably wrong with the build. A CET-= IBT - * enabled system might have exploded already. - * - * Otherwise, skip the endbr64 instruction. This is a - * marginal perf improvement which saves on instruction - * decode bandwidth. - */ - if ( IS_ENABLED(CONFIG_XEN_IBT) ) - { - if ( is_endbr64(dest) ) - dest +=3D ENDBR64_LEN; - else - printk(XENLOG_WARNING - "altcall %ps dest %ps has no endbr64\n", - orig, dest); - } - - disp =3D dest - (orig + 5); - ASSERT(disp =3D=3D (int32_t)disp); - *(int32_t *)(buf + 1) =3D disp; - } - else if ( force ) - { - buf[0] =3D 0x0f; - buf[1] =3D 0x0b; - buf[2] =3D 0x0f; - buf[3] =3D 0xff; - buf[4] =3D 0xff; - } - else - continue; - } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); - else - *(int32_t *)(buf + 1) +=3D repl - orig; - } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); + *(int32_t *)(buf + 1) +=3D repl - orig; =20 a->priv =3D 1; =20 @@ -470,7 +393,7 @@ static int init_or_livepatch apply_alt_calls( #ifdef CONFIG_LIVEPATCH int apply_alternatives(struct alt_instr *start, struct alt_instr *end) { - return _apply_alternatives(start, end, true); + return _apply_alternatives(start, end); } =20 int livepatch_apply_alt_calls(const struct alt_call *start, @@ -516,10 +439,13 @@ static int __init cf_check nmi_apply_alternatives( PAGE_HYPERVISOR_RWX); flush_local(FLUSH_TLB_GLOBAL); =20 - rc =3D _apply_alternatives(__alt_instructions, __alt_instructions_= end, - alt_todo =3D=3D ALT_CALLS); - if ( rc ) - panic("Unable to apply alternatives: %d\n", rc); + if ( alt_todo & ALT_INSNS ) + { + rc =3D _apply_alternatives(__alt_instructions, + __alt_instructions_end); + if ( rc ) + panic("Unable to apply alternatives: %d\n", rc); + } =20 if ( alt_todo & ALT_CALLS ) { --=20 2.39.5