From nobody Fri Jan 2 05:00:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE682C46CA1 for ; Mon, 16 Oct 2023 12:39:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233291AbjJPMj4 (ORCPT ); Mon, 16 Oct 2023 08:39:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233276AbjJPMjv (ORCPT ); Mon, 16 Oct 2023 08:39:51 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5FB1BB4 for ; Mon, 16 Oct 2023 05:39:49 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4DD4421958; Mon, 16 Oct 2023 12:39:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1697459987; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Dn2km2xrB+B5vKWOSjXaC3oIb3VD9PnFa2PlRKN8Us=; b=M7Ov8DqINADiD52TRl8/PYXfFhhBpU4i2WSWbMKAH+IiBF5C4W3TJ3t+AQ9ShNRFnVpH1P iurcbVMyuY847AEFOrrEtHbm9FtBGujgkm8gkLAEiyauSvnCMh4SZqUwh+focrVE6ewdKc 0vkAnN76iDyYl7PzC6MCHWxPpVqjHgo= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0A8E5133B7; Mon, 16 Oct 2023 12:39:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id O5JZARMvLWUbIwAAMHmgww (envelope-from ); Mon, 16 Oct 2023 12:39:47 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Peter Zijlstra Subject: [PATCH v2 2/4] x86/alternative: add indirect call patching Date: Mon, 16 Oct 2023 14:39:31 +0200 Message-Id: <20231016123933.17284-3-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231016123933.17284-1-jgross@suse.com> References: <20231016123933.17284-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Authentication-Results: smtp-out1.suse.de; none X-Spamd-Result: default: False [-0.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; REPLY(-4.00)[]; BROKEN_CONTENT_TYPE(1.50)[]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_SPAM_LONG(3.00)[1.000]; RCPT_COUNT_SEVEN(0.00)[9]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In order to prepare replacing of paravirt patching with alternative patching, add the capability to replace an indirect call with a direct one to alternative patching. This is done via a new flag ALT_FLAG_CALL as the target of the call instruction needs to be evaluated using the value of the location addressed by the indirect call. For convenience add a macro for a default call instruction. In case it is being used without the new flag being set, it will result in a BUG() when being executed. As in most cases the feature used will be X86_FEATURE_ALWAYS add another macro ALT_CALL_ALWAYS usable for the flags parameter of the ALTERNATIVE macros. For a complete replacement handle the special cases of calling a nop function and an indirect call of NULL the same way as paravirt does. Signed-off-by: Juergen Gross Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/include/asm/alternative.h | 5 ++++ arch/x86/kernel/alternative.c | 40 ++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/arch/x86/include/asm/alternative.h b/arch/x86/include/asm/alte= rnative.h index 67e50bd40bb8..dd63b96577e9 100644 --- a/arch/x86/include/asm/alternative.h +++ b/arch/x86/include/asm/alternative.h @@ -10,6 +10,9 @@ =20 #define ALT_FLAG_NOT (1 << 0) #define ALT_NOT(feature) ((ALT_FLAG_NOT << ALT_FLAGS_SHIFT) | (feature)) +#define ALT_FLAG_CALL (1 << 1) +#define ALT_CALL(feature) ((ALT_FLAG_CALL << ALT_FLAGS_SHIFT) | (feature)) +#define ALT_CALL_ALWAYS ALT_CALL(X86_FEATURE_ALWAYS) =20 #ifndef __ASSEMBLY__ =20 @@ -150,6 +153,8 @@ static inline int alternatives_text_reserved(void *star= t, void *end) } #endif /* CONFIG_SMP */ =20 +#define ALT_CALL_INSTR "call x86_BUG" + #define b_replacement(num) "664"#num #define e_replacement(num) "665"#num =20 diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 1c8dd8e05f3f..01b89a10d219 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -395,6 +395,40 @@ noinstr void x86_BUG(void) } EXPORT_SYMBOL_GPL(x86_BUG); =20 +/* + * Rewrite the "call x86_BUG" replacement to point to the target of the + * indirect pv_ops call "call *disp(%ip)". + */ +static int alt_replace_call(u8 *instr, u8 *insn_buff, struct alt_instr *a) +{ + void *target, *bug =3D &x86_BUG; + + if (a->replacementlen !=3D 5 || insn_buff[0] !=3D CALL_INSN_OPCODE) { + pr_err("Alternative: ALT_FLAG_CALL set for a non-call replacement instru= ction\n"); + pr_err(" Ignoring the flag for the instruction at %pS (%px)\n", instr, = instr); + return 5; + } + + if (a->instrlen !=3D 6 || instr[0] !=3D 0xff || instr[1] !=3D 0x15) { + pr_err("Alternative: ALT_FLAG_CALL set for unrecognized indirect call\n"= ); + pr_err(" Not replacing the instruction at %pS (%px)\n", instr, instr); + return -1; + } + + /* ff 15 00 00 00 00 call *0x0(%rip) */ + target =3D *(void **)(instr + a->instrlen + *(s32 *)(instr + 2)); + if (!target) + target =3D bug; + + /* (x86_BUG - .) + (target - x86_BUG) :=3D target - . */ + *(s32 *)(insn_buff + 1) +=3D target - bug; + + if (target =3D=3D &x86_nop) + return 0; + + return 5; +} + /* * Replace instructions with better alternatives for this CPU type. This r= uns * before SMP is initialized to avoid SMP problems with self modifying cod= e. @@ -462,6 +496,12 @@ void __init_or_module noinline apply_alternatives(stru= ct alt_instr *start, memcpy(insn_buff, replacement, a->replacementlen); insn_buff_sz =3D a->replacementlen; =20 + if (a->flags & ALT_FLAG_CALL) { + insn_buff_sz =3D alt_replace_call(instr, insn_buff, a); + if (insn_buff_sz < 0) + continue; + } + for (; insn_buff_sz < a->instrlen; insn_buff_sz++) insn_buff[insn_buff_sz] =3D 0x90; =20 --=20 2.35.3