From nobody Sun May 19 06:23:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1713809707; cv=none; d=zohomail.com; s=zohoarc; b=NTTckVmje104Acz3X/hA8AWEwfw0F63QpFvgnw9HfgGsTR0mwIi29QJ/L/iHeU4/qGMsOqm6IGdA+34mWz1VsXyjqUHuKmnvN8mEQidMA9Y2+2ahfcQjZZDR+0tNGmhgR190U0jZXS9jZ1taer4Ur31mx5jZQPaVI0YGohwF56c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1713809707; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6RRljp8ilqBOCq7+riUQu2IxAcNGdCvaCKnCfHGbuws=; b=PRCPnCJb2VnkbL8C4rMK1BdkHdR7EJ26iLLj5/KcEqkqwdMXBhf8fNEB4dpEKS2vxcBCv1joCQjyX9q0NkOv3VPVP4A1p0pwcIak1KDfl/BUAid90AtIwUSHgeQMKNvmBo8chEqoL5L/ufd+sZMKroJq3j8uANFoY1CsluT/0nM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1713809707315672.4978871560098; Mon, 22 Apr 2024 11:15:07 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.710193.1109312 (Exim 4.92) (envelope-from ) id 1ryyBe-0000iX-TO; Mon, 22 Apr 2024 18:14:42 +0000 Received: by outflank-mailman (output) from mailman id 710193.1109312; Mon, 22 Apr 2024 18:14:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBe-0000g7-N5; Mon, 22 Apr 2024 18:14:42 +0000 Received: by outflank-mailman (input) for mailman id 710193; Mon, 22 Apr 2024 18:14:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBd-0000Oz-FA for xen-devel@lists.xenproject.org; Mon, 22 Apr 2024 18:14:41 +0000 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [2a00:1450:4864:20::530]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 2eaaf11d-00d4-11ef-b4bb-af5377834399; Mon, 22 Apr 2024 20:14:38 +0200 (CEST) Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-56e136cbcecso6788378a12.3 for ; Mon, 22 Apr 2024 11:14:38 -0700 (PDT) Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id z15-20020a170906668f00b00a51d073da7esm5994224ejo.82.2024.04.22.11.14.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 11:14:37 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2eaaf11d-00d4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1713809678; x=1714414478; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6RRljp8ilqBOCq7+riUQu2IxAcNGdCvaCKnCfHGbuws=; b=u73wAZeaC42onoCzpA2HsXTCOhDXT4Y7CH8on5ZJOI8H5KnR91YSaSgn0cO63+pCc8 jL/8itLcWkrLbZvNjWphMaib1bQtDNmUfH4GmUiUPCUzryL90KpKuft+CxGDVaXjneWo 2HUUrh5sErCC3FziPwUDMLMfP8O0PGIaGsK8c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713809678; x=1714414478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6RRljp8ilqBOCq7+riUQu2IxAcNGdCvaCKnCfHGbuws=; b=RHJV3rB8EF5RXAKd3cKU1XgWmG82ie52JZqqRVOvuz2un+K0lD+T5vdkXjRU05CvDd hVhU2jdFCtNuPqXi5OtPE5EkjhkIDO0otbnSCy+yG7ZSb3smoxFcGksF3VmKVvyN9L4k 76dBnjJUPZ+4ZCu+LiaJ72+exu8ShsWmM5pTLNC1+V8tTKiMqJWSlZ10Nub/ljiSNdXa 2YPKEXs1yub8csebe7P/4k+gWjBIwBTWCS4ld4nJ9f5lYVA6CdLiTY+b7CitZPghyi28 YXjYYzvzeKr0hA0B8n4jRDi2unDiee6ffRqtX5g0ec8i7ThXipHB01Nf5hB84ynZuNTw nxfQ== X-Gm-Message-State: AOJu0Yw9NidK+KP2Q57ra+jcEv4PrqDfveRTI48HjHxnDVyZaLEcL763 nawJjWanaRICJqZ3tPcSAoSX1mx2iuUTz0zI59J+AoCKLB0fr2yDDFK9qFTAjLQejjmI3adY+Mg 9nnI= X-Google-Smtp-Source: AGHT+IGXtcazIcbQnNsHc45oDqFJsnTRiN7QccNtav/aj2Lc2owoQkx7kwbiFs6TnAPwRJubHQtmzg== X-Received: by 2002:a17:906:eb4f:b0:a58:725b:9fea with SMTP id mc15-20020a170906eb4f00b00a58725b9feamr846018ejb.21.1713809677900; Mon, 22 Apr 2024 11:14:37 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 1/6] x86: Introduce x86_decode_lite() Date: Mon, 22 Apr 2024 19:14:29 +0100 Message-Id: <20240422181434.3463252-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240422181434.3463252-1-andrew.cooper3@citrix.com> References: <20240422181434.3463252-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1713809708600100004 In order to relocate all IP-relative fields in an alternative replacement block, we need to decode the instructions enough to obtain their length and any relative fields. Full x86_decode() is far too heavyweight, so introduce a minimal form which can make several simplifying assumptions. This logic is sufficient to decode all alternative blocks that exist in Xen right now. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/x86_emulate/Makefile | 1 + xen/arch/x86/x86_emulate/decode-lite.c | 245 +++++++++++++++++++++++++ xen/arch/x86/x86_emulate/private.h | 2 + xen/arch/x86/x86_emulate/x86_emulate.h | 17 ++ 4 files changed, 265 insertions(+) create mode 100644 xen/arch/x86/x86_emulate/decode-lite.c diff --git a/xen/arch/x86/x86_emulate/Makefile b/xen/arch/x86/x86_emulate/M= akefile index 2e20d65d788a..f06731913d67 100644 --- a/xen/arch/x86/x86_emulate/Makefile +++ b/xen/arch/x86/x86_emulate/Makefile @@ -3,6 +3,7 @@ obj-y +=3D 0fae.o obj-y +=3D 0fc7.o obj-y +=3D blk.o obj-y +=3D decode.o +obj-y +=3D decode-lite.o obj-$(CONFIG_HVM) +=3D fpu.o obj-y +=3D util.o obj-y +=3D util-xen.o diff --git a/xen/arch/x86/x86_emulate/decode-lite.c b/xen/arch/x86/x86_emul= ate/decode-lite.c new file mode 100644 index 000000000000..050b0d8cf4a8 --- /dev/null +++ b/xen/arch/x86/x86_emulate/decode-lite.c @@ -0,0 +1,245 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include "private.h" + +#define Imm8 (1 << 0) +#define Imm (1 << 1) +#define Branch (1 << 5) /* ... that we care about */ +/* ModRM (1 << 6) */ +#define Known (1 << 7) + +#define ALU_OPS \ + (Known|ModRM), \ + (Known|ModRM), \ + (Known|ModRM), \ + (Known|ModRM), \ + (Known|Imm8), \ + (Known|Imm) + +static const uint8_t init_or_livepatch_const onebyte[256] =3D { + [0x00] =3D ALU_OPS, /* ADD */ [0x08] =3D ALU_OPS, /* OR */ + [0x10] =3D ALU_OPS, /* ADC */ [0x18] =3D ALU_OPS, /* SBB */ + [0x20] =3D ALU_OPS, /* AND */ [0x28] =3D ALU_OPS, /* SUB */ + [0x30] =3D ALU_OPS, /* XOR */ [0x38] =3D ALU_OPS, /* CMP */ + + [0x50 ... 0x5f] =3D (Known), /* PUSH/POP %reg */ + + [0x70 ... 0x7f] =3D (Known|Branch|Imm8), /* Jcc disp8 */ + [0x80] =3D (Known|ModRM|Imm8), + [0x81] =3D (Known|ModRM|Imm), + + [0x83] =3D (Known|ModRM|Imm8), + [0x84 ... 0x8e] =3D (Known|ModRM), /* TEST/XCHG/MOV/MOV-SREG/LEA= r/rm */ + + [0xb0 ... 0xb7] =3D (Known|Imm8), /* MOV $imm8, %reg */ + [0xb8 ... 0xbf] =3D (Known|Imm), /* MOV $imm32, %reg */ + + [0xcc] =3D (Known), /* INT3 */ + [0xcd] =3D (Known|Imm8), /* INT $imm8 */ + + [0xe8 ... 0xe9] =3D (Known|Branch|Imm), /* CALL/JMP disp32 */ + [0xeb] =3D (Known|Branch|Imm8), /* JMP disp8 */ + + [0xf1] =3D (Known), /* ICEBP */ + [0xf4] =3D (Known), /* HLT */ + [0xf5] =3D (Known), /* CMC */ + [0xf6 ... 0xf7] =3D (Known|ModRM), /* Grp3 */ + [0xf8 ... 0xfd] =3D (Known), /* CLC ... STD */ + [0xfe ... 0xff] =3D (Known|ModRM), /* Grp4 */ +}; +static const uint8_t init_or_livepatch_const twobyte[256] =3D { + [0x00 ... 0x01] =3D (Known|ModRM), /* Grp6/Grp7 */ + + [0x18 ... 0x1f] =3D (Known|ModRM), /* Grp16 (Hint Nop) */ + + [0x20 ... 0x23] =3D (Known|ModRM), /* MOV CR/DR */ + + [0x30 ... 0x34] =3D (Known), /* WRMSR ... RDPMC */ + [0x40 ... 0x4f] =3D (Known|ModRM), /* CMOVcc */ + + [0x80 ... 0x8f] =3D (Known|Branch|Imm), /* Jcc disp32 */ + [0x90 ... 0x9f] =3D (Known|ModRM), /* SETcc */ + + [0xab] =3D (Known|ModRM), /* BTS */ + [0xac] =3D (Known|ModRM|Imm8), /* SHRD $imm8 */ + [0xad ... 0xaf] =3D (Known|ModRM), /* SHRD %cl / Grp15 / IMUL */ + + [0xb8 ... 0xb9] =3D (Known|ModRM), /* POPCNT/Grp10 (UD1) */ + [0xba] =3D (Known|ModRM|Imm8), /* Grp8 */ + [0xbb ... 0xbf] =3D (Known|ModRM), /* BSR/BSF/BSR/MOVSX */ +}; + +/* + * Bare minimum x86 instruction decoder to parse the alternative replaceme= nt + * instructions and locate the IP-relative references that may need updati= ng. + * + * These are: + * - disp8/32 from near branches + * - RIP-relative memory references + * + * The following simplifications are used: + * - All code is 64bit, and the instruction stream is safe to read. + * - The 67 prefix is not implemented, so the address size is only 64bit. + * + * Inputs: + * @ip The position to start decoding from. + * @end End of the replacement block. Exceeding this is considered an er= ror. + * + * Returns: x86_decode_lite_t + * - On failure, length of -1. + * - On success, length > 0 and REL_TYPE_*. For REL_TYPE !=3D NONE, rel = points + * at the relative field in the instruction stream. + */ +x86_decode_lite_t init_or_livepatch x86_decode_lite(void *ip, void *end) +{ + void *start =3D ip, *rel =3D NULL; + unsigned int opc, type =3D REL_TYPE_NONE; + uint8_t b, d, osize =3D 4; + +#define OPC_TWOBYTE (1 << 8) + + /* Mutates IP, uses END. */ +#define FETCH(ty) \ + ({ \ + ty _val; \ + \ + if ( (ip + sizeof(ty)) > end ) \ + goto overrun; \ + _val =3D *(ty *)ip; \ + ip +=3D sizeof(ty); \ + _val; \ + }) + + for ( ;; ) /* Prefixes */ + { + switch ( b =3D FETCH(uint8_t) ) + { + case 0x26: /* ES override */ + case 0x2e: /* CS override */ + case 0x36: /* DS override */ + case 0x3e: /* SS override */ + case 0x64: /* FS override */ + case 0x65: /* GS override */ + case 0xf0: /* Lock */ + case 0xf2: /* REPNE */ + case 0xf3: /* REP */ + break; + + case 0x66: /* Operand size override */ + osize =3D 2; + break; + + /* case 0x67: Address size override, not implemented */ + + case 0x40 ... 0x4f: /* REX */ + continue; + + default: + goto prefixes_done; + } + } + prefixes_done: + + /* Fetch the main opcode byte(s) */ + if ( b =3D=3D 0x0f ) + { + b =3D FETCH(uint8_t); + opc =3D OPC_TWOBYTE | b; + + d =3D twobyte[b]; + } + else + { + opc =3D b; + d =3D onebyte[b]; + } + + if ( unlikely(!(d & Known)) ) + goto unknown; + + if ( d & ModRM ) + { + uint8_t modrm =3D FETCH(uint8_t); + uint8_t mod =3D modrm >> 6; + uint8_t reg =3D (modrm >> 3) & 7; + uint8_t rm =3D modrm & 7; + + /* ModRM/SIB decode */ + if ( mod =3D=3D 0 && rm =3D=3D 5 ) /* RIP relative */ + { + rel =3D ip; + type =3D REL_TYPE_d32; + FETCH(uint32_t); + } + else if ( mod !=3D 3 && rm =3D=3D 4 ) /* SIB */ + FETCH(uint8_t); + + if ( mod =3D=3D 1 ) /* disp8 */ + FETCH(uint8_t); + else if ( mod =3D=3D 2 ) /* disp32 */ + FETCH(uint32_t); + + /* ModRM based decode adjustements */ + switch ( opc ) + { + case 0xf6: /* Grp3 TEST(s) have extra Imm8 */ + if ( reg =3D=3D 0 || reg =3D=3D 1 ) + d |=3D Imm8; + break; + case 0xf7: /* Grp3 TEST(s) have extra Imm */ + if ( reg =3D=3D 0 || reg =3D=3D 1 ) + d |=3D Imm; + break; + } + } + + if ( d & Branch ) + { + /* + * Don't tolerate 66-prefixed call/jmp in alternatives. Some are + * genuinely decoded differently between Intel and AMD CPUs. + */ + if ( osize !=3D 4 ) + goto bad_osize; + + rel =3D ip; + type =3D (d & Imm8) ? REL_TYPE_d8 : REL_TYPE_d32; + } + + if ( d & (Imm|Imm8) ) + { + if ( d & Imm8 ) + osize =3D 1; + + switch ( osize ) + { + case 1: FETCH(uint8_t); break; + case 2: FETCH(uint16_t); break; + case 4: FETCH(uint32_t); break; + default: goto bad_osize; + } + } + + return (x86_decode_lite_t){ ip - start, type, rel }; + + bad_osize: + printk(XENLOG_ERR "%s() Bad osize %u in %*ph\n", + __func__, osize, + (int)(unsigned long)(end - start), start); + return (x86_decode_lite_t){ -1, REL_TYPE_NONE, NULL }; + + unknown: + printk(XENLOG_ERR "%s() Unknown opcode in %*ph <%02x> %*ph\n", + __func__, + (int)(unsigned long)(ip - 1 - start), start, b, + (int)(unsigned long)(end - ip), ip); + return (x86_decode_lite_t){ -1, REL_TYPE_NONE, NULL }; + + overrun: + printk(XENLOG_ERR "%s() Decode overrun, got %*ph\n", + __func__, + (int)(unsigned long)(end - start), start); + return (x86_decode_lite_t){ -1, REL_TYPE_NONE, NULL }; + +#undef FETCH +} diff --git a/xen/arch/x86/x86_emulate/private.h b/xen/arch/x86/x86_emulate/= private.h index 0fa26ba00aa5..792b04889ce6 100644 --- a/xen/arch/x86/x86_emulate/private.h +++ b/xen/arch/x86/x86_emulate/private.h @@ -9,7 +9,9 @@ #ifdef __XEN__ =20 # include +# include # include +# include # include # include # include diff --git a/xen/arch/x86/x86_emulate/x86_emulate.h b/xen/arch/x86/x86_emul= ate/x86_emulate.h index d92be69d84d9..b7cbcd8b70ba 100644 --- a/xen/arch/x86/x86_emulate/x86_emulate.h +++ b/xen/arch/x86/x86_emulate/x86_emulate.h @@ -842,4 +842,21 @@ static inline void x86_emul_reset_event(struct x86_emu= late_ctxt *ctxt) ctxt->event =3D (struct x86_event){}; } =20 +/* + * x86_decode_lite(). Very minimal decoder for managing alternatives. + * + * @len is -1 on error, or positive on success. If the instruction has a + * relative field, REL_TYPE_* gives the size, and @rel points at the field. + */ +typedef struct { + int8_t len; + uint8_t rel_type; +#define REL_TYPE_NONE 0 +#define REL_TYPE_d8 1 +#define REL_TYPE_d32 2 + void *rel; +} x86_decode_lite_t; + +x86_decode_lite_t x86_decode_lite(void *ip, void *end); + #endif /* __X86_EMULATE_H__ */ --=20 2.30.2 From nobody Sun May 19 06:23:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1713809708; cv=none; d=zohomail.com; s=zohoarc; b=bngFVkQcZaTg38Cx9XiFmAp+UkBI1dtiHiqBB5xSsaag8T0mBwsFhC4STTAS3VAvOPCD7Bvjtfxn+LORHp1W5UcJccUx5GNz2eg/Bqd7m35GONYMsU6RkyRDknd5/gtZOktG8NoFYy6R2FOKu471a6wZyRQti0UiePI7GCtHAbk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1713809708; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MQHzbKjDaYMcDY7B8SUQVz+z/Tfqym852xnTGj/zaZg=; b=HKJcmsGdpVxgv4/k22bPaPhH7IUgW2evaxRFmgpQXJ3+Pbgzlu8+T0yJFi23DFQShN4umbRXCt7FWWE2Ht7ZIrKzINhR3K5rexQul81mGm4DGQgh2bGJA9b6+U2/toJ/xl/8IGN2GDVmPcnEGqQj+DrCGH7dYQTjTTP3RP81Nh4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1713809708975631.2458850914343; Mon, 22 Apr 2024 11:15:08 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.710191.1109300 (Exim 4.92) (envelope-from ) id 1ryyBe-0000aL-9I; Mon, 22 Apr 2024 18:14:42 +0000 Received: by outflank-mailman (output) from mailman id 710191.1109300; Mon, 22 Apr 2024 18:14:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBe-0000aE-63; Mon, 22 Apr 2024 18:14:42 +0000 Received: by outflank-mailman (input) for mailman id 710191; Mon, 22 Apr 2024 18:14:40 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBc-0000IX-66 for xen-devel@lists.xenproject.org; Mon, 22 Apr 2024 18:14:40 +0000 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [2a00:1450:4864:20::430]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 2edcb1ec-00d4-11ef-909a-e314d9c70b13; Mon, 22 Apr 2024 20:14:39 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-34b1e35155aso1321394f8f.3 for ; Mon, 22 Apr 2024 11:14:39 -0700 (PDT) Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id z15-20020a170906668f00b00a51d073da7esm5994224ejo.82.2024.04.22.11.14.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 11:14:38 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2edcb1ec-00d4-11ef-909a-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1713809678; x=1714414478; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MQHzbKjDaYMcDY7B8SUQVz+z/Tfqym852xnTGj/zaZg=; b=fkoAkh4o4FnDA99uQNTZb7WraJWXKC2aG6unhYS0rrZIufQCDn5N70MWM/DTyZSA5z NeAjzvm/PJimblHVJgb4bHXzM9Rr0w7UkGOTJJ+nuzOZmXh9lx9GQJw22NhP/Tj8+/5R ONR1uM/azVB0mFU3xpualaMwOxWW1MqD8zjLU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713809678; x=1714414478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MQHzbKjDaYMcDY7B8SUQVz+z/Tfqym852xnTGj/zaZg=; b=Z7KNvKuGyja0RlV16mPa1yOseYZr+SWfrDlIsD7bMyGklWTHPTswUlpk60ISuWE/vP dc6QSsNuoPQANdB/hkBgM2FPllFzgn/iebrIsBhUzxbZQxBySnuJdu4AgoZDcGQqRIRQ wwni3mbGpvxsTfsAZxYhWLO2I8FnlyxRVcZgS8DftwgzDcgfCVGywSAhYZc0tCUXvTW6 TzsoOdz2cmRbtRxA38NEkKe4bW6EofTpZ2tXGJHDQ65YZnfFRg8cywtAJkZG367sjW2F 4oMZ7mch5BHDJNvmkGaZiVm+ccdGx3XhC5zxiXoB3fMM5PsQmBTUg7ezuShTlyJshJdr 7Cag== X-Gm-Message-State: AOJu0YxiIHH3elVtiVQoD3vnH7iU0qSBo0jS+i1B6QYMJG+U4DX56ZRW Xg4d6SnOQJbQzmOLIVog4+Q0/c3PKIiHtfyPN3AhK2azt8gN/eYHk4/Ow4bQnC+7rDkD1LKPwpp HXPE= X-Google-Smtp-Source: AGHT+IEW6kofzVvscdUp6IMKGrRkIje/J7JBh/Nc5qR/k31CnySuyyiwls0zSTGzIDDorCf57TnjJQ== X-Received: by 2002:a5d:4a81:0:b0:341:cd0d:b78a with SMTP id o1-20020a5d4a81000000b00341cd0db78amr8767835wrq.48.1713809678533; Mon, 22 Apr 2024 11:14:38 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 2/6] x86/alternative: Walk all replacements in debug builds Date: Mon, 22 Apr 2024 19:14:30 +0100 Message-Id: <20240422181434.3463252-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240422181434.3463252-1-andrew.cooper3@citrix.com> References: <20240422181434.3463252-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1713809710480100007 In debug builds, walk all alternative replacements with x86_decode_lite(). This checks that we can decode all instructions, and also lets us check that disp8's don't leave the replacement block. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 49 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 2e7ba6e0b833..5bd256365def 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -15,6 +15,7 @@ #include #include #include +#include #include =20 #define MAX_PATCH_LEN (255-1) @@ -464,6 +465,54 @@ static void __init _alternative_instructions(bool forc= e) void __init alternative_instructions(void) { arch_init_ideal_nops(); + + /* + * Walk all replacement instructions with x86_decode_lite(). This che= cks + * both that we can decode all instructions within the replacement, and + * that any near branch with a disp8 stays within the alternative itse= lf. + */ + if ( IS_ENABLED(CONFIG_DEBUG) ) + { + struct alt_instr *a; + + for ( a =3D __alt_instructions; + a < __alt_instructions_end; ++a ) + { + void *repl =3D ALT_REPL_PTR(a); + void *ip =3D repl, *end =3D ip + a->repl_len; + + if ( !a->repl_len ) + continue; + + for ( x86_decode_lite_t res; ip < end; ip +=3D res.len ) + { + res =3D x86_decode_lite(ip, end); + + if ( res.len <=3D 0 ) + { + printk("Alternative for %ps [%*ph]\n", + ALT_ORIG_PTR(a), a->repl_len, repl); + panic("Unable to decode instruction at +%u in alternat= ive\n", + (unsigned int)(unsigned long)(ip - repl)); + } + + if ( res.rel_type =3D=3D REL_TYPE_d8 ) + { + int8_t *d8 =3D res.rel; + void *target =3D ip + res.len + *d8; + + if ( target < repl || target > end ) + { + printk("Alternative for %ps [%*ph]\n", + ALT_ORIG_PTR(a), a->repl_len, repl); + panic("'JMP/Jcc disp8' at +%u leaves alternative b= lock\n", + (unsigned int)(unsigned long)(ip - repl)); + } + } + } + } + } + _alternative_instructions(false); } =20 --=20 2.30.2 From nobody Sun May 19 06:23:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1713809706; cv=none; d=zohomail.com; s=zohoarc; b=LbGhcagZQI1dki2yuVRyBbx/azbN9m7ekjHCzX/SE8/FrU2HEKV23EWhB2pTi70VbiMnaKI7go+quzkDgVCUlgovk15POshFEIOfaMBN8ubf7BVDdM6DYuBcjuMynDfdkZaukxWC06ug4c4HsCG1wR8b5rAwYKLkN7sl0EJjHC4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1713809706; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=31yUgd5YK+pN4dRkn1LMnihd6GkdbjUm1b+8i5iacEg=; b=NazNr895FMYT7pSSBVp1mJ+la2vAFuU+2JzcgMKQdrjG3FwI57i2U/BbXbSemtF8z9QazuGwbRGSW2MKM8kNccoLd65Q0plIPtVz/TF+QRcB0noBF+0/0BA77b7ey/kLVXi1PhIrfF0EgXWF1kWHvK9UfHYxpIyHo2pAqMjMnrI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1713809706779108.91001885411515; Mon, 22 Apr 2024 11:15:06 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.710192.1109306 (Exim 4.92) (envelope-from ) id 1ryyBe-0000cG-Kx; Mon, 22 Apr 2024 18:14:42 +0000 Received: by outflank-mailman (output) from mailman id 710192.1109306; Mon, 22 Apr 2024 18:14:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBe-0000bt-E3; Mon, 22 Apr 2024 18:14:42 +0000 Received: by outflank-mailman (input) for mailman id 710192; Mon, 22 Apr 2024 18:14:41 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBd-0000IX-6N for xen-devel@lists.xenproject.org; Mon, 22 Apr 2024 18:14:41 +0000 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [2a00:1450:4864:20::636]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 2f64d62a-00d4-11ef-909a-e314d9c70b13; Mon, 22 Apr 2024 20:14:40 +0200 (CEST) Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-a5872b74c44so46930866b.3 for ; Mon, 22 Apr 2024 11:14:40 -0700 (PDT) Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id z15-20020a170906668f00b00a51d073da7esm5994224ejo.82.2024.04.22.11.14.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 11:14:38 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2f64d62a-00d4-11ef-909a-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1713809679; x=1714414479; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=31yUgd5YK+pN4dRkn1LMnihd6GkdbjUm1b+8i5iacEg=; b=R8PB8SYRh8z3v+Cn85Jjbb74Z10LovtcXj6ymwCs2nfNJZki8WLNxy/auZMP+8np1g 3UrKFthyTFQU4JYfs7JkbUYkNsTBzzy5D5aqEAA08QulrinS90Bqd6DckYwBhlL4jigh Lsn50J/FNrHi9cBv5RTpwIxvV3ENJ71hnw6ZU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713809679; x=1714414479; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=31yUgd5YK+pN4dRkn1LMnihd6GkdbjUm1b+8i5iacEg=; b=oM2Zm/nKzwWEWmg9mARcP99h3lv4HWNNCHi+Lpv2rkSFjPysZSzcb+ewU0WosV8jwn SuISFefjdc9tvZcGx/wdq26bNpM2Kf8JgMDpM+3Z0HvIAqSZ1Y/vTBrjKguUMZsTC2Ic 0iwV+POn0begZM5WI25cr7kuYyXCAxblY9syDpjiYuR8yzjxjdS1Ozpwbmr9ReeSEdH1 aXtukBXWyvvBV/yPaMF724KuQQD1YBsJ7T4XLHf5V/nO+k4qG7EHoAWhnrCOmCiZ8PQ1 1xMtvJ6b6QVQ+AbcOyz5zbLJeDRzDkbI9uoE6stycf4117DYukwDqXwP0Xpa2Z26VPle /IOQ== X-Gm-Message-State: AOJu0YzZk8fI+mWgpVG6674mu7AFPXHXoiIimJ6kfcmp1FtJlpUvbYVz VtA7kzFu3+SZAn6BnPyOxjWO4BBYhUZfXkvQU8N78rGZi5p3Kn3KmhlTJGYbRtHFz+QJW78JYcA 1Q2E= X-Google-Smtp-Source: AGHT+IG+1HMvF1SwFEh23Uw5BDoD1AN36ExhwVuxXRvPOWRV9QQwPpCghdo+Q9ydNqUU6Qfd296HkA== X-Received: by 2002:a17:906:7708:b0:a51:969e:ba0 with SMTP id q8-20020a170906770800b00a51969e0ba0mr6257278ejm.44.1713809679358; Mon, 22 Apr 2024 11:14:39 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 3/6] x86/alternative: Intend the relocation logic Date: Mon, 22 Apr 2024 19:14:31 +0100 Message-Id: <20240422181434.3463252-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240422181434.3463252-1-andrew.cooper3@citrix.com> References: <20240422181434.3463252-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1713809708526100001 ... to make subsequent patches legible. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 126 +++++++++++++++++++------------------ 1 file changed, 64 insertions(+), 62 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 5bd256365def..2ca4dfd569bc 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -244,78 +244,80 @@ static void init_or_livepatch _apply_alternatives(str= uct alt_instr *start, =20 memcpy(buf, repl, a->repl_len); =20 - /* 0xe8/0xe9 are relative branches; fix the offset. */ - if ( a->repl_len >=3D 5 && (*buf & 0xfe) =3D=3D 0xe8 ) { - /* - * Detect the special case of indirect-to-direct branch patchi= ng: - * - replacement is a direct CALL/JMP (opcodes 0xE8/0xE9; alre= ady - * checked above), - * - replacement's displacement is -5 (pointing back at the ve= ry - * insn, which makes no sense in a real replacement insn), - * - original is an indirect CALL/JMP (opcodes 0xFF/2 or 0xFF/= 4) - * using RIP-relative addressing. - * Some branch destinations may still be NULL when we come here - * the first time. Defer patching of those until the post-pres= mp- - * initcalls re-invocation (with force set to true). If at that - * point the branch destination is still NULL, insert "UD2; UD= 0" - * (for ease of recognition) instead of CALL/JMP. - */ - if ( a->cpuid =3D=3D X86_FEATURE_ALWAYS && - *(int32_t *)(buf + 1) =3D=3D -5 && - a->orig_len >=3D 6 && - orig[0] =3D=3D 0xff && - orig[1] =3D=3D (*buf & 1 ? 0x25 : 0x15) ) + /* 0xe8/0xe9 are relative branches; fix the offset. */ + if ( a->repl_len >=3D 5 && (*buf & 0xfe) =3D=3D 0xe8 ) { - long disp =3D *(int32_t *)(orig + 2); - const uint8_t *dest =3D *(void **)(orig + 6 + disp); - - if ( dest ) + /* + * Detect the special case of indirect-to-direct branch pa= tching: + * - replacement is a direct CALL/JMP (opcodes 0xE8/0xE9; = already + * checked above), + * - replacement's displacement is -5 (pointing back at th= e very + * insn, which makes no sense in a real replacement insn= ), + * - original is an indirect CALL/JMP (opcodes 0xFF/2 or 0= xFF/4) + * using RIP-relative addressing. + * Some branch destinations may still be NULL when we come= here + * the first time. Defer patching of those until the post-= presmp- + * initcalls re-invocation (with force set to true). If at= that + * point the branch destination is still NULL, insert "UD2= ; UD0" + * (for ease of recognition) instead of CALL/JMP. + */ + if ( a->cpuid =3D=3D X86_FEATURE_ALWAYS && + *(int32_t *)(buf + 1) =3D=3D -5 && + a->orig_len >=3D 6 && + orig[0] =3D=3D 0xff && + orig[1] =3D=3D (*buf & 1 ? 0x25 : 0x15) ) { - /* - * When building for CET-IBT, all function pointer tar= gets - * should have an endbr64 instruction. - * - * If this is not the case, leave a warning because - * something is probably wrong with the build. A CET-= IBT - * enabled system might have exploded already. - * - * Otherwise, skip the endbr64 instruction. This is a - * marginal perf improvement which saves on instruction - * decode bandwidth. - */ - if ( IS_ENABLED(CONFIG_XEN_IBT) ) + long disp =3D *(int32_t *)(orig + 2); + const uint8_t *dest =3D *(void **)(orig + 6 + disp); + + if ( dest ) { - if ( is_endbr64(dest) ) - dest +=3D ENDBR64_LEN; - else - printk(XENLOG_WARNING - "altcall %ps dest %ps has no endbr64\n", - orig, dest); + /* + * When building for CET-IBT, all function pointer= targets + * should have an endbr64 instruction. + * + * If this is not the case, leave a warning because + * something is probably wrong with the build. A = CET-IBT + * enabled system might have exploded already. + * + * Otherwise, skip the endbr64 instruction. This = is a + * marginal perf improvement which saves on instru= ction + * decode bandwidth. + */ + if ( IS_ENABLED(CONFIG_XEN_IBT) ) + { + if ( is_endbr64(dest) ) + dest +=3D ENDBR64_LEN; + else + printk(XENLOG_WARNING + "altcall %ps dest %ps has no endbr6= 4\n", + orig, dest); + } + + disp =3D dest - (orig + 5); + ASSERT(disp =3D=3D (int32_t)disp); + *(int32_t *)(buf + 1) =3D disp; } - - disp =3D dest - (orig + 5); - ASSERT(disp =3D=3D (int32_t)disp); - *(int32_t *)(buf + 1) =3D disp; - } - else if ( force ) - { - buf[0] =3D 0x0f; - buf[1] =3D 0x0b; - buf[2] =3D 0x0f; - buf[3] =3D 0xff; - buf[4] =3D 0xff; + else if ( force ) + { + buf[0] =3D 0x0f; + buf[1] =3D 0x0b; + buf[2] =3D 0x0f; + buf[3] =3D 0xff; + buf[4] =3D 0xff; + } + else + continue; } + else if ( force && system_state < SYS_STATE_active ) + ASSERT_UNREACHABLE(); else - continue; + *(int32_t *)(buf + 1) +=3D repl - orig; } - else if ( force && system_state < SYS_STATE_active ) + else if ( force && system_state < SYS_STATE_active ) ASSERT_UNREACHABLE(); - else - *(int32_t *)(buf + 1) +=3D repl - orig; } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); =20 a->priv =3D 1; =20 --=20 2.30.2 From nobody Sun May 19 06:23:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1713809701; cv=none; d=zohomail.com; s=zohoarc; b=jMTU7aaVClVQoMRhNYrlA7up8tMPfQavpkfGRtw6B1RMacbdxRAsJGwaxkayz32kbS3TvDW7CZPKRPmbSD6jtR7/RCVWwlPW2i2sE7ZB8V1a++EkSar1PTJyByXk6cO1kfbNrlka/PTolivOw7nIv6AOHm5hnqW6YbtDpf0hgSc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1713809701; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=FaF1+jq0lOG0Zt+SDR+w7NABaf8lTc4H+2KHEf8aMyg=; b=VJSGB/Z2FeF+xT1togvA7UhV158GG/JEjpJF0qePrJJwbnL1FfUwvVyQ0T2bkqOomdAE8jAq+VXqdjDOa7YKLJNfWPnNu9p7W31jiShSaUud6mbNqMDT+gs6cy2ZVSTM7ImsSCEsmirQc35L2xwxv41ifPdaqyGGstpg26jXhN0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1713809701640286.7490846727742; Mon, 22 Apr 2024 11:15:01 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.710194.1109320 (Exim 4.92) (envelope-from ) id 1ryyBf-0000pk-9U; Mon, 22 Apr 2024 18:14:43 +0000 Received: by outflank-mailman (output) from mailman id 710194.1109320; Mon, 22 Apr 2024 18:14:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBf-0000oY-1G; Mon, 22 Apr 2024 18:14:43 +0000 Received: by outflank-mailman (input) for mailman id 710194; Mon, 22 Apr 2024 18:14:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBe-0000IX-6f for xen-devel@lists.xenproject.org; Mon, 22 Apr 2024 18:14:42 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 2ffeed76-00d4-11ef-909a-e314d9c70b13; Mon, 22 Apr 2024 20:14:41 +0200 (CEST) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-a55ab922260so203926466b.3 for ; Mon, 22 Apr 2024 11:14:41 -0700 (PDT) Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id z15-20020a170906668f00b00a51d073da7esm5994224ejo.82.2024.04.22.11.14.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 11:14:39 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2ffeed76-00d4-11ef-909a-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1713809680; x=1714414480; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FaF1+jq0lOG0Zt+SDR+w7NABaf8lTc4H+2KHEf8aMyg=; b=BJFT2JallZ/j//2MXYLr2gSSAMRBSs91NowYyRZw2r13Q85exIpfAnABt4KqaiUoV9 0Vg30zKnMkh7CSHkaXkjpQmHLqMv/eMn/dG6qDzr4n7wIL8+9NM4h8tNpEpACGpMMUVi 2IF4ZFViGMHFOKUmCUZNq3Cab7WnNgcCkC+bU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713809680; x=1714414480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FaF1+jq0lOG0Zt+SDR+w7NABaf8lTc4H+2KHEf8aMyg=; b=JZuVlHWOy8PWnTFXnKhTy85wWu56b2o6nIHlo7rLskynZ+KP2v3B00CjHAttAzd7wo g2SFKt70VgrhytUxHDVEuATy1lCw0vy0cNBiOqfWwuaCUZRO5CXqBzVzavjp7EXWLHdL qcB5YpHWdt9JPbd9DryaGaWOGs3S0LzhrYBBAcJpaIRuvyV5gIA+jhPKISlxPrz6rWfE pfiJe9P9Vs8gH/NHOke4iblbd+VkQKHfYaqOj7V/bIiP3z2njEL0aXfcQkvHAWzeQx29 H6i7pvn0a8sjNZ3V2Kj7jmuP9ztHBSEzl7VsclLlVXPR6K1aBJpFRCEQy5WH11SeFSxT K2Ng== X-Gm-Message-State: AOJu0Yy80mRYXNI8DOw+y1h1t3mqlIhPVyK2Egz8VlOk/pcRpgke57Qu kY0z5d+/Y8RkiBRK1rRQSwvocAhdEN8BXh8VAqxELbsluSXH/jB/LbpLnT42H0vm83EgjLyz+sn ZRr4= X-Google-Smtp-Source: AGHT+IHmqFdmdVeYAFVG6c/DuwA7Wp3vBh4M41S2ewoBL7esKFiKG2mvjUAjL/oIHx92bkOiFta4Xg== X-Received: by 2002:a17:906:1d47:b0:a58:7507:d4ed with SMTP id o7-20020a1709061d4700b00a587507d4edmr440811ejh.15.1713809680447; Mon, 22 Apr 2024 11:14:40 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 4/6] x86/alternative: Replace a continue with a goto Date: Mon, 22 Apr 2024 19:14:32 +0100 Message-Id: <20240422181434.3463252-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240422181434.3463252-1-andrew.cooper3@citrix.com> References: <20240422181434.3463252-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1713809702561100001 A subsequent patch is going to insert a loop, which interferes with the continue in the devirtualisation logic. Replace it with a goto, and a paragraph explaining why we intentionally avo= id setting a->priv =3D 1. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 2ca4dfd569bc..c86ea235e865 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -308,7 +308,15 @@ static void init_or_livepatch _apply_alternatives(stru= ct alt_instr *start, buf[4] =3D 0xff; } else - continue; + { + /* + * The function pointer we're wanting to devirtual= ise + * is still NULL, and we're not sealing yet. Leave + * the alternative fully un-processed, in order to + * process it the next time around. + */ + goto skip_this_alternative; + } } else if ( force && system_state < SYS_STATE_active ) ASSERT_UNREACHABLE(); @@ -323,6 +331,7 @@ static void init_or_livepatch _apply_alternatives(struc= t alt_instr *start, =20 add_nops(buf + a->repl_len, total_len - a->repl_len); text_poke(orig, buf, total_len); + skip_this_alternative:; } =20 /* --=20 2.30.2 From nobody Sun May 19 06:23:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1713809706; cv=none; d=zohomail.com; s=zohoarc; b=Ai5ssR/QF7CTaMB7O3BrOfYdClO73q32hYZm7E6vugQW6XUA7/OlftMZFm4TP/MHPz59x8GmhICJPvwTd2A8TxHWhtKtaa+AuGzqxotBifBRBD4KKq/uUCS+O0Ps8jKBfAaOatOQMcQqfxduFfJVtm1YMh0282bNu9vM64gHNv0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1713809706; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qnv/4DDYEmGIANJ07M3XY3Ct96pjBr7ZvuszXz6nRKc=; b=W0zrJwY532qpQ7Bx7GbDE2IIfhF4sYnDaPbUHSoFqolrMdpqLupIXDBkXc8mm5cmfqyp8Z9QewlMamh86YeNr6Innj9kKrCkhmRAkFcmMeSB/7v/6KE8+gyX4/hfIpTIm0tLGcqP89adP3/m3qc7SMm1/0lC/Sxv6i55YVj9lAg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1713809706871442.13004641657847; Mon, 22 Apr 2024 11:15:06 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.710195.1109341 (Exim 4.92) (envelope-from ) id 1ryyBh-0001aA-HA; Mon, 22 Apr 2024 18:14:45 +0000 Received: by outflank-mailman (output) from mailman id 710195.1109341; Mon, 22 Apr 2024 18:14:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBh-0001Zz-Dp; Mon, 22 Apr 2024 18:14:45 +0000 Received: by outflank-mailman (input) for mailman id 710195; Mon, 22 Apr 2024 18:14:43 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBf-0000Oz-If for xen-devel@lists.xenproject.org; Mon, 22 Apr 2024 18:14:43 +0000 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [2a00:1450:4864:20::631]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 30822915-00d4-11ef-b4bb-af5377834399; Mon, 22 Apr 2024 20:14:42 +0200 (CEST) Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-a5872b74c44so46933066b.3 for ; Mon, 22 Apr 2024 11:14:42 -0700 (PDT) Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id z15-20020a170906668f00b00a51d073da7esm5994224ejo.82.2024.04.22.11.14.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 11:14:40 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 30822915-00d4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1713809681; x=1714414481; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qnv/4DDYEmGIANJ07M3XY3Ct96pjBr7ZvuszXz6nRKc=; b=QzpJtI54AXreZkGZCRZZWsDlIs9qxLUDDh/+69vjiW9tEa6Gan13OcnMTcnl2GBhLJ CtUdJNSCtiiW0pMXxkR7GhaMosMBBj4oilGVCrFvaZ/CX6M7pIDI8vEP3hAp6cO/+/FA ltgIqXVkf0Ybz8/1fTA2dUqFp01Q2YIFJG/Mk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713809681; x=1714414481; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qnv/4DDYEmGIANJ07M3XY3Ct96pjBr7ZvuszXz6nRKc=; b=UV3y7Ab06TOJMUZW5M3QB05GIvIt7ukm4F1PAk0/e9gwG67NAS9At8CLF3IWgXI14N 13AczqnawMxz4DE6a6A2KcryJA5/NpvClssuHw2shom85xsZha+wG/Bu76/tdYPPSn6Z YxtIi/dmr5Ggyyxs5QB4OvRFjp4iFtnB9CLLSDSzS77hjFZ+2H3jEUMLw7scFewgiY5R /MEe/ZxOPmfvoKH4zJWDVw4rRQZ7tp1uiKeQrwrt12gmiP25qrZ4CRo1FKgyovY1o/WC XWWWXLufWzLA9WAXbUXggq7fuoM52YAZ2d5mdALLxpKx/LZnKV0J5XDDoIsofjmUy7zM ZNWA== X-Gm-Message-State: AOJu0Yw6J/krc7Kv9OPWmHiLH+GkI1w45lajDjaF6NBWzl2CTcB9m0Vl 0pRTxk1nZ38hkoUqKUO2PVBt1QstdJHktHFBnXSkUf5zFMB60o5n+s6/zMeWkEHyryRNjQBdLy4 jXds= X-Google-Smtp-Source: AGHT+IENu/P+oqdgoIPmfD0QCWOFEWg+4TnWMcAxsx2C9L/ECrh0opFdNPWnQ9cnJbJNMSXQoIR0AQ== X-Received: by 2002:a17:906:a0b:b0:a52:33b0:fcb1 with SMTP id w11-20020a1709060a0b00b00a5233b0fcb1mr6665155ejf.32.1713809681291; Mon, 22 Apr 2024 11:14:41 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 5/6] x86/alternative: Relocate all insn-relative fields Date: Mon, 22 Apr 2024 19:14:33 +0100 Message-Id: <20240422181434.3463252-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240422181434.3463252-1-andrew.cooper3@citrix.com> References: <20240422181434.3463252-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1713809708575100002 Right now, relocation of displacements is restricted to finding 0xe8/e9 as = the first byte of the replacement, but this is overly restrictive. Use x86_decode_lite() to find and adjust all insn-relative fields. As with disp8's not leaving the replacemnet block, some disp32's don't eith= er. e.g. the RSB stuffing loop. These stay unmodified. For now, leave the altcall devirtualisation alone. These require more care= to transform into the new scheme. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/alternative.c | 46 +++++++++++++++++++++++++++++++------- 1 file changed, 38 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index c86ea235e865..4d7dc9418cf0 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -244,10 +244,31 @@ static void init_or_livepatch _apply_alternatives(str= uct alt_instr *start, =20 memcpy(buf, repl, a->repl_len); =20 + /* Walk buf[] and adjust any insn-relative operands. */ + if ( a->repl_len ) { - /* 0xe8/0xe9 are relative branches; fix the offset. */ - if ( a->repl_len >=3D 5 && (*buf & 0xfe) =3D=3D 0xe8 ) + uint8_t *ip =3D buf, *end =3D ip + a->repl_len; + + for ( x86_decode_lite_t res; ip < end; ip +=3D res.len ) { + int32_t *d32; + uint8_t *target; + + res =3D x86_decode_lite(ip, end); + + if ( res.len <=3D 0 ) + { + printk("Alternative for %ps [%*ph]\n", + ALT_ORIG_PTR(a), a->repl_len, repl); + printk("Unable to decode instruction in alternative - = ignoring.\n"); + goto skip_this_alternative; + } + + if ( res.rel_type !=3D REL_TYPE_d32 ) + continue; + + d32 =3D res.rel; + /* * Detect the special case of indirect-to-direct branch pa= tching: * - replacement is a direct CALL/JMP (opcodes 0xE8/0xE9; = already @@ -317,14 +338,23 @@ static void init_or_livepatch _apply_alternatives(str= uct alt_instr *start, */ goto skip_this_alternative; } + + continue; } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); - else - *(int32_t *)(buf + 1) +=3D repl - orig; + + target =3D ip + res.len + *d32; + + if ( target >=3D buf && target <=3D end ) + { + /* + * Target doesn't leave the replacement block. e.g. R= SB + * stuffing. Leave it unmodified. + */ + continue; + } + + *d32 +=3D repl - orig; } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); } =20 a->priv =3D 1; --=20 2.30.2 From nobody Sun May 19 06:23:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1713809703; cv=none; d=zohomail.com; s=zohoarc; b=VoGyZ9nAgqtXPHv90LbtGOu3+XN9jT414bdQ6r/2GjE+A13SBLb4ToVfstyNbY8Chh7UN+1Ia1Wk1YX3RGCp1OcDuECc+3QjxsOi2gl0KGYrtjXJWR7f/6BFY5Hm4/A6Jc/FUPAPjb8MKJ3JNWlruR5Ku8EeuPKFtaEw7wcwpwk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1713809703; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dB+pEYt/xsRXM0pv02BxjDs01ExeTPOKtKLGLzOzv0s=; b=hOfwOjc6pDmg2XfcLwtiaWL4SwteztuwjRAv8+/vZYgxmh4B4ut2DaE9YItajrgSpcU0dz15wbGgp6NTmykCDqA4vuq8PMrjVVmEHEYmg+Tpva2Z+iBhSjyNuFYcDkfbBHXjIm6Ez4qmCNcaYG9YWYhTMPj7NaaLxRHkHbY0C0w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 171380970309758.722337292551515; Mon, 22 Apr 2024 11:15:03 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.710196.1109346 (Exim 4.92) (envelope-from ) id 1ryyBh-0001dp-TO; Mon, 22 Apr 2024 18:14:45 +0000 Received: by outflank-mailman (output) from mailman id 710196.1109346; Mon, 22 Apr 2024 18:14:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBh-0001cX-Ml; Mon, 22 Apr 2024 18:14:45 +0000 Received: by outflank-mailman (input) for mailman id 710196; Mon, 22 Apr 2024 18:14:43 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ryyBf-0000IX-OD for xen-devel@lists.xenproject.org; Mon, 22 Apr 2024 18:14:43 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 313ada63-00d4-11ef-909a-e314d9c70b13; Mon, 22 Apr 2024 20:14:43 +0200 (CEST) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a559b919303so249730966b.1 for ; Mon, 22 Apr 2024 11:14:43 -0700 (PDT) Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id z15-20020a170906668f00b00a51d073da7esm5994224ejo.82.2024.04.22.11.14.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 11:14:41 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 313ada63-00d4-11ef-909a-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1713809682; x=1714414482; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dB+pEYt/xsRXM0pv02BxjDs01ExeTPOKtKLGLzOzv0s=; b=m/YD7PW9Cinj9EGqD1JjCAgTChK87yX6Uvl6xvYJxsOl35BZPNaiEJAng9P1i57KOa gaZh+J2Ef3hUfcbgW6d01L0UpVYiLgX6baUO0KnTnZV0wgYOPAjIB24VwRZgo9BcaeWq yxIeEpAGtfYCnaOsSKNN+8W0Dz0CMi9L08mkQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713809682; x=1714414482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dB+pEYt/xsRXM0pv02BxjDs01ExeTPOKtKLGLzOzv0s=; b=O+hyWgCzHxiPtOv9ZnkfVqlvje80IYYtmlIrBUYejy1k/r9pT863FXu2zyg2yCUZMa xBPB0qkNFwNilMIPRqe9+Mpb2TdhyaasY8t4X0JtIjt5SpEVZcRgjDuXXJizAZWYlgcg rKTjPPp56IGaMnavArrNOBaBY2HcHeKGglXG+bwFvSYkKbLxi1vOx0wXiF33aRmjdV1E 0wQSsdXVJpS62DC99mSByl7YWUQvIauVkfWwqEZddeTmdwD2X/nEy864f5OL8BTMdPWc SycWpd8lSfwbSfaGZ1+tgicvQ3OGsWBX2lIXtQHqeCshXQfmfrf0Urx/HhvjJuHaL+77 VPwQ== X-Gm-Message-State: AOJu0YwF6/8QhXhe7xBsz9KWpSctht05QZKdoNoQ4z5ebrq8EoztWUyb YSJKbDLXXMWiEl795QtbxRwFMeYe2/z19+bDdyBCnWd8c7TurCsBn0Zkh/fHc9/9HHJoEcM+tzO sfQU= X-Google-Smtp-Source: AGHT+IGqd/HqMIOYUnziS1lqHwIAw5CNOJQ9KnM2G4JVD+h+CP14AE9vAz3vT5t6yXjZ6d/aUx+h6Q== X-Received: by 2002:a17:906:b092:b0:a55:387b:ef07 with SMTP id x18-20020a170906b09200b00a55387bef07mr7177532ejy.13.1713809682179; Mon, 22 Apr 2024 11:14:42 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 6/6] x86/spec-ctrl: Introduce and use DO_COND_BHB_SEQ Date: Mon, 22 Apr 2024 19:14:34 +0100 Message-Id: <20240422181434.3463252-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240422181434.3463252-1-andrew.cooper3@citrix.com> References: <20240422181434.3463252-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1713809704490100003 Now that alternatives can fix up call displacements even when they're not t= he first instruction of the replacement, move the SCF_entry_bhb conditional inside the replacement block. This removes a conditional branch from the fastpaths of BHI-unaffected hardware. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 --- xen/arch/x86/hvm/vmx/entry.S | 12 +++---- xen/arch/x86/include/asm/spec_ctrl_asm.h | 43 +++++++++++++----------- 2 files changed, 30 insertions(+), 25 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S index 7233e771d88a..77bf9ea564ea 100644 --- a/xen/arch/x86/hvm/vmx/entry.S +++ b/xen/arch/x86/hvm/vmx/entry.S @@ -62,12 +62,12 @@ ENTRY(vmx_asm_vmexit_handler) * Clear the BHB to mitigate BHI. Used on eIBRS parts, and uses R= ETs * itself so must be after we've perfomed all the RET-safety we ca= n. */ - testb $SCF_entry_bhb, CPUINFO_scf(%rsp) - jz .L_skip_bhb - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L_skip_bhb: + .macro VMX_BHB_SEQ fn:req + DO_COND_BHB_SEQ \fn scf=3DCPUINFO_scf(%rsp) + .endm + ALTERNATIVE_2 "", \ + "VMX_BHB_SEQ fn=3Dclear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "VMX_BHB_SEQ fn=3Dclear_bhb_tsx", X86_SPEC_BHB_TSX =20 ALTERNATIVE "lfence", "", X86_SPEC_NO_LFENCE_ENTRY_VMX /* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */ diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/includ= e/asm/spec_ctrl_asm.h index 729a830411eb..559dad88f967 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -92,6 +92,21 @@ .L\@_skip: .endm =20 +.macro DO_COND_BHB_SEQ fn:req, scf=3D%bl +/* + * Requires SCF (defaults to %rbx), fn=3Dclear_bhb_{loops,tsx} + * Clobbers %rax, %rcx + * + * Conditionally use a BHB clearing software sequence. + */ + testb $SCF_entry_bhb, \scf + jz .L\@_skip_bhb + + call \fn + +.L\@_skip_bhb: +.endm + .macro DO_OVERWRITE_RSB tmp=3Drax, xu /* * Requires nothing @@ -277,12 +292,9 @@ * Clear the BHB to mitigate BHI. Used on eIBRS parts, and uses RETs * itself so must be after we've perfomed all the RET-safety we can. */ - testb $SCF_entry_bhb, %bl - jz .L\@_skip_bhb - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L\@_skip_bhb: + ALTERNATIVE_2 "", \ + "DO_COND_BHB_SEQ clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "DO_COND_BHB_SEQ clear_bhb_tsx", X86_SPEC_BHB_TSX =20 ALTERNATIVE "lfence", "", X86_SPEC_NO_LFENCE_ENTRY_PV .endm @@ -322,12 +334,9 @@ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=3D1), \ X86_FEATURE_SC_MSR_PV =20 - testb $SCF_entry_bhb, %bl - jz .L\@_skip_bhb - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L\@_skip_bhb: + ALTERNATIVE_2 "", \ + "DO_COND_BHB_SEQ clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "DO_COND_BHB_SEQ clear_bhb_tsx", X86_SPEC_BHB_TSX =20 ALTERNATIVE "lfence", "", X86_SPEC_NO_LFENCE_ENTRY_INTR .endm @@ -433,13 +442,9 @@ * Clear the BHB to mitigate BHI. Used on eIBRS parts, and uses RETs * itself so must be after we've perfomed all the RET-safety we can. */ - testb $SCF_entry_bhb, %bl - jz .L\@_skip_bhb - - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L\@_skip_bhb: + ALTERNATIVE_2 "", \ + "DO_COND_BHB_SEQ clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "DO_COND_BHB_SEQ clear_bhb_tsx", X86_SPEC_BHB_TSX =20 lfence .endm --=20 2.30.2