From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9909CC636D4 for ; Thu, 2 Feb 2023 07:42:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231743AbjBBHm3 (ORCPT ); Thu, 2 Feb 2023 02:42:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229495AbjBBHm1 (ORCPT ); Thu, 2 Feb 2023 02:42:27 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B68EB751AD; Wed, 1 Feb 2023 23:42:26 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id AC81120B74F7; Wed, 1 Feb 2023 23:42:25 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com AC81120B74F7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323746; bh=KrZSg2bNqxWMiHZDDiwh5bJWJjaRLRpKmuEYjI5Br/E=; h=From:To:Subject:Date:In-Reply-To:References:From; b=j/su7JMbNc+4R20aW2hpwxnbcloXGFZlzmdtyO1JoBF0wIRExhA9W+ndzz00e88qy A/9GbYu6efRe+uezEvfN3dhaN/5crHTkHKcT5h3tmqFZ3p2WeqrvQuK8AQLYkGA0ag d4AreYpr5nAWD9AdM7TbqugHMbsWOdtPxLgTX1go= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 01/22] objtool: Reorganize CFI code Date: Thu, 2 Feb 2023 01:40:15 -0600 Message-Id: <20230202074036.507249-2-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" check.c implements static stack validation. But the CFI code that it contains can be shared with other types of validation. E.g., dynamic FP validation. Move the CFI code to its own files - cfi.h and cfi.c. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/Build | 1 + tools/objtool/cfi.c | 108 ++++++++++++++++++++++++++++ tools/objtool/check.c | 96 ------------------------- tools/objtool/include/objtool/cfi.h | 12 ++++ 4 files changed, 121 insertions(+), 96 deletions(-) create mode 100644 tools/objtool/cfi.c diff --git a/tools/objtool/Build b/tools/objtool/Build index 33f2ee5a46d3..21db9d79c69f 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -5,6 +5,7 @@ objtool-y +=3D weak.o objtool-y +=3D check.o objtool-y +=3D special.o objtool-y +=3D builtin-check.o +objtool-y +=3D cfi.o objtool-y +=3D elf.o objtool-y +=3D objtool.o =20 diff --git a/tools/objtool/cfi.c b/tools/objtool/cfi.c new file mode 100644 index 000000000000..18b460e066c8 --- /dev/null +++ b/tools/objtool/cfi.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2015-2017 Josh Poimboeuf + */ + +#include +#include + +#include +#include +#include + +unsigned long nr_cfi, nr_cfi_reused, nr_cfi_cache; + +struct cfi_init_state initial_func_cfi; +struct cfi_state init_cfi; +struct cfi_state func_cfi; + +void init_cfi_state(struct cfi_state *cfi) +{ + int i; + + for (i =3D 0; i < CFI_NUM_REGS; i++) { + cfi->regs[i].base =3D CFI_UNDEFINED; + cfi->vals[i].base =3D CFI_UNDEFINED; + } + cfi->cfa.base =3D CFI_UNDEFINED; + cfi->drap_reg =3D CFI_UNDEFINED; + cfi->drap_offset =3D -1; +} + +static struct cfi_state *cfi_alloc(void) +{ + struct cfi_state *cfi =3D calloc(sizeof(struct cfi_state), 1); + + if (!cfi) { + WARN("calloc failed"); + exit(1); + } + nr_cfi++; + return cfi; +} + +static int cfi_bits; +static struct hlist_head *cfi_hash; + +inline bool cficmp(struct cfi_state *cfi1, struct cfi_state *cfi2) +{ + return memcmp((void *)cfi1 + sizeof(cfi1->hash), + (void *)cfi2 + sizeof(cfi2->hash), + sizeof(struct cfi_state) - sizeof(struct hlist_node)); +} + +static inline u32 cfi_key(struct cfi_state *cfi) +{ + return jhash((void *)cfi + sizeof(cfi->hash), + sizeof(*cfi) - sizeof(cfi->hash), 0); +} + +struct cfi_state *cfi_hash_find_or_add(struct cfi_state *cfi) +{ + struct hlist_head *head =3D &cfi_hash[hash_min(cfi_key(cfi), cfi_bits)]; + struct cfi_state *obj; + + hlist_for_each_entry(obj, head, hash) { + if (!cficmp(cfi, obj)) { + nr_cfi_cache++; + return obj; + } + } + + obj =3D cfi_alloc(); + *obj =3D *cfi; + hlist_add_head(&obj->hash, head); + + return obj; +} + +void cfi_hash_add(struct cfi_state *cfi) +{ + struct hlist_head *head =3D &cfi_hash[hash_min(cfi_key(cfi), cfi_bits)]; + + hlist_add_head(&cfi->hash, head); +} + +void *cfi_hash_alloc(unsigned long size) +{ + cfi_bits =3D max(10, ilog2(size)); + cfi_hash =3D mmap(NULL, sizeof(struct hlist_head) << cfi_bits, + PROT_READ|PROT_WRITE, + MAP_PRIVATE|MAP_ANON, -1, 0); + if (cfi_hash =3D=3D (void *)-1L) { + WARN("mmap fail cfi_hash"); + cfi_hash =3D NULL; + } else if (opts.stats) { + printf("cfi_bits: %d\n", cfi_bits); + } + + return cfi_hash; +} + +void set_func_state(struct cfi_state *state) +{ + state->cfa =3D initial_func_cfi.cfa; + memcpy(&state->regs, &initial_func_cfi.regs, + CFI_NUM_REGS * sizeof(struct cfi_reg)); + state->stack_size =3D initial_func_cfi.cfa.offset; +} diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 43ec14c29a60..e6a2afa08748 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -28,12 +28,6 @@ struct alternative { bool skip_orig; }; =20 -static unsigned long nr_cfi, nr_cfi_reused, nr_cfi_cache; - -static struct cfi_init_state initial_func_cfi; -static struct cfi_state init_cfi; -static struct cfi_state func_cfi; - struct instruction *find_insn(struct objtool_file *file, struct section *sec, unsigned long offset) { @@ -255,19 +249,6 @@ static bool dead_end_function(struct objtool_file *fil= e, struct symbol *func) return __dead_end_function(file, func, 0); } =20 -static void init_cfi_state(struct cfi_state *cfi) -{ - int i; - - for (i =3D 0; i < CFI_NUM_REGS; i++) { - cfi->regs[i].base =3D CFI_UNDEFINED; - cfi->vals[i].base =3D CFI_UNDEFINED; - } - cfi->cfa.base =3D CFI_UNDEFINED; - cfi->drap_reg =3D CFI_UNDEFINED; - cfi->drap_offset =3D -1; -} - static void init_insn_state(struct objtool_file *file, struct insn_state *= state, struct section *sec) { @@ -283,75 +264,6 @@ static void init_insn_state(struct objtool_file *file,= struct insn_state *state, state->noinstr =3D sec->noinstr; } =20 -static struct cfi_state *cfi_alloc(void) -{ - struct cfi_state *cfi =3D calloc(sizeof(struct cfi_state), 1); - if (!cfi) { - WARN("calloc failed"); - exit(1); - } - nr_cfi++; - return cfi; -} - -static int cfi_bits; -static struct hlist_head *cfi_hash; - -static inline bool cficmp(struct cfi_state *cfi1, struct cfi_state *cfi2) -{ - return memcmp((void *)cfi1 + sizeof(cfi1->hash), - (void *)cfi2 + sizeof(cfi2->hash), - sizeof(struct cfi_state) - sizeof(struct hlist_node)); -} - -static inline u32 cfi_key(struct cfi_state *cfi) -{ - return jhash((void *)cfi + sizeof(cfi->hash), - sizeof(*cfi) - sizeof(cfi->hash), 0); -} - -static struct cfi_state *cfi_hash_find_or_add(struct cfi_state *cfi) -{ - struct hlist_head *head =3D &cfi_hash[hash_min(cfi_key(cfi), cfi_bits)]; - struct cfi_state *obj; - - hlist_for_each_entry(obj, head, hash) { - if (!cficmp(cfi, obj)) { - nr_cfi_cache++; - return obj; - } - } - - obj =3D cfi_alloc(); - *obj =3D *cfi; - hlist_add_head(&obj->hash, head); - - return obj; -} - -static void cfi_hash_add(struct cfi_state *cfi) -{ - struct hlist_head *head =3D &cfi_hash[hash_min(cfi_key(cfi), cfi_bits)]; - - hlist_add_head(&cfi->hash, head); -} - -static void *cfi_hash_alloc(unsigned long size) -{ - cfi_bits =3D max(10, ilog2(size)); - cfi_hash =3D mmap(NULL, sizeof(struct hlist_head) << cfi_bits, - PROT_READ|PROT_WRITE, - MAP_PRIVATE|MAP_ANON, -1, 0); - if (cfi_hash =3D=3D (void *)-1L) { - WARN("mmap fail cfi_hash"); - cfi_hash =3D NULL; - } else if (opts.stats) { - printf("cfi_bits: %d\n", cfi_bits); - } - - return cfi_hash; -} - static unsigned long nr_insns; static unsigned long nr_insns_visited; =20 @@ -2006,14 +1918,6 @@ static int add_jump_table_alts(struct objtool_file *= file) return 0; } =20 -static void set_func_state(struct cfi_state *state) -{ - state->cfa =3D initial_func_cfi.cfa; - memcpy(&state->regs, &initial_func_cfi.regs, - CFI_NUM_REGS * sizeof(struct cfi_reg)); - state->stack_size =3D initial_func_cfi.cfa.offset; -} - static int read_unwind_hints(struct objtool_file *file) { struct cfi_state cfi =3D init_cfi; diff --git a/tools/objtool/include/objtool/cfi.h b/tools/objtool/include/ob= jtool/cfi.h index f11d1ac1dadf..58391a4c49d4 100644 --- a/tools/objtool/include/objtool/cfi.h +++ b/tools/objtool/include/objtool/cfi.h @@ -37,4 +37,16 @@ struct cfi_state { bool end; }; =20 +void init_cfi_state(struct cfi_state *cfi); +bool cficmp(struct cfi_state *cfi1, struct cfi_state *cfi2); +struct cfi_state *cfi_hash_find_or_add(struct cfi_state *cfi); +void cfi_hash_add(struct cfi_state *cfi); +void *cfi_hash_alloc(unsigned long size); +void set_func_state(struct cfi_state *state); + +extern unsigned long nr_cfi, nr_cfi_reused, nr_cfi_cache; +extern struct cfi_init_state initial_func_cfi; +extern struct cfi_state init_cfi; +extern struct cfi_state func_cfi; + #endif /* _OBJTOOL_CFI_H */ --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 457FAC05027 for ; Thu, 2 Feb 2023 07:42:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231868AbjBBHmd (ORCPT ); Thu, 2 Feb 2023 02:42:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231751AbjBBHm3 (ORCPT ); Thu, 2 Feb 2023 02:42:29 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C66AD751BE; Wed, 1 Feb 2023 23:42:27 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id AA81E20B74F8; Wed, 1 Feb 2023 23:42:26 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com AA81E20B74F8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323747; bh=Dvi0sg7fNuFFbBH2nQXmc7LKL1McUIEsYXR3ernuKUg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=l+zQFcwZXVAVxrTLIKvrKvC5DKsTieLcvIIOOTnUFliWxXaOMEkHTL0uh8OoX+eRF sYk7GrU4mZiOu51Q7on0MD7dozaRld9DmufEc16kdV/4nTDLBRok9zdjCQmjsWBIWx kdwO1HwVANVeHSVImlg/sim8QNdBCHYtd34/lGcc= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 02/22] objtool: Reorganize instruction-related code Date: Thu, 2 Feb 2023 01:40:16 -0600 Message-Id: <20230202074036.507249-3-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" check.c implements static stack validation. But the instruction-related code that it contains can be shared with other types of validation. E.g., dynamic FP validation. Move the instruction-related code to its own files - insn.h and insn.c. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/Build | 1 + tools/objtool/check.c | 201 -------------------------- tools/objtool/include/objtool/check.h | 77 +--------- tools/objtool/include/objtool/insn.h | 125 ++++++++++++++++ tools/objtool/insn.c | 186 ++++++++++++++++++++++++ 5 files changed, 313 insertions(+), 277 deletions(-) create mode 100644 tools/objtool/include/objtool/insn.h create mode 100644 tools/objtool/insn.c diff --git a/tools/objtool/Build b/tools/objtool/Build index 21db9d79c69f..1149048e6b3e 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -6,6 +6,7 @@ objtool-y +=3D check.o objtool-y +=3D special.o objtool-y +=3D builtin-check.o objtool-y +=3D cfi.o +objtool-y +=3D insn.o objtool-y +=3D elf.o objtool-y +=3D objtool.o =20 diff --git a/tools/objtool/check.c b/tools/objtool/check.c index e6a2afa08748..d208086a8a18 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -28,86 +28,6 @@ struct alternative { bool skip_orig; }; =20 -struct instruction *find_insn(struct objtool_file *file, - struct section *sec, unsigned long offset) -{ - struct instruction *insn; - - hash_for_each_possible(file->insn_hash, insn, hash, sec_offset_hash(sec, = offset)) { - if (insn->sec =3D=3D sec && insn->offset =3D=3D offset) - return insn; - } - - return NULL; -} - -static struct instruction *next_insn_same_sec(struct objtool_file *file, - struct instruction *insn) -{ - struct instruction *next =3D list_next_entry(insn, list); - - if (!next || &next->list =3D=3D &file->insn_list || next->sec !=3D insn->= sec) - return NULL; - - return next; -} - -static struct instruction *next_insn_same_func(struct objtool_file *file, - struct instruction *insn) -{ - struct instruction *next =3D list_next_entry(insn, list); - struct symbol *func =3D insn->func; - - if (!func) - return NULL; - - if (&next->list !=3D &file->insn_list && next->func =3D=3D func) - return next; - - /* Check if we're already in the subfunction: */ - if (func =3D=3D func->cfunc) - return NULL; - - /* Move to the subfunction: */ - return find_insn(file, func->cfunc->sec, func->cfunc->offset); -} - -static struct instruction *prev_insn_same_sym(struct objtool_file *file, - struct instruction *insn) -{ - struct instruction *prev =3D list_prev_entry(insn, list); - - if (&prev->list !=3D &file->insn_list && prev->func =3D=3D insn->func) - return prev; - - return NULL; -} - -#define func_for_each_insn(file, func, insn) \ - for (insn =3D find_insn(file, func->sec, func->offset); \ - insn; \ - insn =3D next_insn_same_func(file, insn)) - -#define sym_for_each_insn(file, sym, insn) \ - for (insn =3D find_insn(file, sym->sec, sym->offset); \ - insn && &insn->list !=3D &file->insn_list && \ - insn->sec =3D=3D sym->sec && \ - insn->offset < sym->offset + sym->len; \ - insn =3D list_next_entry(insn, list)) - -#define sym_for_each_insn_continue_reverse(file, sym, insn) \ - for (insn =3D list_prev_entry(insn, list); \ - &insn->list !=3D &file->insn_list && \ - insn->sec =3D=3D sym->sec && insn->offset >=3D sym->offset; \ - insn =3D list_prev_entry(insn, list)) - -#define sec_for_each_insn_from(file, insn) \ - for (; insn; insn =3D next_insn_same_sec(file, insn)) - -#define sec_for_each_insn_continue(file, insn) \ - for (insn =3D next_insn_same_sec(file, insn); insn; \ - insn =3D next_insn_same_sec(file, insn)) - static bool is_jump_table_jump(struct instruction *insn) { struct alt_group *alt_group =3D insn->alt_group; @@ -249,21 +169,6 @@ static bool dead_end_function(struct objtool_file *fil= e, struct symbol *func) return __dead_end_function(file, func, 0); } =20 -static void init_insn_state(struct objtool_file *file, struct insn_state *= state, - struct section *sec) -{ - memset(state, 0, sizeof(*state)); - init_cfi_state(&state->cfi); - - /* - * We need the full vmlinux for noinstr validation, otherwise we can - * not correctly determine insn->call_dest->sec (external symbols do - * not have a section). - */ - if (opts.link && opts.noinstr && sec) - state->noinstr =3D sec->noinstr; -} - static unsigned long nr_insns; static unsigned long nr_insns_visited; =20 @@ -439,19 +344,6 @@ static int init_pv_ops(struct objtool_file *file) return 0; } =20 -static struct instruction *find_last_insn(struct objtool_file *file, - struct section *sec) -{ - struct instruction *insn =3D NULL; - unsigned int offset; - unsigned int end =3D (sec->sh.sh_size > 10) ? sec->sh.sh_size - 10 : 0; - - for (offset =3D sec->sh.sh_size - 1; offset >=3D end && !insn; offset--) - insn =3D find_insn(file, sec, offset); - - return insn; -} - /* * Mark "ud2" instructions and manually annotated dead ends. */ @@ -1072,28 +964,6 @@ __weak bool arch_is_rethunk(struct symbol *sym) return false; } =20 -#define NEGATIVE_RELOC ((void *)-1L) - -static struct reloc *insn_reloc(struct objtool_file *file, struct instruct= ion *insn) -{ - if (insn->reloc =3D=3D NEGATIVE_RELOC) - return NULL; - - if (!insn->reloc) { - if (!file) - return NULL; - - insn->reloc =3D find_reloc_by_dest_range(file->elf, insn->sec, - insn->offset, insn->len); - if (!insn->reloc) { - insn->reloc =3D NEGATIVE_RELOC; - return NULL; - } - } - - return insn->reloc; -} - static void remove_insn_ops(struct instruction *insn) { struct stack_op *op, *tmp; @@ -1252,27 +1122,6 @@ static void add_return_call(struct objtool_file *fil= e, struct instruction *insn, list_add_tail(&insn->call_node, &file->return_thunk_list); } =20 -static bool same_function(struct instruction *insn1, struct instruction *i= nsn2) -{ - return insn1->func->pfunc =3D=3D insn2->func->pfunc; -} - -static bool is_first_func_insn(struct objtool_file *file, struct instructi= on *insn) -{ - if (insn->offset =3D=3D insn->func->offset) - return true; - - if (opts.ibt) { - struct instruction *prev =3D prev_insn_same_sym(file, insn); - - if (prev && prev->type =3D=3D INSN_ENDBR && - insn->offset =3D=3D insn->func->offset + prev->len) - return true; - } - - return false; -} - /* * Find the destination instructions for all jumps. */ @@ -2987,56 +2836,6 @@ static int handle_insn_ops(struct instruction *insn, return 0; } =20 -static bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi= 2) -{ - struct cfi_state *cfi1 =3D insn->cfi; - int i; - - if (!cfi1) { - WARN("CFI missing"); - return false; - } - - if (memcmp(&cfi1->cfa, &cfi2->cfa, sizeof(cfi1->cfa))) { - - WARN_FUNC("stack state mismatch: cfa1=3D%d%+d cfa2=3D%d%+d", - insn->sec, insn->offset, - cfi1->cfa.base, cfi1->cfa.offset, - cfi2->cfa.base, cfi2->cfa.offset); - - } else if (memcmp(&cfi1->regs, &cfi2->regs, sizeof(cfi1->regs))) { - for (i =3D 0; i < CFI_NUM_REGS; i++) { - if (!memcmp(&cfi1->regs[i], &cfi2->regs[i], - sizeof(struct cfi_reg))) - continue; - - WARN_FUNC("stack state mismatch: reg1[%d]=3D%d%+d reg2[%d]=3D%d%+d", - insn->sec, insn->offset, - i, cfi1->regs[i].base, cfi1->regs[i].offset, - i, cfi2->regs[i].base, cfi2->regs[i].offset); - break; - } - - } else if (cfi1->type !=3D cfi2->type) { - - WARN_FUNC("stack state mismatch: type1=3D%d type2=3D%d", - insn->sec, insn->offset, cfi1->type, cfi2->type); - - } else if (cfi1->drap !=3D cfi2->drap || - (cfi1->drap && cfi1->drap_reg !=3D cfi2->drap_reg) || - (cfi1->drap && cfi1->drap_offset !=3D cfi2->drap_offset)) { - - WARN_FUNC("stack state mismatch: drap1=3D%d(%d,%d) drap2=3D%d(%d,%d)", - insn->sec, insn->offset, - cfi1->drap, cfi1->drap_reg, cfi1->drap_offset, - cfi2->drap, cfi2->drap_reg, cfi2->drap_offset); - - } else - return true; - - return false; -} - static inline bool func_uaccess_safe(struct symbol *func) { if (func) diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/= objtool/check.h index 036129cebeee..a093f5cb100a 100644 --- a/tools/objtool/include/objtool/check.h +++ b/tools/objtool/include/objtool/check.h @@ -7,17 +7,7 @@ #define _CHECK_H =20 #include -#include -#include - -struct insn_state { - struct cfi_state cfi; - unsigned int uaccess_stack; - bool uaccess; - bool df; - bool noinstr; - s8 instr; -}; +#include =20 struct alt_group { /* @@ -36,74 +26,9 @@ struct alt_group { struct cfi_state **cfi; }; =20 -struct instruction { - struct list_head list; - struct hlist_node hash; - struct list_head call_node; - struct section *sec; - unsigned long offset; - unsigned int len; - enum insn_type type; - unsigned long immediate; - - u16 dead_end : 1, - ignore : 1, - ignore_alts : 1, - hint : 1, - save : 1, - restore : 1, - retpoline_safe : 1, - noendbr : 1, - entry : 1; - /* 7 bit hole */ - - s8 instr; - u8 visited; - - struct alt_group *alt_group; - struct symbol *call_dest; - struct instruction *jump_dest; - struct instruction *first_jump_src; - struct reloc *jump_table; - struct reloc *reloc; - struct list_head alts; - struct symbol *func; - struct list_head stack_ops; - struct cfi_state *cfi; -}; - #define VISITED_BRANCH 0x01 #define VISITED_BRANCH_UACCESS 0x02 #define VISITED_BRANCH_MASK 0x03 #define VISITED_ENTRY 0x04 =20 -static inline bool is_static_jump(struct instruction *insn) -{ - return insn->type =3D=3D INSN_JUMP_CONDITIONAL || - insn->type =3D=3D INSN_JUMP_UNCONDITIONAL; -} - -static inline bool is_dynamic_jump(struct instruction *insn) -{ - return insn->type =3D=3D INSN_JUMP_DYNAMIC || - insn->type =3D=3D INSN_JUMP_DYNAMIC_CONDITIONAL; -} - -static inline bool is_jump(struct instruction *insn) -{ - return is_static_jump(insn) || is_dynamic_jump(insn); -} - -struct instruction *find_insn(struct objtool_file *file, - struct section *sec, unsigned long offset); - -#define for_each_insn(file, insn) \ - list_for_each_entry(insn, &file->insn_list, list) - -#define sec_for_each_insn(file, sec, insn) \ - for (insn =3D find_insn(file, sec, 0); \ - insn && &insn->list !=3D &file->insn_list && \ - insn->sec =3D=3D sec; \ - insn =3D list_next_entry(insn, list)) - #endif /* _CHECK_H */ diff --git a/tools/objtool/include/objtool/insn.h b/tools/objtool/include/o= bjtool/insn.h new file mode 100644 index 000000000000..b40756a38994 --- /dev/null +++ b/tools/objtool/include/objtool/insn.h @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2017 Josh Poimboeuf + */ + +#ifndef _INSN_H +#define _INSN_H + +#include +#include + +struct insn_state { + struct cfi_state cfi; + unsigned int uaccess_stack; + bool uaccess; + bool df; + bool noinstr; + s8 instr; +}; + +struct instruction { + struct list_head list; + struct hlist_node hash; + struct list_head call_node; + struct section *sec; + unsigned long offset; + unsigned int len; + enum insn_type type; + unsigned long immediate; + + u16 dead_end : 1, + ignore : 1, + ignore_alts : 1, + hint : 1, + save : 1, + restore : 1, + retpoline_safe : 1, + noendbr : 1, + entry : 1; + /* 7 bit hole */ + + s8 instr; + u8 visited; + + struct alt_group *alt_group; + struct symbol *call_dest; + struct instruction *jump_dest; + struct instruction *first_jump_src; + struct reloc *jump_table; + struct reloc *reloc; + struct list_head alts; + struct symbol *func; + struct list_head stack_ops; + struct cfi_state *cfi; +}; + +static inline bool is_static_jump(struct instruction *insn) +{ + return insn->type =3D=3D INSN_JUMP_CONDITIONAL || + insn->type =3D=3D INSN_JUMP_UNCONDITIONAL; +} + +static inline bool is_dynamic_jump(struct instruction *insn) +{ + return insn->type =3D=3D INSN_JUMP_DYNAMIC || + insn->type =3D=3D INSN_JUMP_DYNAMIC_CONDITIONAL; +} + +static inline bool is_jump(struct instruction *insn) +{ + return is_static_jump(insn) || is_dynamic_jump(insn); +} + +void init_insn_state(struct objtool_file *file, struct insn_state *state, + struct section *sec); +struct instruction *find_insn(struct objtool_file *file, + struct section *sec, unsigned long offset); +struct instruction *find_last_insn(struct objtool_file *file, + struct section *sec); +struct instruction *prev_insn_same_sym(struct objtool_file *file, + struct instruction *insn); +struct instruction *next_insn_same_sec(struct objtool_file *file, + struct instruction *insn); +struct instruction *next_insn_same_func(struct objtool_file *file, + struct instruction *insn); +struct reloc *insn_reloc(struct objtool_file *file, struct instruction *in= sn); +bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi2); +bool same_function(struct instruction *insn1, struct instruction *insn2); +bool is_first_func_insn(struct objtool_file *file, struct instruction *ins= n); + +#define for_each_insn(file, insn) \ + list_for_each_entry(insn, &file->insn_list, list) + +#define sec_for_each_insn(file, sec, insn) \ + for (insn =3D find_insn(file, sec, 0); \ + insn && &insn->list !=3D &file->insn_list && \ + insn->sec =3D=3D sec; \ + insn =3D list_next_entry(insn, list)) + +#define func_for_each_insn(file, func, insn) \ + for (insn =3D find_insn(file, func->sec, func->offset); \ + insn; \ + insn =3D next_insn_same_func(file, insn)) + +#define sym_for_each_insn(file, sym, insn) \ + for (insn =3D find_insn(file, sym->sec, sym->offset); \ + insn && &insn->list !=3D &file->insn_list && \ + insn->sec =3D=3D sym->sec && \ + insn->offset < sym->offset + sym->len; \ + insn =3D list_next_entry(insn, list)) + +#define sym_for_each_insn_continue_reverse(file, sym, insn) \ + for (insn =3D list_prev_entry(insn, list); \ + &insn->list !=3D &file->insn_list && \ + insn->sec =3D=3D sym->sec && insn->offset >=3D sym->offset; \ + insn =3D list_prev_entry(insn, list)) + +#define sec_for_each_insn_from(file, insn) \ + for (; insn; insn =3D next_insn_same_sec(file, insn)) + +#define sec_for_each_insn_continue(file, insn) \ + for (insn =3D next_insn_same_sec(file, insn); insn; \ + insn =3D next_insn_same_sec(file, insn)) + +#endif /* _INSN_H */ diff --git a/tools/objtool/insn.c b/tools/objtool/insn.c new file mode 100644 index 000000000000..e570b46ad39e --- /dev/null +++ b/tools/objtool/insn.c @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2015-2017 Josh Poimboeuf + */ + +#include + +#include +#include +#include + +struct instruction *find_insn(struct objtool_file *file, + struct section *sec, unsigned long offset) +{ + struct instruction *insn; + + hash_for_each_possible(file->insn_hash, insn, hash, sec_offset_hash(sec, = offset)) { + if (insn->sec =3D=3D sec && insn->offset =3D=3D offset) + return insn; + } + + return NULL; +} + +struct instruction *next_insn_same_sec(struct objtool_file *file, + struct instruction *insn) +{ + struct instruction *next =3D list_next_entry(insn, list); + + if (!next || &next->list =3D=3D &file->insn_list || next->sec !=3D insn->= sec) + return NULL; + + return next; +} + +struct instruction *next_insn_same_func(struct objtool_file *file, + struct instruction *insn) +{ + struct instruction *next =3D list_next_entry(insn, list); + struct symbol *func =3D insn->func; + + if (!func) + return NULL; + + if (&next->list !=3D &file->insn_list && next->func =3D=3D func) + return next; + + /* Check if we're already in the subfunction: */ + if (func =3D=3D func->cfunc) + return NULL; + + /* Move to the subfunction: */ + return find_insn(file, func->cfunc->sec, func->cfunc->offset); +} + +struct instruction *prev_insn_same_sym(struct objtool_file *file, + struct instruction *insn) +{ + struct instruction *prev =3D list_prev_entry(insn, list); + + if (&prev->list !=3D &file->insn_list && prev->func =3D=3D insn->func) + return prev; + + return NULL; +} + +void init_insn_state(struct objtool_file *file, struct insn_state *state, + struct section *sec) +{ + memset(state, 0, sizeof(*state)); + init_cfi_state(&state->cfi); + + /* + * We need the full vmlinux for noinstr validation, otherwise we can + * not correctly determine insn->call_dest->sec (external symbols do + * not have a section). + */ + if (opts.link && opts.noinstr && sec) + state->noinstr =3D sec->noinstr; +} + +struct instruction *find_last_insn(struct objtool_file *file, + struct section *sec) +{ + struct instruction *insn =3D NULL; + unsigned int offset; + unsigned int end =3D (sec->sh.sh_size > 10) ? sec->sh.sh_size - 10 : 0; + + for (offset =3D sec->sh.sh_size - 1; offset >=3D end && !insn; offset--) + insn =3D find_insn(file, sec, offset); + + return insn; +} + +#define NEGATIVE_RELOC ((void *)-1L) + +struct reloc *insn_reloc(struct objtool_file *file, struct instruction *in= sn) +{ + if (insn->reloc =3D=3D NEGATIVE_RELOC) + return NULL; + + if (!insn->reloc) { + if (!file) + return NULL; + + insn->reloc =3D find_reloc_by_dest_range(file->elf, insn->sec, + insn->offset, insn->len); + if (!insn->reloc) { + insn->reloc =3D NEGATIVE_RELOC; + return NULL; + } + } + + return insn->reloc; +} + +bool same_function(struct instruction *insn1, struct instruction *insn2) +{ + return insn1->func->pfunc =3D=3D insn2->func->pfunc; +} + +bool is_first_func_insn(struct objtool_file *file, struct instruction *ins= n) +{ + if (insn->offset =3D=3D insn->func->offset) + return true; + + if (opts.ibt) { + struct instruction *prev =3D prev_insn_same_sym(file, insn); + + if (prev && prev->type =3D=3D INSN_ENDBR && + insn->offset =3D=3D insn->func->offset + prev->len) + return true; + } + + return false; +} + +bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi2) +{ + struct cfi_state *cfi1 =3D insn->cfi; + int i; + + if (!cfi1) { + WARN("CFI missing"); + return false; + } + + if (memcmp(&cfi1->cfa, &cfi2->cfa, sizeof(cfi1->cfa))) { + + WARN_FUNC("stack state mismatch: cfa1=3D%d%+d cfa2=3D%d%+d", + insn->sec, insn->offset, + cfi1->cfa.base, cfi1->cfa.offset, + cfi2->cfa.base, cfi2->cfa.offset); + + } else if (memcmp(&cfi1->regs, &cfi2->regs, sizeof(cfi1->regs))) { + for (i =3D 0; i < CFI_NUM_REGS; i++) { + if (!memcmp(&cfi1->regs[i], &cfi2->regs[i], + sizeof(struct cfi_reg))) + continue; + + WARN_FUNC("stack state mismatch: reg1[%d]=3D%d%+d reg2[%d]=3D%d%+d", + insn->sec, insn->offset, + i, cfi1->regs[i].base, cfi1->regs[i].offset, + i, cfi2->regs[i].base, cfi2->regs[i].offset); + break; + } + + } else if (cfi1->type !=3D cfi2->type) { + + WARN_FUNC("stack state mismatch: type1=3D%d type2=3D%d", + insn->sec, insn->offset, cfi1->type, cfi2->type); + + } else if (cfi1->drap !=3D cfi2->drap || + (cfi1->drap && cfi1->drap_reg !=3D cfi2->drap_reg) || + (cfi1->drap && cfi1->drap_offset !=3D cfi2->drap_offset)) { + + WARN_FUNC("stack state mismatch: drap1=3D%d(%d,%d) drap2=3D%d(%d,%d)", + insn->sec, insn->offset, + cfi1->drap, cfi1->drap_reg, cfi1->drap_offset, + cfi2->drap, cfi2->drap_reg, cfi2->drap_offset); + + } else + return true; + + return false; +} --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EE0CC05027 for ; Thu, 2 Feb 2023 07:42:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231894AbjBBHmh (ORCPT ); Thu, 2 Feb 2023 02:42:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231752AbjBBHma (ORCPT ); Thu, 2 Feb 2023 02:42:30 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B6F96751AD; Wed, 1 Feb 2023 23:42:28 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id B550620B74F9; Wed, 1 Feb 2023 23:42:27 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B550620B74F9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323748; bh=VgJmVJl6Papix+FhD8GwfbZQPwIUNdokfUvzBDbDxWI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=IDYMbof1UTFfrC5kbDefzSA8lVeQcs595K/O7jT7jQiIyQjeMmrU+FNh9s6oyigYw 6+fCMZAcA9nxed7a8EJRiRrdCto8ECYtpXzlriME7w0dNdRd+ltE0j2FGHEbb72Nzy jBt5KRq7KPVb3EHA6Va+x3Mb2KZ+l6S5jVdGVq+A= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 03/22] objtool: Move decode_instructions() to a separate file Date: Thu, 2 Feb 2023 01:40:17 -0600 Message-Id: <20230202074036.507249-4-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" check.c implements static stack validation. But decode_instructions() which resides in it can be shared with other types of validation. E.g., dynamic FP validation. Move the function to its own file - decode.c. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/Build | 1 + tools/objtool/check.c | 97 ------------------------ tools/objtool/decode.c | 107 +++++++++++++++++++++++++++ tools/objtool/include/objtool/insn.h | 2 + 4 files changed, 110 insertions(+), 97 deletions(-) create mode 100644 tools/objtool/decode.c diff --git a/tools/objtool/Build b/tools/objtool/Build index 1149048e6b3e..8afe56cd0c2d 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -7,6 +7,7 @@ objtool-y +=3D special.o objtool-y +=3D builtin-check.o objtool-y +=3D cfi.o objtool-y +=3D insn.o +objtool-y +=3D decode.o objtool-y +=3D elf.o objtool-y +=3D objtool.o =20 diff --git a/tools/objtool/check.c b/tools/objtool/check.c index d208086a8a18..be3f6564104a 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -169,105 +169,8 @@ static bool dead_end_function(struct objtool_file *fi= le, struct symbol *func) return __dead_end_function(file, func, 0); } =20 -static unsigned long nr_insns; static unsigned long nr_insns_visited; =20 -/* - * Call the arch-specific instruction decoder for all the instructions and= add - * them to the global instruction list. - */ -static int decode_instructions(struct objtool_file *file) -{ - struct section *sec; - struct symbol *func; - unsigned long offset; - struct instruction *insn; - int ret; - - for_each_sec(file, sec) { - - if (!(sec->sh.sh_flags & SHF_EXECINSTR)) - continue; - - if (strcmp(sec->name, ".altinstr_replacement") && - strcmp(sec->name, ".altinstr_aux") && - strncmp(sec->name, ".discard.", 9)) - sec->text =3D true; - - if (!strcmp(sec->name, ".noinstr.text") || - !strcmp(sec->name, ".entry.text") || - !strncmp(sec->name, ".text.__x86.", 12)) - sec->noinstr =3D true; - - for (offset =3D 0; offset < sec->sh.sh_size; offset +=3D insn->len) { - insn =3D malloc(sizeof(*insn)); - if (!insn) { - WARN("malloc failed"); - return -1; - } - memset(insn, 0, sizeof(*insn)); - INIT_LIST_HEAD(&insn->alts); - INIT_LIST_HEAD(&insn->stack_ops); - INIT_LIST_HEAD(&insn->call_node); - - insn->sec =3D sec; - insn->offset =3D offset; - - ret =3D arch_decode_instruction(file, sec, offset, - sec->sh.sh_size - offset, - &insn->len, &insn->type, - &insn->immediate, - &insn->stack_ops); - if (ret) - goto err; - - /* - * By default, "ud2" is a dead end unless otherwise - * annotated, because GCC 7 inserts it for certain - * divide-by-zero cases. - */ - if (insn->type =3D=3D INSN_BUG) - insn->dead_end =3D true; - - hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offse= t)); - list_add_tail(&insn->list, &file->insn_list); - nr_insns++; - } - - list_for_each_entry(func, &sec->symbol_list, list) { - if (func->type !=3D STT_FUNC || func->alias !=3D func) - continue; - - if (!find_insn(file, sec, func->offset)) { - WARN("%s(): can't find starting instruction", - func->name); - return -1; - } - - sym_for_each_insn(file, func, insn) { - insn->func =3D func; - if (insn->type =3D=3D INSN_ENDBR && list_empty(&insn->call_node)) { - if (insn->offset =3D=3D insn->func->offset) { - list_add_tail(&insn->call_node, &file->endbr_list); - file->nr_endbr++; - } else { - file->nr_endbr_int++; - } - } - } - } - } - - if (opts.stats) - printf("nr_insns: %lu\n", nr_insns); - - return 0; - -err: - free(insn); - return ret; -} - /* * Read the pv_ops[] .data table to find the static initialized values. */ diff --git a/tools/objtool/decode.c b/tools/objtool/decode.c new file mode 100644 index 000000000000..dcec3efc2afb --- /dev/null +++ b/tools/objtool/decode.c @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2015-2017 Josh Poimboeuf + */ +#include + +#include +#include +#include + +static unsigned long nr_insns; + +/* + * Call the arch-specific instruction decoder for all the instructions and= add + * them to the global instruction list. + */ +int decode_instructions(struct objtool_file *file) +{ + struct section *sec; + struct symbol *func; + unsigned long offset; + struct instruction *insn; + int ret; + + for_each_sec(file, sec) { + + if (!(sec->sh.sh_flags & SHF_EXECINSTR)) + continue; + + if (strcmp(sec->name, ".altinstr_replacement") && + strcmp(sec->name, ".altinstr_aux") && + strncmp(sec->name, ".discard.", 9)) + sec->text =3D true; + + if (!strcmp(sec->name, ".noinstr.text") || + !strcmp(sec->name, ".entry.text") || + !strncmp(sec->name, ".text.__x86.", 12)) + sec->noinstr =3D true; + + for (offset =3D 0; offset < sec->sh.sh_size; offset +=3D insn->len) { + insn =3D malloc(sizeof(*insn)); + if (!insn) { + WARN("malloc failed"); + return -1; + } + memset(insn, 0, sizeof(*insn)); + INIT_LIST_HEAD(&insn->alts); + INIT_LIST_HEAD(&insn->stack_ops); + INIT_LIST_HEAD(&insn->call_node); + + insn->sec =3D sec; + insn->offset =3D offset; + + ret =3D arch_decode_instruction(file, sec, offset, + sec->sh.sh_size - offset, + &insn->len, &insn->type, + &insn->immediate, + &insn->stack_ops); + if (ret) + goto err; + + /* + * By default, "ud2" is a dead end unless otherwise + * annotated, because GCC 7 inserts it for certain + * divide-by-zero cases. + */ + if (insn->type =3D=3D INSN_BUG) + insn->dead_end =3D true; + + hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offse= t)); + list_add_tail(&insn->list, &file->insn_list); + nr_insns++; + } + + list_for_each_entry(func, &sec->symbol_list, list) { + if (func->type !=3D STT_FUNC || func->alias !=3D func) + continue; + + if (!find_insn(file, sec, func->offset)) { + WARN("%s(): can't find starting instruction", + func->name); + return -1; + } + + sym_for_each_insn(file, func, insn) { + insn->func =3D func; + if (insn->type =3D=3D INSN_ENDBR && list_empty(&insn->call_node)) { + if (insn->offset =3D=3D insn->func->offset) { + list_add_tail(&insn->call_node, &file->endbr_list); + file->nr_endbr++; + } else { + file->nr_endbr_int++; + } + } + } + } + } + + if (opts.stats) + printf("nr_insns: %lu\n", nr_insns); + + return 0; + +err: + free(insn); + return ret; +} diff --git a/tools/objtool/include/objtool/insn.h b/tools/objtool/include/o= bjtool/insn.h index b40756a38994..b74c7f0d9076 100644 --- a/tools/objtool/include/objtool/insn.h +++ b/tools/objtool/include/objtool/insn.h @@ -88,6 +88,8 @@ bool insn_cfi_match(struct instruction *insn, struct cfi_= state *cfi2); bool same_function(struct instruction *insn1, struct instruction *insn2); bool is_first_func_insn(struct objtool_file *file, struct instruction *ins= n); =20 +int decode_instructions(struct objtool_file *file); + #define for_each_insn(file, insn) \ list_for_each_entry(insn, &file->insn_list, list) =20 --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87275C61DA4 for ; Thu, 2 Feb 2023 07:42:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231860AbjBBHmo (ORCPT ); Thu, 2 Feb 2023 02:42:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231835AbjBBHmc (ORCPT ); Thu, 2 Feb 2023 02:42:32 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B917E7E6F5; Wed, 1 Feb 2023 23:42:29 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id B454320B74FA; Wed, 1 Feb 2023 23:42:28 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B454320B74FA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323749; bh=+/y2tMn2FnirOqRAseCTBeZchbeMMKab+5yuM1+A6NU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=O2QqQRpqku/OxCzPiuBhq1viDeLThz+6r1uMsO3PE+Mp3Gp7BuUYoNEqpm1rdRw1K fBqCRxnsBcEIStU/GABez5rQd2r13Yhkzs5ukcaZ93ckKqGSGwdCrvZStMdEbu9pFo uK0GQBZ1IszPYuZBjmDQlylVLr3DdjzvDlFdUrbw= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 04/22] objtool: Reorganize Unwind hint code Date: Thu, 2 Feb 2023 01:40:18 -0600 Message-Id: <20230202074036.507249-5-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Unwind hint macros and struct unwind_hint are arch-specific. Move them into the arch-specific file asm/unwind_hints.h. But the unwind hint types are generic. Retain them in linux/objtool.h. Unwind hints can be used with static stack validation as well as other forms of validation such as dynamic FP validation. Move the function read_unwind_hints() from check.c to a new file unwind_hints.c so that it can be shared across validation schemes. Signed-off-by: Madhavan T. Venkataraman --- arch/x86/include/asm/unwind_hints.h | 83 ++++++++++++ arch/x86/kernel/unwind_orc.c | 2 +- include/linux/objtool.h | 67 --------- tools/arch/x86/include/asm/unwind_hints.h | 157 ++++++++++++++++++++++ tools/include/linux/objtool.h | 67 --------- tools/objtool/Build | 1 + tools/objtool/check.c | 96 ------------- tools/objtool/include/objtool/insn.h | 1 + tools/objtool/sync-check.sh | 1 + tools/objtool/unwind_hints.c | 106 +++++++++++++++ 10 files changed, 350 insertions(+), 231 deletions(-) create mode 100644 tools/arch/x86/include/asm/unwind_hints.h create mode 100644 tools/objtool/unwind_hints.c diff --git a/arch/x86/include/asm/unwind_hints.h b/arch/x86/include/asm/unw= ind_hints.h index f66fbe6537dd..07c8d911266c 100644 --- a/arch/x86/include/asm/unwind_hints.h +++ b/arch/x86/include/asm/unwind_hints.h @@ -1,10 +1,93 @@ #ifndef _ASM_X86_UNWIND_HINTS_H #define _ASM_X86_UNWIND_HINTS_H =20 +#ifndef __ASSEMBLY__ + +#include + +/* + * This struct is used by asm and inline asm code to manually annotate the + * location of registers on the stack. + */ +struct unwind_hint { + u32 ip; + s16 sp_offset; + u8 sp_reg; + u8 type; + u8 end; +}; +#endif + #include =20 #include "orc_types.h" =20 +#ifdef CONFIG_OBJTOOL + +#ifndef __ASSEMBLY__ + +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "987: \n\t" \ + ".pushsection .discard.unwind_hints\n\t" \ + /* struct unwind_hint */ \ + ".long 987b - .\n\t" \ + ".short " __stringify(sp_offset) "\n\t" \ + ".byte " __stringify(sp_reg) "\n\t" \ + ".byte " __stringify(type) "\n\t" \ + ".byte " __stringify(end) "\n\t" \ + ".balign 4 \n\t" \ + ".popsection\n\t" + +#else /* __ASSEMBLY__ */ + +/* + * In asm, there are two kinds of code: normal C-type callable functions a= nd + * the rest. The normal callable functions can be called by other code, a= nd + * don't do anything unusual with the stack. Such normal callable functio= ns + * are annotated with the ENTRY/ENDPROC macros. Most asm code falls in th= is + * category. In this case, no special debugging annotations are needed be= cause + * objtool can automatically generate the ORC data for the ORC unwinder to= read + * at runtime. + * + * Anything which doesn't fall into the above category, such as syscall and + * interrupt handlers, tends to not be called directly by other functions,= and + * often does unusual non-C-function-type things with the stack pointer. = Such + * code needs to be annotated such that objtool can understand it. The + * following CFI hint macros are for this type of code. + * + * These macros provide hints to objtool about the state of the stack at e= ach + * instruction. Objtool starts from the hints and follows the code flow, + * making automatic CFI adjustments when it sees pushes and pops, filling = out + * the debuginfo as necessary. It will also warn if it sees any + * inconsistencies. + */ +.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 +.Lunwind_hint_ip_\@: + .pushsection .discard.unwind_hints + /* struct unwind_hint */ + .long .Lunwind_hint_ip_\@ - . + .short \sp_offset + .byte \sp_reg + .byte \type + .byte \end + .balign 4 + .popsection +.endm + +#endif /* __ASSEMBLY__ */ + +#else /* !CONFIG_OBJTOOL */ + +#ifndef __ASSEMBLY__ +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "\n\t" +#else +.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 +.endm +#endif + +#endif /* CONFIG_OBJTOOL */ + #ifdef __ASSEMBLY__ =20 .macro UNWIND_HINT_EMPTY diff --git a/arch/x86/kernel/unwind_orc.c b/arch/x86/kernel/unwind_orc.c index c059820dfaea..c2bfc597d909 100644 --- a/arch/x86/kernel/unwind_orc.c +++ b/arch/x86/kernel/unwind_orc.c @@ -1,10 +1,10 @@ // SPDX-License-Identifier: GPL-2.0-only -#include #include #include #include #include #include +#include #include #include =20 diff --git a/include/linux/objtool.h b/include/linux/objtool.h index 62c54ffbeeaa..1af295efc12c 100644 --- a/include/linux/objtool.h +++ b/include/linux/objtool.h @@ -2,23 +2,6 @@ #ifndef _LINUX_OBJTOOL_H #define _LINUX_OBJTOOL_H =20 -#ifndef __ASSEMBLY__ - -#include - -/* - * This struct is used by asm and inline asm code to manually annotate the - * location of registers on the stack. - */ -struct unwind_hint { - u32 ip; - s16 sp_offset; - u8 sp_reg; - u8 type; - u8 end; -}; -#endif - /* * UNWIND_HINT_TYPE_CALL: Indicates that sp_reg+sp_offset resolves to PREV= _SP * (the caller's SP right before it made the call). Used for all callable @@ -49,18 +32,6 @@ struct unwind_hint { =20 #ifndef __ASSEMBLY__ =20 -#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ - "987: \n\t" \ - ".pushsection .discard.unwind_hints\n\t" \ - /* struct unwind_hint */ \ - ".long 987b - .\n\t" \ - ".short " __stringify(sp_offset) "\n\t" \ - ".byte " __stringify(sp_reg) "\n\t" \ - ".byte " __stringify(type) "\n\t" \ - ".byte " __stringify(end) "\n\t" \ - ".balign 4 \n\t" \ - ".popsection\n\t" - /* * This macro marks the given function's stack frame as "non-standard", wh= ich * tells objtool to ignore the function when doing stack metadata validati= on. @@ -108,40 +79,6 @@ struct unwind_hint { .long 999b; \ .popsection; =20 -/* - * In asm, there are two kinds of code: normal C-type callable functions a= nd - * the rest. The normal callable functions can be called by other code, a= nd - * don't do anything unusual with the stack. Such normal callable functio= ns - * are annotated with the ENTRY/ENDPROC macros. Most asm code falls in th= is - * category. In this case, no special debugging annotations are needed be= cause - * objtool can automatically generate the ORC data for the ORC unwinder to= read - * at runtime. - * - * Anything which doesn't fall into the above category, such as syscall and - * interrupt handlers, tends to not be called directly by other functions,= and - * often does unusual non-C-function-type things with the stack pointer. = Such - * code needs to be annotated such that objtool can understand it. The - * following CFI hint macros are for this type of code. - * - * These macros provide hints to objtool about the state of the stack at e= ach - * instruction. Objtool starts from the hints and follows the code flow, - * making automatic CFI adjustments when it sees pushes and pops, filling = out - * the debuginfo as necessary. It will also warn if it sees any - * inconsistencies. - */ -.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 -.Lunwind_hint_ip_\@: - .pushsection .discard.unwind_hints - /* struct unwind_hint */ - .long .Lunwind_hint_ip_\@ - . - .short \sp_offset - .byte \sp_reg - .byte \type - .byte \end - .balign 4 - .popsection -.endm - .macro STACK_FRAME_NON_STANDARD func:req .pushsection .discard.func_stack_frame_non_standard, "aw" _ASM_PTR \func @@ -174,16 +111,12 @@ struct unwind_hint { =20 #ifndef __ASSEMBLY__ =20 -#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ - "\n\t" #define STACK_FRAME_NON_STANDARD(func) #define STACK_FRAME_NON_STANDARD_FP(func) #define ANNOTATE_NOENDBR #define ASM_REACHABLE #else #define ANNOTATE_INTRA_FUNCTION_CALL -.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 -.endm .macro STACK_FRAME_NON_STANDARD func:req .endm .macro ANNOTATE_NOENDBR diff --git a/tools/arch/x86/include/asm/unwind_hints.h b/tools/arch/x86/inc= lude/asm/unwind_hints.h new file mode 100644 index 000000000000..07c8d911266c --- /dev/null +++ b/tools/arch/x86/include/asm/unwind_hints.h @@ -0,0 +1,157 @@ +#ifndef _ASM_X86_UNWIND_HINTS_H +#define _ASM_X86_UNWIND_HINTS_H + +#ifndef __ASSEMBLY__ + +#include + +/* + * This struct is used by asm and inline asm code to manually annotate the + * location of registers on the stack. + */ +struct unwind_hint { + u32 ip; + s16 sp_offset; + u8 sp_reg; + u8 type; + u8 end; +}; +#endif + +#include + +#include "orc_types.h" + +#ifdef CONFIG_OBJTOOL + +#ifndef __ASSEMBLY__ + +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "987: \n\t" \ + ".pushsection .discard.unwind_hints\n\t" \ + /* struct unwind_hint */ \ + ".long 987b - .\n\t" \ + ".short " __stringify(sp_offset) "\n\t" \ + ".byte " __stringify(sp_reg) "\n\t" \ + ".byte " __stringify(type) "\n\t" \ + ".byte " __stringify(end) "\n\t" \ + ".balign 4 \n\t" \ + ".popsection\n\t" + +#else /* __ASSEMBLY__ */ + +/* + * In asm, there are two kinds of code: normal C-type callable functions a= nd + * the rest. The normal callable functions can be called by other code, a= nd + * don't do anything unusual with the stack. Such normal callable functio= ns + * are annotated with the ENTRY/ENDPROC macros. Most asm code falls in th= is + * category. In this case, no special debugging annotations are needed be= cause + * objtool can automatically generate the ORC data for the ORC unwinder to= read + * at runtime. + * + * Anything which doesn't fall into the above category, such as syscall and + * interrupt handlers, tends to not be called directly by other functions,= and + * often does unusual non-C-function-type things with the stack pointer. = Such + * code needs to be annotated such that objtool can understand it. The + * following CFI hint macros are for this type of code. + * + * These macros provide hints to objtool about the state of the stack at e= ach + * instruction. Objtool starts from the hints and follows the code flow, + * making automatic CFI adjustments when it sees pushes and pops, filling = out + * the debuginfo as necessary. It will also warn if it sees any + * inconsistencies. + */ +.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 +.Lunwind_hint_ip_\@: + .pushsection .discard.unwind_hints + /* struct unwind_hint */ + .long .Lunwind_hint_ip_\@ - . + .short \sp_offset + .byte \sp_reg + .byte \type + .byte \end + .balign 4 + .popsection +.endm + +#endif /* __ASSEMBLY__ */ + +#else /* !CONFIG_OBJTOOL */ + +#ifndef __ASSEMBLY__ +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "\n\t" +#else +.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 +.endm +#endif + +#endif /* CONFIG_OBJTOOL */ + +#ifdef __ASSEMBLY__ + +.macro UNWIND_HINT_EMPTY + UNWIND_HINT type=3DUNWIND_HINT_TYPE_CALL end=3D1 +.endm + +.macro UNWIND_HINT_ENTRY + UNWIND_HINT type=3DUNWIND_HINT_TYPE_ENTRY end=3D1 +.endm + +.macro UNWIND_HINT_REGS base=3D%rsp offset=3D0 indirect=3D0 extra=3D1 part= ial=3D0 + .if \base =3D=3D %rsp + .if \indirect + .set sp_reg, ORC_REG_SP_INDIRECT + .else + .set sp_reg, ORC_REG_SP + .endif + .elseif \base =3D=3D %rbp + .set sp_reg, ORC_REG_BP + .elseif \base =3D=3D %rdi + .set sp_reg, ORC_REG_DI + .elseif \base =3D=3D %rdx + .set sp_reg, ORC_REG_DX + .elseif \base =3D=3D %r10 + .set sp_reg, ORC_REG_R10 + .else + .error "UNWIND_HINT_REGS: bad base register" + .endif + + .set sp_offset, \offset + + .if \partial + .set type, UNWIND_HINT_TYPE_REGS_PARTIAL + .elseif \extra =3D=3D 0 + .set type, UNWIND_HINT_TYPE_REGS_PARTIAL + .set sp_offset, \offset + (16*8) + .else + .set type, UNWIND_HINT_TYPE_REGS + .endif + + UNWIND_HINT sp_reg=3Dsp_reg sp_offset=3Dsp_offset type=3Dtype +.endm + +.macro UNWIND_HINT_IRET_REGS base=3D%rsp offset=3D0 + UNWIND_HINT_REGS base=3D\base offset=3D\offset partial=3D1 +.endm + +.macro UNWIND_HINT_FUNC + UNWIND_HINT sp_reg=3DORC_REG_SP sp_offset=3D8 type=3DUNWIND_HINT_TYPE_FUNC +.endm + +.macro UNWIND_HINT_SAVE + UNWIND_HINT type=3DUNWIND_HINT_TYPE_SAVE +.endm + +.macro UNWIND_HINT_RESTORE + UNWIND_HINT type=3DUNWIND_HINT_TYPE_RESTORE +.endm + +#else + +#define UNWIND_HINT_FUNC \ + UNWIND_HINT(ORC_REG_SP, 8, UNWIND_HINT_TYPE_FUNC, 0) + +#endif /* __ASSEMBLY__ */ + +#endif /* _ASM_X86_UNWIND_HINTS_H */ diff --git a/tools/include/linux/objtool.h b/tools/include/linux/objtool.h index 62c54ffbeeaa..1af295efc12c 100644 --- a/tools/include/linux/objtool.h +++ b/tools/include/linux/objtool.h @@ -2,23 +2,6 @@ #ifndef _LINUX_OBJTOOL_H #define _LINUX_OBJTOOL_H =20 -#ifndef __ASSEMBLY__ - -#include - -/* - * This struct is used by asm and inline asm code to manually annotate the - * location of registers on the stack. - */ -struct unwind_hint { - u32 ip; - s16 sp_offset; - u8 sp_reg; - u8 type; - u8 end; -}; -#endif - /* * UNWIND_HINT_TYPE_CALL: Indicates that sp_reg+sp_offset resolves to PREV= _SP * (the caller's SP right before it made the call). Used for all callable @@ -49,18 +32,6 @@ struct unwind_hint { =20 #ifndef __ASSEMBLY__ =20 -#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ - "987: \n\t" \ - ".pushsection .discard.unwind_hints\n\t" \ - /* struct unwind_hint */ \ - ".long 987b - .\n\t" \ - ".short " __stringify(sp_offset) "\n\t" \ - ".byte " __stringify(sp_reg) "\n\t" \ - ".byte " __stringify(type) "\n\t" \ - ".byte " __stringify(end) "\n\t" \ - ".balign 4 \n\t" \ - ".popsection\n\t" - /* * This macro marks the given function's stack frame as "non-standard", wh= ich * tells objtool to ignore the function when doing stack metadata validati= on. @@ -108,40 +79,6 @@ struct unwind_hint { .long 999b; \ .popsection; =20 -/* - * In asm, there are two kinds of code: normal C-type callable functions a= nd - * the rest. The normal callable functions can be called by other code, a= nd - * don't do anything unusual with the stack. Such normal callable functio= ns - * are annotated with the ENTRY/ENDPROC macros. Most asm code falls in th= is - * category. In this case, no special debugging annotations are needed be= cause - * objtool can automatically generate the ORC data for the ORC unwinder to= read - * at runtime. - * - * Anything which doesn't fall into the above category, such as syscall and - * interrupt handlers, tends to not be called directly by other functions,= and - * often does unusual non-C-function-type things with the stack pointer. = Such - * code needs to be annotated such that objtool can understand it. The - * following CFI hint macros are for this type of code. - * - * These macros provide hints to objtool about the state of the stack at e= ach - * instruction. Objtool starts from the hints and follows the code flow, - * making automatic CFI adjustments when it sees pushes and pops, filling = out - * the debuginfo as necessary. It will also warn if it sees any - * inconsistencies. - */ -.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 -.Lunwind_hint_ip_\@: - .pushsection .discard.unwind_hints - /* struct unwind_hint */ - .long .Lunwind_hint_ip_\@ - . - .short \sp_offset - .byte \sp_reg - .byte \type - .byte \end - .balign 4 - .popsection -.endm - .macro STACK_FRAME_NON_STANDARD func:req .pushsection .discard.func_stack_frame_non_standard, "aw" _ASM_PTR \func @@ -174,16 +111,12 @@ struct unwind_hint { =20 #ifndef __ASSEMBLY__ =20 -#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ - "\n\t" #define STACK_FRAME_NON_STANDARD(func) #define STACK_FRAME_NON_STANDARD_FP(func) #define ANNOTATE_NOENDBR #define ASM_REACHABLE #else #define ANNOTATE_INTRA_FUNCTION_CALL -.macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 end=3D0 -.endm .macro STACK_FRAME_NON_STANDARD func:req .endm .macro ANNOTATE_NOENDBR diff --git a/tools/objtool/Build b/tools/objtool/Build index 8afe56cd0c2d..c4666d0b40ba 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -8,6 +8,7 @@ objtool-y +=3D builtin-check.o objtool-y +=3D cfi.o objtool-y +=3D insn.o objtool-y +=3D decode.o +objtool-y +=3D unwind_hints.o objtool-y +=3D elf.o objtool-y +=3D objtool.o =20 diff --git a/tools/objtool/check.c b/tools/objtool/check.c index be3f6564104a..d14a2b7b8b37 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -1670,102 +1670,6 @@ static int add_jump_table_alts(struct objtool_file = *file) return 0; } =20 -static int read_unwind_hints(struct objtool_file *file) -{ - struct cfi_state cfi =3D init_cfi; - struct section *sec, *relocsec; - struct unwind_hint *hint; - struct instruction *insn; - struct reloc *reloc; - int i; - - sec =3D find_section_by_name(file->elf, ".discard.unwind_hints"); - if (!sec) - return 0; - - relocsec =3D sec->reloc; - if (!relocsec) { - WARN("missing .rela.discard.unwind_hints section"); - return -1; - } - - if (sec->sh.sh_size % sizeof(struct unwind_hint)) { - WARN("struct unwind_hint size mismatch"); - return -1; - } - - file->hints =3D true; - - for (i =3D 0; i < sec->sh.sh_size / sizeof(struct unwind_hint); i++) { - hint =3D (struct unwind_hint *)sec->data->d_buf + i; - - reloc =3D find_reloc_by_dest(file->elf, sec, i * sizeof(*hint)); - if (!reloc) { - WARN("can't find reloc for unwind_hints[%d]", i); - return -1; - } - - insn =3D find_insn(file, reloc->sym->sec, reloc->addend); - if (!insn) { - WARN("can't find insn for unwind_hints[%d]", i); - return -1; - } - - insn->hint =3D true; - - if (hint->type =3D=3D UNWIND_HINT_TYPE_SAVE) { - insn->hint =3D false; - insn->save =3D true; - continue; - } - - if (hint->type =3D=3D UNWIND_HINT_TYPE_RESTORE) { - insn->restore =3D true; - continue; - } - - if (hint->type =3D=3D UNWIND_HINT_TYPE_REGS_PARTIAL) { - struct symbol *sym =3D find_symbol_by_offset(insn->sec, insn->offset); - - if (sym && sym->bind =3D=3D STB_GLOBAL) { - if (opts.ibt && insn->type !=3D INSN_ENDBR && !insn->noendbr) { - WARN_FUNC("UNWIND_HINT_IRET_REGS without ENDBR", - insn->sec, insn->offset); - } - - insn->entry =3D 1; - } - } - - if (hint->type =3D=3D UNWIND_HINT_TYPE_ENTRY) { - hint->type =3D UNWIND_HINT_TYPE_CALL; - insn->entry =3D 1; - } - - if (hint->type =3D=3D UNWIND_HINT_TYPE_FUNC) { - insn->cfi =3D &func_cfi; - continue; - } - - if (insn->cfi) - cfi =3D *(insn->cfi); - - if (arch_decode_hint_reg(hint->sp_reg, &cfi.cfa.base)) { - WARN_FUNC("unsupported unwind_hint sp base reg %d", - insn->sec, insn->offset, hint->sp_reg); - return -1; - } - - cfi.cfa.offset =3D bswap_if_needed(hint->sp_offset); - cfi.type =3D hint->type; - cfi.end =3D hint->end; - - insn->cfi =3D cfi_hash_find_or_add(&cfi); - } - - return 0; -} - static int read_noendbr_hints(struct objtool_file *file) { struct section *sec; diff --git a/tools/objtool/include/objtool/insn.h b/tools/objtool/include/o= bjtool/insn.h index b74c7f0d9076..cfd1ae7e2e8e 100644 --- a/tools/objtool/include/objtool/insn.h +++ b/tools/objtool/include/objtool/insn.h @@ -89,6 +89,7 @@ bool same_function(struct instruction *insn1, struct inst= ruction *insn2); bool is_first_func_insn(struct objtool_file *file, struct instruction *ins= n); =20 int decode_instructions(struct objtool_file *file); +int read_unwind_hints(struct objtool_file *file); =20 #define for_each_insn(file, insn) \ list_for_each_entry(insn, &file->insn_list, list) diff --git a/tools/objtool/sync-check.sh b/tools/objtool/sync-check.sh index 105a291ff8e7..ee49b4e9e72c 100755 --- a/tools/objtool/sync-check.sh +++ b/tools/objtool/sync-check.sh @@ -14,6 +14,7 @@ arch/x86/include/asm/nops.h arch/x86/include/asm/inat_types.h arch/x86/include/asm/orc_types.h arch/x86/include/asm/emulate_prefix.h +arch/x86/include/asm/unwind_hints.h arch/x86/lib/x86-opcode-map.txt arch/x86/tools/gen-insn-attr-x86.awk include/linux/static_call_types.h diff --git a/tools/objtool/unwind_hints.c b/tools/objtool/unwind_hints.c new file mode 100644 index 000000000000..f2521659bae5 --- /dev/null +++ b/tools/objtool/unwind_hints.c @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2015-2017 Josh Poimboeuf + */ +#include + +#include +#include +#include +#include + +int read_unwind_hints(struct objtool_file *file) +{ + struct cfi_state cfi =3D init_cfi; + struct section *sec, *relocsec; + struct unwind_hint *hint; + struct instruction *insn; + struct reloc *reloc; + int i; + + sec =3D find_section_by_name(file->elf, ".discard.unwind_hints"); + if (!sec) + return 0; + + relocsec =3D sec->reloc; + if (!relocsec) { + WARN("missing .rela.discard.unwind_hints section"); + return -1; + } + + if (sec->sh.sh_size % sizeof(struct unwind_hint)) { + WARN("struct unwind_hint size mismatch"); + return -1; + } + + file->hints =3D true; + + for (i =3D 0; i < sec->sh.sh_size / sizeof(struct unwind_hint); i++) { + hint =3D (struct unwind_hint *)sec->data->d_buf + i; + + reloc =3D find_reloc_by_dest(file->elf, sec, i * sizeof(*hint)); + if (!reloc) { + WARN("can't find reloc for unwind_hints[%d]", i); + return -1; + } + + insn =3D find_insn(file, reloc->sym->sec, reloc->addend); + if (!insn) { + WARN("can't find insn for unwind_hints[%d]", i); + return -1; + } + + insn->hint =3D true; + + if (hint->type =3D=3D UNWIND_HINT_TYPE_SAVE) { + insn->hint =3D false; + insn->save =3D true; + continue; + } + + if (hint->type =3D=3D UNWIND_HINT_TYPE_RESTORE) { + insn->restore =3D true; + continue; + } + + if (hint->type =3D=3D UNWIND_HINT_TYPE_REGS_PARTIAL) { + struct symbol *sym =3D find_symbol_by_offset(insn->sec, insn->offset); + + if (sym && sym->bind =3D=3D STB_GLOBAL) { + if (opts.ibt && insn->type !=3D INSN_ENDBR && !insn->noendbr) { + WARN_FUNC("UNWIND_HINT_IRET_REGS without ENDBR", + insn->sec, insn->offset); + } + + insn->entry =3D 1; + } + } + + if (hint->type =3D=3D UNWIND_HINT_TYPE_ENTRY) { + hint->type =3D UNWIND_HINT_TYPE_CALL; + insn->entry =3D 1; + } + + if (hint->type =3D=3D UNWIND_HINT_TYPE_FUNC) { + insn->cfi =3D &func_cfi; + continue; + } + + if (insn->cfi) + cfi =3D *(insn->cfi); + + if (arch_decode_hint_reg(hint->sp_reg, &cfi.cfa.base)) { + WARN_FUNC("unsupported unwind_hint sp base reg %d", + insn->sec, insn->offset, hint->sp_reg); + return -1; + } + + cfi.cfa.offset =3D bswap_if_needed(hint->sp_offset); + cfi.type =3D hint->type; + cfi.end =3D hint->end; + + insn->cfi =3D cfi_hash_find_or_add(&cfi); + } + + return 0; +} --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ED68C61DA4 for ; Thu, 2 Feb 2023 07:42:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231904AbjBBHmk (ORCPT ); Thu, 2 Feb 2023 02:42:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231820AbjBBHmc (ORCPT ); Thu, 2 Feb 2023 02:42:32 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B64FF8418C; Wed, 1 Feb 2023 23:42:30 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id B245320B74FB; Wed, 1 Feb 2023 23:42:29 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B245320B74FB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323750; bh=hqr6+5sz0cEVD673vwUEknaY+FCv/C1YhbsmJe+cR/0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=roO/AeTxRHwHYtYjzzyXJBFrDp08ZJ8xwGcGGk8Y7SS+sN8edGCP3nR/B/3vki4H8 HCKmcS1m5TH2JsjqVZgdXejdT0C6CAkHMSF8E3LX5NhOGI8+zHSpAVc3PXx+l//fcx +aXVFCRNKCUM/RL3FHrcSwtvtM5rdYsbeUn2Lnf4= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 05/22] objtool: Reorganize ORC types Date: Thu, 2 Feb 2023 01:40:19 -0600 Message-Id: <20230202074036.507249-6-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" The ORC code needs to be reorganized into arch-specific and generic parts so that architectures other than X86 can use the generic parts. orc_types.h contains the following ORC definitions shared between objtool and the kernel: - ORC register definitions which are arch-specific. - orc_entry structure which is generic. Move orc_entry into a new file include/linux/orc_entry.h. Also, the field names bp_reg and bp_offset in struct orc_entry are x86-specific. Change them to fp_reg and fp_offset. FP stands for frame pointer. Currently, the type field in orc_entry is only 2 bits. For other architectures, we will need more. So, expand this to 3 bits. Signed-off-by: Madhavan T. Venkataraman --- arch/x86/include/asm/orc_types.h | 37 +++++------------------- include/linux/orc_entry.h | 39 ++++++++++++++++++++++++++ tools/arch/x86/include/asm/orc_types.h | 37 +++++------------------- tools/include/linux/orc_entry.h | 39 ++++++++++++++++++++++++++ tools/objtool/orc_gen.c | 4 +-- tools/objtool/sync-check.sh | 1 + 6 files changed, 95 insertions(+), 62 deletions(-) create mode 100644 include/linux/orc_entry.h create mode 100644 tools/include/linux/orc_entry.h diff --git a/arch/x86/include/asm/orc_types.h b/arch/x86/include/asm/orc_ty= pes.h index 5a2baf28a1dc..851c9fb9f695 100644 --- a/arch/x86/include/asm/orc_types.h +++ b/arch/x86/include/asm/orc_types.h @@ -8,6 +8,13 @@ =20 #include #include +#include + +/* + * For x86, use the appripriate name for the frame pointer in orc_entry. + */ +#define bp_offset fp_offset +#define bp_reg fp_reg =20 /* * The ORC_REG_* registers are base registers which are used to find other @@ -39,34 +46,4 @@ #define ORC_REG_SP_INDIRECT 9 #define ORC_REG_MAX 15 =20 -#ifndef __ASSEMBLY__ -#include - -/* - * This struct is more or less a vastly simplified version of the DWARF Ca= ll - * Frame Information standard. It contains only the necessary parts of DW= ARF - * CFI, simplified for ease of access by the in-kernel unwinder. It tells= the - * unwinder how to find the previous SP and BP (and sometimes entry regs) = on - * the stack for a given code address. Each instance of the struct corres= ponds - * to one or more code locations. - */ -struct orc_entry { - s16 sp_offset; - s16 bp_offset; -#if defined(__LITTLE_ENDIAN_BITFIELD) - unsigned sp_reg:4; - unsigned bp_reg:4; - unsigned type:2; - unsigned end:1; -#elif defined(__BIG_ENDIAN_BITFIELD) - unsigned bp_reg:4; - unsigned sp_reg:4; - unsigned unused:5; - unsigned end:1; - unsigned type:2; -#endif -} __packed; - -#endif /* __ASSEMBLY__ */ - #endif /* _ORC_TYPES_H */ diff --git a/include/linux/orc_entry.h b/include/linux/orc_entry.h new file mode 100644 index 000000000000..3d49e3b9dabe --- /dev/null +++ b/include/linux/orc_entry.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2017 Josh Poimboeuf + */ + +#ifndef _ORC_ENTRY_H +#define _ORC_ENTRY_H + +#ifndef __ASSEMBLY__ +#include + +/* + * This struct is more or less a vastly simplified version of the DWARF Ca= ll + * Frame Information standard. It contains only the necessary parts of DW= ARF + * CFI, simplified for ease of access by the in-kernel unwinder. It tells= the + * unwinder how to find the previous SP and BP (and sometimes entry regs) = on + * the stack for a given code address. Each instance of the struct corres= ponds + * to one or more code locations. + */ +struct orc_entry { + s16 sp_offset; + s16 fp_offset; +#if defined(__LITTLE_ENDIAN_BITFIELD) + unsigned sp_reg:4; + unsigned fp_reg:4; + unsigned type:3; + unsigned end:1; +#elif defined(__BIG_ENDIAN_BITFIELD) + unsigned fp_reg:4; + unsigned sp_reg:4; + unsigned unused:4; + unsigned end:1; + unsigned type:3; +#endif +} __packed; + +#endif /* __ASSEMBLY__ */ + +#endif /* _ORC_ENTRY_H */ diff --git a/tools/arch/x86/include/asm/orc_types.h b/tools/arch/x86/includ= e/asm/orc_types.h index 5a2baf28a1dc..851c9fb9f695 100644 --- a/tools/arch/x86/include/asm/orc_types.h +++ b/tools/arch/x86/include/asm/orc_types.h @@ -8,6 +8,13 @@ =20 #include #include +#include + +/* + * For x86, use the appripriate name for the frame pointer in orc_entry. + */ +#define bp_offset fp_offset +#define bp_reg fp_reg =20 /* * The ORC_REG_* registers are base registers which are used to find other @@ -39,34 +46,4 @@ #define ORC_REG_SP_INDIRECT 9 #define ORC_REG_MAX 15 =20 -#ifndef __ASSEMBLY__ -#include - -/* - * This struct is more or less a vastly simplified version of the DWARF Ca= ll - * Frame Information standard. It contains only the necessary parts of DW= ARF - * CFI, simplified for ease of access by the in-kernel unwinder. It tells= the - * unwinder how to find the previous SP and BP (and sometimes entry regs) = on - * the stack for a given code address. Each instance of the struct corres= ponds - * to one or more code locations. - */ -struct orc_entry { - s16 sp_offset; - s16 bp_offset; -#if defined(__LITTLE_ENDIAN_BITFIELD) - unsigned sp_reg:4; - unsigned bp_reg:4; - unsigned type:2; - unsigned end:1; -#elif defined(__BIG_ENDIAN_BITFIELD) - unsigned bp_reg:4; - unsigned sp_reg:4; - unsigned unused:5; - unsigned end:1; - unsigned type:2; -#endif -} __packed; - -#endif /* __ASSEMBLY__ */ - #endif /* _ORC_TYPES_H */ diff --git a/tools/include/linux/orc_entry.h b/tools/include/linux/orc_entr= y.h new file mode 100644 index 000000000000..3d49e3b9dabe --- /dev/null +++ b/tools/include/linux/orc_entry.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2017 Josh Poimboeuf + */ + +#ifndef _ORC_ENTRY_H +#define _ORC_ENTRY_H + +#ifndef __ASSEMBLY__ +#include + +/* + * This struct is more or less a vastly simplified version of the DWARF Ca= ll + * Frame Information standard. It contains only the necessary parts of DW= ARF + * CFI, simplified for ease of access by the in-kernel unwinder. It tells= the + * unwinder how to find the previous SP and BP (and sometimes entry regs) = on + * the stack for a given code address. Each instance of the struct corres= ponds + * to one or more code locations. + */ +struct orc_entry { + s16 sp_offset; + s16 fp_offset; +#if defined(__LITTLE_ENDIAN_BITFIELD) + unsigned sp_reg:4; + unsigned fp_reg:4; + unsigned type:3; + unsigned end:1; +#elif defined(__BIG_ENDIAN_BITFIELD) + unsigned fp_reg:4; + unsigned sp_reg:4; + unsigned unused:4; + unsigned end:1; + unsigned type:3; +#endif +} __packed; + +#endif /* __ASSEMBLY__ */ + +#endif /* _ORC_ENTRY_H */ diff --git a/tools/objtool/orc_gen.c b/tools/objtool/orc_gen.c index dd3c64af9db2..68c317daadbf 100644 --- a/tools/objtool/orc_gen.c +++ b/tools/objtool/orc_gen.c @@ -98,7 +98,7 @@ static int write_orc_entry(struct elf *elf, struct sectio= n *orc_sec, orc =3D (struct orc_entry *)orc_sec->data->d_buf + idx; memcpy(orc, o, sizeof(*orc)); orc->sp_offset =3D bswap_if_needed(orc->sp_offset); - orc->bp_offset =3D bswap_if_needed(orc->bp_offset); + orc->fp_offset =3D bswap_if_needed(orc->fp_offset); =20 /* populate reloc for ip */ if (elf_add_reloc_to_insn(elf, ip_sec, idx * sizeof(int), R_X86_64_PC32, @@ -149,7 +149,7 @@ int orc_create(struct objtool_file *file) =20 struct orc_entry null =3D { .sp_reg =3D ORC_REG_UNDEFINED, - .bp_reg =3D ORC_REG_UNDEFINED, + .fp_reg =3D ORC_REG_UNDEFINED, .type =3D UNWIND_HINT_TYPE_CALL, }; =20 diff --git a/tools/objtool/sync-check.sh b/tools/objtool/sync-check.sh index ee49b4e9e72c..ef1acb064605 100755 --- a/tools/objtool/sync-check.sh +++ b/tools/objtool/sync-check.sh @@ -18,6 +18,7 @@ arch/x86/include/asm/unwind_hints.h arch/x86/lib/x86-opcode-map.txt arch/x86/tools/gen-insn-attr-x86.awk include/linux/static_call_types.h +include/linux/orc_entry.h " =20 SYNC_CHECK_FILES=3D' --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64E5CC63797 for ; Thu, 2 Feb 2023 07:42:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232039AbjBBHmw (ORCPT ); Thu, 2 Feb 2023 02:42:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231883AbjBBHmd (ORCPT ); Thu, 2 Feb 2023 02:42:33 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C1B25841B2; Wed, 1 Feb 2023 23:42:31 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id B152720B74FC; Wed, 1 Feb 2023 23:42:30 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B152720B74FC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323751; bh=+AlBF++HfEjWxbMQxYX0cu/v4CKaFSwJ25deZh05K3w=; h=From:To:Subject:Date:In-Reply-To:References:From; b=fByeDPulsa8VK8LD0iM3o4DwrsmWvw5a4JRV6lMdFJOUjt7oH293G6lCnqyPLJNCX 03e6pPeuzI4X3M5IitSNPt1IYI3xd5FhPfxaaSIY5rpoy/lpxyjaMTF8bbXrkIL2rO SE1QxDuW+y0RsBCih8HV5UV/HBPX2iMtV5TCngBw= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 06/22] objtool: Reorganize ORC code Date: Thu, 2 Feb 2023 01:40:20 -0600 Message-Id: <20230202074036.507249-7-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" The ORC code needs to be reorganized into arch-specific and generic parts so that architectures other than X86 can avail the generic parts. Some arch-specific ORC code is present in orc_gen.c and orc_dump.c. Create the following two files for such code: - tools/objtool/include/objtool/orc.h - tools/objtool/arch/x86/orc.c Move the following arch-specific function from tools/objtool/orc_gen.c to tools/objtool/arch/x86/orc.c: - init_orc_entry() Move the following arch-specific functions from tools/objtool/orc_dump.c to tools/objtool/arch/x86/orc.c: - reg_name() - orc_type_name() - print_reg() Create arch-specific functions to print the names of the SP and FP registers. The relocation type for relocation entries for ORC structures is arch-specific. Define it in tools/objtool/arch/x86/include/arch/elf.h: #define R_PCREL R_X86_64_PC32 and use that in orc_gen.c so each architecture can provide its own relocation type. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/arch/x86/Build | 1 + tools/objtool/arch/x86/include/arch/elf.h | 1 + tools/objtool/arch/x86/orc.c | 150 ++++++++++++++++++++++ tools/objtool/include/objtool/orc.h | 18 +++ tools/objtool/orc_dump.c | 63 +-------- tools/objtool/orc_gen.c | 79 +----------- 6 files changed, 179 insertions(+), 133 deletions(-) create mode 100644 tools/objtool/arch/x86/orc.c create mode 100644 tools/objtool/include/objtool/orc.h diff --git a/tools/objtool/arch/x86/Build b/tools/objtool/arch/x86/Build index 9f7869b5c5e0..77b9a66cd6da 100644 --- a/tools/objtool/arch/x86/Build +++ b/tools/objtool/arch/x86/Build @@ -1,5 +1,6 @@ objtool-y +=3D special.o objtool-y +=3D decode.o +objtool-$(BUILD_ORC) +=3D orc.o =20 inat_tables_script =3D ../arch/x86/tools/gen-insn-attr-x86.awk inat_tables_maps =3D ../arch/x86/lib/x86-opcode-map.txt diff --git a/tools/objtool/arch/x86/include/arch/elf.h b/tools/objtool/arch= /x86/include/arch/elf.h index 69cc4264b28a..3a7eb515dbb9 100644 --- a/tools/objtool/arch/x86/include/arch/elf.h +++ b/tools/objtool/arch/x86/include/arch/elf.h @@ -2,5 +2,6 @@ #define _OBJTOOL_ARCH_ELF =20 #define R_NONE R_X86_64_NONE +#define R_PCREL R_X86_64_PC32 =20 #endif /* _OBJTOOL_ARCH_ELF */ diff --git a/tools/objtool/arch/x86/orc.c b/tools/objtool/arch/x86/orc.c new file mode 100644 index 000000000000..a075737d4503 --- /dev/null +++ b/tools/objtool/arch/x86/orc.c @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2017 Josh Poimboeuf + */ + +#include +#include + +#include + +#include +#include +#include +#include + +int init_orc_entry(struct orc_entry *orc, struct cfi_state *cfi, + struct instruction *insn) +{ + struct cfi_reg *bp =3D &cfi->regs[CFI_BP]; + + memset(orc, 0, sizeof(*orc)); + + if (!cfi) { + orc->end =3D 0; + orc->sp_reg =3D ORC_REG_UNDEFINED; + return 0; + } + + orc->end =3D cfi->end; + + if (cfi->cfa.base =3D=3D CFI_UNDEFINED) { + orc->sp_reg =3D ORC_REG_UNDEFINED; + return 0; + } + + switch (cfi->cfa.base) { + case CFI_SP: + orc->sp_reg =3D ORC_REG_SP; + break; + case CFI_SP_INDIRECT: + orc->sp_reg =3D ORC_REG_SP_INDIRECT; + break; + case CFI_BP: + orc->sp_reg =3D ORC_REG_BP; + break; + case CFI_BP_INDIRECT: + orc->sp_reg =3D ORC_REG_BP_INDIRECT; + break; + case CFI_R10: + orc->sp_reg =3D ORC_REG_R10; + break; + case CFI_R13: + orc->sp_reg =3D ORC_REG_R13; + break; + case CFI_DI: + orc->sp_reg =3D ORC_REG_DI; + break; + case CFI_DX: + orc->sp_reg =3D ORC_REG_DX; + break; + default: + WARN_FUNC("unknown CFA base reg %d", + insn->sec, insn->offset, cfi->cfa.base); + return -1; + } + + switch (bp->base) { + case CFI_UNDEFINED: + orc->bp_reg =3D ORC_REG_UNDEFINED; + break; + case CFI_CFA: + orc->bp_reg =3D ORC_REG_PREV_SP; + break; + case CFI_BP: + orc->bp_reg =3D ORC_REG_BP; + break; + default: + WARN_FUNC("unknown BP base reg %d", + insn->sec, insn->offset, bp->base); + return -1; + } + + orc->sp_offset =3D cfi->cfa.offset; + orc->bp_offset =3D bp->offset; + orc->type =3D cfi->type; + + return 0; +} + +static const char *reg_name(unsigned int reg) +{ + switch (reg) { + case ORC_REG_PREV_SP: + return "prevsp"; + case ORC_REG_DX: + return "dx"; + case ORC_REG_DI: + return "di"; + case ORC_REG_BP: + return "bp"; + case ORC_REG_SP: + return "sp"; + case ORC_REG_R10: + return "r10"; + case ORC_REG_R13: + return "r13"; + case ORC_REG_BP_INDIRECT: + return "bp(ind)"; + case ORC_REG_SP_INDIRECT: + return "sp(ind)"; + default: + return "?"; + } +} + +const char *orc_type_name(unsigned int type) +{ + switch (type) { + case UNWIND_HINT_TYPE_CALL: + return "call"; + case UNWIND_HINT_TYPE_REGS: + return "regs"; + case UNWIND_HINT_TYPE_REGS_PARTIAL: + return "regs (partial)"; + default: + return "?"; + } +} + +void orc_print_reg(unsigned int reg, int offset) +{ + if (reg =3D=3D ORC_REG_BP_INDIRECT) + printf("(bp%+d)", offset); + else if (reg =3D=3D ORC_REG_SP_INDIRECT) + printf("(sp)%+d", offset); + else if (reg =3D=3D ORC_REG_UNDEFINED) + printf("(und)"); + else + printf("%s%+d", reg_name(reg), offset); +} + +void orc_print_sp(void) +{ + printf(" sp:"); +} + +void orc_print_fp(void) +{ + printf(" bp:"); +} diff --git a/tools/objtool/include/objtool/orc.h b/tools/objtool/include/ob= jtool/orc.h new file mode 100644 index 000000000000..bf141134c56f --- /dev/null +++ b/tools/objtool/include/objtool/orc.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2015-2017 Josh Poimboeuf + */ + +#ifndef _OBJTOOL_ORC_H +#define _OBJTOOL_ORC_H + +#include + +int init_orc_entry(struct orc_entry *orc, struct cfi_state *cfi, + struct instruction *insn); +const char *orc_type_name(unsigned int type); +void orc_print_reg(unsigned int reg, int offset); +void orc_print_sp(void); +void orc_print_fp(void); + +#endif /* _OBJTOOL_ORC_H */ diff --git a/tools/objtool/orc_dump.c b/tools/objtool/orc_dump.c index f5a8508c42d6..61b39960ab6a 100644 --- a/tools/objtool/orc_dump.c +++ b/tools/objtool/orc_dump.c @@ -5,63 +5,12 @@ =20 #include #include -#include #include +#include +#include #include #include =20 -static const char *reg_name(unsigned int reg) -{ - switch (reg) { - case ORC_REG_PREV_SP: - return "prevsp"; - case ORC_REG_DX: - return "dx"; - case ORC_REG_DI: - return "di"; - case ORC_REG_BP: - return "bp"; - case ORC_REG_SP: - return "sp"; - case ORC_REG_R10: - return "r10"; - case ORC_REG_R13: - return "r13"; - case ORC_REG_BP_INDIRECT: - return "bp(ind)"; - case ORC_REG_SP_INDIRECT: - return "sp(ind)"; - default: - return "?"; - } -} - -static const char *orc_type_name(unsigned int type) -{ - switch (type) { - case UNWIND_HINT_TYPE_CALL: - return "call"; - case UNWIND_HINT_TYPE_REGS: - return "regs"; - case UNWIND_HINT_TYPE_REGS_PARTIAL: - return "regs (partial)"; - default: - return "?"; - } -} - -static void print_reg(unsigned int reg, int offset) -{ - if (reg =3D=3D ORC_REG_BP_INDIRECT) - printf("(bp%+d)", offset); - else if (reg =3D=3D ORC_REG_SP_INDIRECT) - printf("(sp)%+d", offset); - else if (reg =3D=3D ORC_REG_UNDEFINED) - printf("(und)"); - else - printf("%s%+d", reg_name(reg), offset); -} - int orc_dump(const char *_objname) { int fd, nr_entries, i, *orc_ip =3D NULL, orc_size =3D 0; @@ -196,13 +145,13 @@ int orc_dump(const char *_objname) } =20 =20 - printf(" sp:"); + orc_print_sp(); =20 - print_reg(orc[i].sp_reg, bswap_if_needed(orc[i].sp_offset)); + orc_print_reg(orc[i].sp_reg, bswap_if_needed(orc[i].sp_offset)); =20 - printf(" bp:"); + orc_print_fp(); =20 - print_reg(orc[i].bp_reg, bswap_if_needed(orc[i].bp_offset)); + orc_print_reg(orc[i].fp_reg, bswap_if_needed(orc[i].fp_offset)); =20 printf(" type:%s end:%d\n", orc_type_name(orc[i].type), orc[i].end); diff --git a/tools/objtool/orc_gen.c b/tools/objtool/orc_gen.c index 68c317daadbf..ea2e361ff7bc 100644 --- a/tools/objtool/orc_gen.c +++ b/tools/objtool/orc_gen.c @@ -7,86 +7,13 @@ #include =20 #include -#include +#include =20 #include +#include #include #include =20 -static int init_orc_entry(struct orc_entry *orc, struct cfi_state *cfi, - struct instruction *insn) -{ - struct cfi_reg *bp =3D &cfi->regs[CFI_BP]; - - memset(orc, 0, sizeof(*orc)); - - if (!cfi) { - orc->end =3D 0; - orc->sp_reg =3D ORC_REG_UNDEFINED; - return 0; - } - - orc->end =3D cfi->end; - - if (cfi->cfa.base =3D=3D CFI_UNDEFINED) { - orc->sp_reg =3D ORC_REG_UNDEFINED; - return 0; - } - - switch (cfi->cfa.base) { - case CFI_SP: - orc->sp_reg =3D ORC_REG_SP; - break; - case CFI_SP_INDIRECT: - orc->sp_reg =3D ORC_REG_SP_INDIRECT; - break; - case CFI_BP: - orc->sp_reg =3D ORC_REG_BP; - break; - case CFI_BP_INDIRECT: - orc->sp_reg =3D ORC_REG_BP_INDIRECT; - break; - case CFI_R10: - orc->sp_reg =3D ORC_REG_R10; - break; - case CFI_R13: - orc->sp_reg =3D ORC_REG_R13; - break; - case CFI_DI: - orc->sp_reg =3D ORC_REG_DI; - break; - case CFI_DX: - orc->sp_reg =3D ORC_REG_DX; - break; - default: - WARN_FUNC("unknown CFA base reg %d", - insn->sec, insn->offset, cfi->cfa.base); - return -1; - } - - switch (bp->base) { - case CFI_UNDEFINED: - orc->bp_reg =3D ORC_REG_UNDEFINED; - break; - case CFI_CFA: - orc->bp_reg =3D ORC_REG_PREV_SP; - break; - case CFI_BP: - orc->bp_reg =3D ORC_REG_BP; - break; - default: - WARN_FUNC("unknown BP base reg %d", - insn->sec, insn->offset, bp->base); - return -1; - } - - orc->sp_offset =3D cfi->cfa.offset; - orc->bp_offset =3D bp->offset; - orc->type =3D cfi->type; - - return 0; -} - static int write_orc_entry(struct elf *elf, struct section *orc_sec, struct section *ip_sec, unsigned int idx, struct section *insn_sec, unsigned long insn_off, @@ -101,7 +28,7 @@ static int write_orc_entry(struct elf *elf, struct secti= on *orc_sec, orc->fp_offset =3D bswap_if_needed(orc->fp_offset); =20 /* populate reloc for ip */ - if (elf_add_reloc_to_insn(elf, ip_sec, idx * sizeof(int), R_X86_64_PC32, + if (elf_add_reloc_to_insn(elf, ip_sec, idx * sizeof(int), R_PCREL, insn_sec, insn_off)) return -1; =20 --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEDB2C05027 for ; Thu, 2 Feb 2023 07:42:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232059AbjBBHm5 (ORCPT ); Thu, 2 Feb 2023 02:42:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231923AbjBBHmm (ORCPT ); Thu, 2 Feb 2023 02:42:42 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9E2DE751BE; Wed, 1 Feb 2023 23:42:32 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id AF8A620B74FD; Wed, 1 Feb 2023 23:42:31 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com AF8A620B74FD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323752; bh=ykIF4r8PM6tvazWbNX1FgcvGvWivCHgNFJw/+58oeDo=; h=From:To:Subject:Date:In-Reply-To:References:From; b=QkDHAtXcFkVb0za/Uygu5L/4KByO+B9N5/zW3FYWkjB1PPK4SPSeo6ayvtafdquIK N5BSP4pVKXAY2uSvqtaC/tOwN9n9l+TbC4v5OLwiansWu9yqxc9K8dElD8RkTbDl8W zzLf2RzKf4VdR7C2PpOPtVOIPlUWVrYUIyT+l7Rc= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 07/22] objtool: Reorganize ORC kernel code Date: Thu, 2 Feb 2023 01:40:21 -0600 Message-Id: <20230202074036.507249-8-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" All of the ORC code in the kernel is currently under arch/x86. The following parts of that code can be shared by other architectures that wish to use ORC. (1) ORC lookup initialization for vmlinux (2) ORC lookup initialization for modules (3) ORC lookup functions Move arch/x86/include/asm/orc_lookup.h to include/asm-generic/orc_lookup.h. Move the ORC lookup code into kernel/orc_lookup.c. Rename the following init functions: unwind_module_init =3D=3D> orc_lookup_module_init unwind_init =3D=3D> orc_lookup_init since that is exactly what they do. orc_find() is the function that locates the ORC entry for a given PC. Currently, it contains an architecture-specific part to locate ftrace entries. Introduce a new arch-specific function called arch_orc_find() and move the ftrace-related lookup there. If orc_find() is unable to locate the ORC entry for a given PC in vmlinux or in the modules, it can call arch_orc_find() to find architecture-specific entries. Signed-off-by: Madhavan T. Venkataraman --- arch/x86/include/asm/unwind.h | 5 - arch/x86/kernel/module.c | 7 +- arch/x86/kernel/unwind_orc.c | 256 +---------------- arch/x86/kernel/vmlinux.lds.S | 2 +- .../asm =3D> include/asm-generic}/orc_lookup.h | 42 +++ kernel/Makefile | 2 + kernel/orc_lookup.c | 261 ++++++++++++++++++ 7 files changed, 316 insertions(+), 259 deletions(-) rename {arch/x86/include/asm =3D> include/asm-generic}/orc_lookup.h (51%) create mode 100644 kernel/orc_lookup.c diff --git a/arch/x86/include/asm/unwind.h b/arch/x86/include/asm/unwind.h index 7cede4dc21f0..71af8246c69e 100644 --- a/arch/x86/include/asm/unwind.h +++ b/arch/x86/include/asm/unwind.h @@ -94,13 +94,8 @@ static inline struct pt_regs *unwind_get_entry_regs(stru= ct unwind_state *state, =20 #ifdef CONFIG_UNWINDER_ORC void unwind_init(void); -void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_si= ze, - void *orc, size_t orc_size); #else static inline void unwind_init(void) {} -static inline -void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_si= ze, - void *orc, size_t orc_size) {} #endif =20 static inline diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index c032edcd3d95..24664930c917 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -23,7 +23,7 @@ #include #include #include -#include +#include =20 #if 0 #define DEBUGP(fmt, ...) \ @@ -311,8 +311,9 @@ int module_finalize(const Elf_Ehdr *hdr, } =20 if (orc && orc_ip) - unwind_module_init(me, (void *)orc_ip->sh_addr, orc_ip->sh_size, - (void *)orc->sh_addr, orc->sh_size); + orc_lookup_module_init(me, + (void *)orc_ip->sh_addr, orc_ip->sh_size, + (void *)orc->sh_addr, orc->sh_size); =20 return 0; } diff --git a/arch/x86/kernel/unwind_orc.c b/arch/x86/kernel/unwind_orc.c index c2bfc597d909..eac9ed762bf8 100644 --- a/arch/x86/kernel/unwind_orc.c +++ b/arch/x86/kernel/unwind_orc.c @@ -6,80 +6,9 @@ #include #include #include -#include - -#define orc_warn(fmt, ...) \ - printk_deferred_once(KERN_WARNING "WARNING: " fmt, ##__VA_ARGS__) - -#define orc_warn_current(args...) \ -({ \ - if (state->task =3D=3D current && !state->error) \ - orc_warn(args); \ -}) - -extern int __start_orc_unwind_ip[]; -extern int __stop_orc_unwind_ip[]; -extern struct orc_entry __start_orc_unwind[]; -extern struct orc_entry __stop_orc_unwind[]; - -static bool orc_init __ro_after_init; -static unsigned int lookup_num_blocks __ro_after_init; - -static inline unsigned long orc_ip(const int *ip) -{ - return (unsigned long)ip + *ip; -} - -static struct orc_entry *__orc_find(int *ip_table, struct orc_entry *u_tab= le, - unsigned int num_entries, unsigned long ip) -{ - int *first =3D ip_table; - int *last =3D ip_table + num_entries - 1; - int *mid =3D first, *found =3D first; - - if (!num_entries) - return NULL; - - /* - * Do a binary range search to find the rightmost duplicate of a given - * starting address. Some entries are section terminators which are - * "weak" entries for ensuring there are no gaps. They should be - * ignored when they conflict with a real entry. - */ - while (first <=3D last) { - mid =3D first + ((last - first) / 2); - - if (orc_ip(mid) <=3D ip) { - found =3D mid; - first =3D mid + 1; - } else - last =3D mid - 1; - } - - return u_table + (found - ip_table); -} - -#ifdef CONFIG_MODULES -static struct orc_entry *orc_module_find(unsigned long ip) -{ - struct module *mod; - - mod =3D __module_address(ip); - if (!mod || !mod->arch.orc_unwind || !mod->arch.orc_unwind_ip) - return NULL; - return __orc_find(mod->arch.orc_unwind_ip, mod->arch.orc_unwind, - mod->arch.num_orcs, ip); -} -#else -static struct orc_entry *orc_module_find(unsigned long ip) -{ - return NULL; -} -#endif +#include =20 #ifdef CONFIG_DYNAMIC_FTRACE -static struct orc_entry *orc_find(unsigned long ip); - /* * Ftrace dynamic trampolines do not have orc entries of their own. * But they are copies of the ftrace entries that are static and @@ -122,19 +51,10 @@ static struct orc_entry *orc_ftrace_find(unsigned long= ip) } #endif =20 -/* - * If we crash with IP=3D=3D0, the last successfully executed instruction - * was probably an indirect function call with a NULL function pointer, - * and we don't have unwind information for NULL. - * This hardcoded ORC entry for IP=3D=3D0 allows us to unwind from a NULL = function - * pointer into its parent and then continue normally from there. - */ -static struct orc_entry null_orc_entry =3D { - .sp_offset =3D sizeof(long), - .sp_reg =3D ORC_REG_SP, - .bp_reg =3D ORC_REG_UNDEFINED, - .type =3D UNWIND_HINT_TYPE_CALL -}; +struct orc_entry *arch_orc_find(unsigned long ip) +{ + return orc_ftrace_find(ip); +} =20 /* Fake frame pointer entry -- used as a fallback for generated code */ static struct orc_entry orc_fp_entry =3D { @@ -146,173 +66,9 @@ static struct orc_entry orc_fp_entry =3D { .end =3D 0, }; =20 -static struct orc_entry *orc_find(unsigned long ip) -{ - static struct orc_entry *orc; - - if (ip =3D=3D 0) - return &null_orc_entry; - - /* For non-init vmlinux addresses, use the fast lookup table: */ - if (ip >=3D LOOKUP_START_IP && ip < LOOKUP_STOP_IP) { - unsigned int idx, start, stop; - - idx =3D (ip - LOOKUP_START_IP) / LOOKUP_BLOCK_SIZE; - - if (unlikely((idx >=3D lookup_num_blocks-1))) { - orc_warn("WARNING: bad lookup idx: idx=3D%u num=3D%u ip=3D%pB\n", - idx, lookup_num_blocks, (void *)ip); - return NULL; - } - - start =3D orc_lookup[idx]; - stop =3D orc_lookup[idx + 1] + 1; - - if (unlikely((__start_orc_unwind + start >=3D __stop_orc_unwind) || - (__start_orc_unwind + stop > __stop_orc_unwind))) { - orc_warn("WARNING: bad lookup value: idx=3D%u num=3D%u start=3D%u stop= =3D%u ip=3D%pB\n", - idx, lookup_num_blocks, start, stop, (void *)ip); - return NULL; - } - - return __orc_find(__start_orc_unwind_ip + start, - __start_orc_unwind + start, stop - start, ip); - } - - /* vmlinux .init slow lookup: */ - if (is_kernel_inittext(ip)) - return __orc_find(__start_orc_unwind_ip, __start_orc_unwind, - __stop_orc_unwind_ip - __start_orc_unwind_ip, ip); - - /* Module lookup: */ - orc =3D orc_module_find(ip); - if (orc) - return orc; - - return orc_ftrace_find(ip); -} - -#ifdef CONFIG_MODULES - -static DEFINE_MUTEX(sort_mutex); -static int *cur_orc_ip_table =3D __start_orc_unwind_ip; -static struct orc_entry *cur_orc_table =3D __start_orc_unwind; - -static void orc_sort_swap(void *_a, void *_b, int size) -{ - struct orc_entry *orc_a, *orc_b; - struct orc_entry orc_tmp; - int *a =3D _a, *b =3D _b, tmp; - int delta =3D _b - _a; - - /* Swap the .orc_unwind_ip entries: */ - tmp =3D *a; - *a =3D *b + delta; - *b =3D tmp - delta; - - /* Swap the corresponding .orc_unwind entries: */ - orc_a =3D cur_orc_table + (a - cur_orc_ip_table); - orc_b =3D cur_orc_table + (b - cur_orc_ip_table); - orc_tmp =3D *orc_a; - *orc_a =3D *orc_b; - *orc_b =3D orc_tmp; -} - -static int orc_sort_cmp(const void *_a, const void *_b) -{ - struct orc_entry *orc_a; - const int *a =3D _a, *b =3D _b; - unsigned long a_val =3D orc_ip(a); - unsigned long b_val =3D orc_ip(b); - - if (a_val > b_val) - return 1; - if (a_val < b_val) - return -1; - - /* - * The "weak" section terminator entries need to always be on the left - * to ensure the lookup code skips them in favor of real entries. - * These terminator entries exist to handle any gaps created by - * whitelisted .o files which didn't get objtool generation. - */ - orc_a =3D cur_orc_table + (a - cur_orc_ip_table); - return orc_a->sp_reg =3D=3D ORC_REG_UNDEFINED && !orc_a->end ? -1 : 1; -} - -void unwind_module_init(struct module *mod, void *_orc_ip, size_t orc_ip_s= ize, - void *_orc, size_t orc_size) -{ - int *orc_ip =3D _orc_ip; - struct orc_entry *orc =3D _orc; - unsigned int num_entries =3D orc_ip_size / sizeof(int); - - WARN_ON_ONCE(orc_ip_size % sizeof(int) !=3D 0 || - orc_size % sizeof(*orc) !=3D 0 || - num_entries !=3D orc_size / sizeof(*orc)); - - /* - * The 'cur_orc_*' globals allow the orc_sort_swap() callback to - * associate an .orc_unwind_ip table entry with its corresponding - * .orc_unwind entry so they can both be swapped. - */ - mutex_lock(&sort_mutex); - cur_orc_ip_table =3D orc_ip; - cur_orc_table =3D orc; - sort(orc_ip, num_entries, sizeof(int), orc_sort_cmp, orc_sort_swap); - mutex_unlock(&sort_mutex); - - mod->arch.orc_unwind_ip =3D orc_ip; - mod->arch.orc_unwind =3D orc; - mod->arch.num_orcs =3D num_entries; -} -#endif - void __init unwind_init(void) { - size_t orc_ip_size =3D (void *)__stop_orc_unwind_ip - (void *)__start_orc= _unwind_ip; - size_t orc_size =3D (void *)__stop_orc_unwind - (void *)__start_orc_unwin= d; - size_t num_entries =3D orc_ip_size / sizeof(int); - struct orc_entry *orc; - int i; - - if (!num_entries || orc_ip_size % sizeof(int) !=3D 0 || - orc_size % sizeof(struct orc_entry) !=3D 0 || - num_entries !=3D orc_size / sizeof(struct orc_entry)) { - orc_warn("WARNING: Bad or missing .orc_unwind table. Disabling unwinder= .\n"); - return; - } - - /* - * Note, the orc_unwind and orc_unwind_ip tables were already - * sorted at build time via the 'sorttable' tool. - * It's ready for binary search straight away, no need to sort it. - */ - - /* Initialize the fast lookup table: */ - lookup_num_blocks =3D orc_lookup_end - orc_lookup; - for (i =3D 0; i < lookup_num_blocks-1; i++) { - orc =3D __orc_find(__start_orc_unwind_ip, __start_orc_unwind, - num_entries, - LOOKUP_START_IP + (LOOKUP_BLOCK_SIZE * i)); - if (!orc) { - orc_warn("WARNING: Corrupt .orc_unwind table. Disabling unwinder.\n"); - return; - } - - orc_lookup[i] =3D orc - __start_orc_unwind; - } - - /* Initialize the ending block: */ - orc =3D __orc_find(__start_orc_unwind_ip, __start_orc_unwind, num_entries, - LOOKUP_STOP_IP); - if (!orc) { - orc_warn("WARNING: Corrupt .orc_unwind table. Disabling unwinder.\n"); - return; - } - orc_lookup[lookup_num_blocks-1] =3D orc - __start_orc_unwind; - - orc_init =3D true; + orc_lookup_init(); } =20 unsigned long unwind_get_return_address(struct unwind_state *state) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 15f29053cec4..b4b93cd68136 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include =20 diff --git a/arch/x86/include/asm/orc_lookup.h b/include/asm-generic/orc_lo= okup.h similarity index 51% rename from arch/x86/include/asm/orc_lookup.h rename to include/asm-generic/orc_lookup.h index 241631282e43..f299fbf41cd0 100644 --- a/arch/x86/include/asm/orc_lookup.h +++ b/include/asm-generic/orc_lookup.h @@ -23,6 +23,8 @@ =20 #ifndef LINKER_SCRIPT =20 +#include + extern unsigned int orc_lookup[]; extern unsigned int orc_lookup_end[]; =20 @@ -31,4 +33,44 @@ extern unsigned int orc_lookup_end[]; =20 #endif /* LINKER_SCRIPT */ =20 +#ifndef __ASSEMBLY__ + +#include + +#ifdef CONFIG_UNWINDER_ORC +void orc_lookup_init(void); +void orc_lookup_module_init(struct module *mod, + void *orc_ip, size_t orc_ip_size, + void *orc, size_t orc_size); +#else +static inline void orc_lookup_init(void) {} +static inline +void orc_lookup_module_init(struct module *mod, + void *orc_ip, size_t orc_ip_size, + void *orc, size_t orc_size) +{ +} +#endif + +struct orc_entry *arch_orc_find(unsigned long ip); + +#define orc_warn(fmt, ...) \ + printk_deferred_once(KERN_WARNING "WARNING: " fmt, ##__VA_ARGS__) + +#define orc_warn_current(args...) \ +({ \ + if (state->task =3D=3D current && !state->error) \ + orc_warn(args); \ +}) + +struct orc_entry *orc_find(unsigned long ip); + +extern bool orc_init; +extern int __start_orc_unwind_ip[]; +extern int __stop_orc_unwind_ip[]; +extern struct orc_entry __start_orc_unwind[]; +extern struct orc_entry __stop_orc_unwind[]; + +#endif /* __ASSEMBLY__ */ + #endif /* _ORC_LOOKUP_H */ diff --git a/kernel/Makefile b/kernel/Makefile index d754e0be1176..9a78612c4568 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -132,6 +132,8 @@ obj-$(CONFIG_WATCH_QUEUE) +=3D watch_queue.o obj-$(CONFIG_RESOURCE_KUNIT_TEST) +=3D resource_kunit.o obj-$(CONFIG_SYSCTL_KUNIT_TEST) +=3D sysctl-test.o =20 +obj-$(CONFIG_UNWINDER_ORC) +=3D orc_lookup.o + CFLAGS_stackleak.o +=3D $(DISABLE_STACKLEAK_PLUGIN) obj-$(CONFIG_GCC_PLUGIN_STACKLEAK) +=3D stackleak.o KASAN_SANITIZE_stackleak.o :=3D n diff --git a/kernel/orc_lookup.c b/kernel/orc_lookup.c new file mode 100644 index 000000000000..88b783c41e94 --- /dev/null +++ b/kernel/orc_lookup.c @@ -0,0 +1,261 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include +#include +#include + +bool orc_init __ro_after_init; +static unsigned int lookup_num_blocks __ro_after_init; + +static inline unsigned long orc_ip(const int *ip) +{ + return (unsigned long)ip + *ip; +} + +static struct orc_entry *__orc_find(int *ip_table, struct orc_entry *u_tab= le, + unsigned int num_entries, unsigned long ip) +{ + int *first =3D ip_table; + int *last =3D ip_table + num_entries - 1; + int *mid =3D first, *found =3D first; + + if (!num_entries) + return NULL; + + /* + * Do a binary range search to find the rightmost duplicate of a given + * starting address. Some entries are section terminators which are + * "weak" entries for ensuring there are no gaps. They should be + * ignored when they conflict with a real entry. + */ + while (first <=3D last) { + mid =3D first + ((last - first) / 2); + + if (orc_ip(mid) <=3D ip) { + found =3D mid; + first =3D mid + 1; + } else + last =3D mid - 1; + } + + return u_table + (found - ip_table); +} + +#ifdef CONFIG_MODULES +static struct orc_entry *orc_module_find(unsigned long ip) +{ + struct module *mod; + + mod =3D __module_address(ip); + if (!mod || !mod->arch.orc_unwind || !mod->arch.orc_unwind_ip) + return NULL; + return __orc_find(mod->arch.orc_unwind_ip, mod->arch.orc_unwind, + mod->arch.num_orcs, ip); +} +#else +static struct orc_entry *orc_module_find(unsigned long ip) +{ + return NULL; +} +#endif + +/* + * If we crash with IP=3D=3D0, the last successfully executed instruction + * was probably an indirect function call with a NULL function pointer, + * and we don't have unwind information for NULL. + * This hardcoded ORC entry for IP=3D=3D0 allows us to unwind from a NULL = function + * pointer into its parent and then continue normally from there. + */ +static struct orc_entry null_orc_entry =3D { + .sp_offset =3D sizeof(long), + .sp_reg =3D ORC_REG_SP, + .fp_reg =3D ORC_REG_UNDEFINED, + .type =3D UNWIND_HINT_TYPE_CALL +}; + +struct orc_entry *orc_find(unsigned long ip) +{ + static struct orc_entry *orc; + + if (ip =3D=3D 0) + return &null_orc_entry; + + /* For non-init vmlinux addresses, use the fast lookup table: */ + if (ip >=3D LOOKUP_START_IP && ip < LOOKUP_STOP_IP) { + unsigned int idx, start, stop; + + if (!orc_init) { + /* + * Take the slow path if the fast lookup tables have + * not yet been initialized. + */ + return __orc_find(__start_orc_unwind_ip, + __start_orc_unwind, + __stop_orc_unwind_ip - + __start_orc_unwind_ip, ip); + } + + idx =3D (ip - LOOKUP_START_IP) / LOOKUP_BLOCK_SIZE; + + if (unlikely((idx >=3D lookup_num_blocks-1))) { + orc_warn("WARNING: bad lookup idx: idx=3D%u num=3D%u ip=3D%pB\n", + idx, lookup_num_blocks, (void *)ip); + return NULL; + } + + start =3D orc_lookup[idx]; + stop =3D orc_lookup[idx + 1] + 1; + + if (unlikely((__start_orc_unwind + start >=3D __stop_orc_unwind) || + (__start_orc_unwind + stop > __stop_orc_unwind))) { + orc_warn("WARNING: bad lookup value: idx=3D%u num=3D%u start=3D%u stop= =3D%u ip=3D%pB\n", + idx, lookup_num_blocks, start, stop, (void *)ip); + return NULL; + } + + return __orc_find(__start_orc_unwind_ip + start, + __start_orc_unwind + start, stop - start, ip); + } + + /* vmlinux .init slow lookup: */ + if (is_kernel_inittext(ip)) + return __orc_find(__start_orc_unwind_ip, __start_orc_unwind, + __stop_orc_unwind_ip - __start_orc_unwind_ip, ip); + + /* Module lookup: */ + orc =3D orc_module_find(ip); + if (orc) + return orc; + + return arch_orc_find(ip); +} + +#ifdef CONFIG_MODULES + +static DEFINE_MUTEX(sort_mutex); +static int *cur_orc_ip_table =3D __start_orc_unwind_ip; +static struct orc_entry *cur_orc_table =3D __start_orc_unwind; + +static void orc_sort_swap(void *_a, void *_b, int size) +{ + struct orc_entry *orc_a, *orc_b; + struct orc_entry orc_tmp; + int *a =3D _a, *b =3D _b, tmp; + int delta =3D _b - _a; + + /* Swap the .orc_unwind_ip entries: */ + tmp =3D *a; + *a =3D *b + delta; + *b =3D tmp - delta; + + /* Swap the corresponding .orc_unwind entries: */ + orc_a =3D cur_orc_table + (a - cur_orc_ip_table); + orc_b =3D cur_orc_table + (b - cur_orc_ip_table); + orc_tmp =3D *orc_a; + *orc_a =3D *orc_b; + *orc_b =3D orc_tmp; +} + +static int orc_sort_cmp(const void *_a, const void *_b) +{ + struct orc_entry *orc_a; + const int *a =3D _a, *b =3D _b; + unsigned long a_val =3D orc_ip(a); + unsigned long b_val =3D orc_ip(b); + + if (a_val > b_val) + return 1; + if (a_val < b_val) + return -1; + + /* + * The "weak" section terminator entries need to always be on the left + * to ensure the lookup code skips them in favor of real entries. + * These terminator entries exist to handle any gaps created by + * whitelisted .o files which didn't get objtool generation. + */ + orc_a =3D cur_orc_table + (a - cur_orc_ip_table); + return orc_a->sp_reg =3D=3D ORC_REG_UNDEFINED && !orc_a->end ? -1 : 1; +} + +void orc_lookup_module_init(struct module *mod, + void *_orc_ip, size_t orc_ip_size, + void *_orc, size_t orc_size) +{ + int *orc_ip =3D _orc_ip; + struct orc_entry *orc =3D _orc; + unsigned int num_entries =3D orc_ip_size / sizeof(int); + + WARN_ON_ONCE(orc_ip_size % sizeof(int) !=3D 0 || + orc_size % sizeof(*orc) !=3D 0 || + num_entries !=3D orc_size / sizeof(*orc)); + + /* + * The 'cur_orc_*' globals allow the orc_sort_swap() callback to + * associate an .orc_unwind_ip table entry with its corresponding + * .orc_unwind entry so they can both be swapped. + */ + mutex_lock(&sort_mutex); + cur_orc_ip_table =3D orc_ip; + cur_orc_table =3D orc; + sort(orc_ip, num_entries, sizeof(int), orc_sort_cmp, orc_sort_swap); + mutex_unlock(&sort_mutex); + + mod->arch.orc_unwind_ip =3D orc_ip; + mod->arch.orc_unwind =3D orc; + mod->arch.num_orcs =3D num_entries; +} +#endif + +void __init orc_lookup_init(void) +{ + size_t orc_ip_size =3D (void *)__stop_orc_unwind_ip - (void *)__start_orc= _unwind_ip; + size_t orc_size =3D (void *)__stop_orc_unwind - (void *)__start_orc_unwin= d; + size_t num_entries =3D orc_ip_size / sizeof(int); + struct orc_entry *orc; + int i; + + if (!num_entries || orc_ip_size % sizeof(int) !=3D 0 || + orc_size % sizeof(struct orc_entry) !=3D 0 || + num_entries !=3D orc_size / sizeof(struct orc_entry)) { + orc_warn("WARNING: Bad or missing .orc_unwind table. Disabling unwinder= .\n"); + return; + } + + /* + * Note, the orc_unwind and orc_unwind_ip tables were already + * sorted at build time via the 'sorttable' tool. + * It's ready for binary search straight away, no need to sort it. + */ + + /* Initialize the fast lookup table: */ + lookup_num_blocks =3D orc_lookup_end - orc_lookup; + for (i =3D 0; i < lookup_num_blocks-1; i++) { + orc =3D __orc_find(__start_orc_unwind_ip, __start_orc_unwind, + num_entries, + LOOKUP_START_IP + (LOOKUP_BLOCK_SIZE * i)); + if (!orc) { + orc_warn("WARNING: Corrupt .orc_unwind table. Disabling unwinder.\n"); + return; + } + + orc_lookup[i] =3D orc - __start_orc_unwind; + } + + /* Initialize the ending block: */ + orc =3D __orc_find(__start_orc_unwind_ip, __start_orc_unwind, num_entries, + LOOKUP_STOP_IP); + if (!orc) { + orc_warn("WARNING: Corrupt .orc_unwind table. Disabling unwinder.\n"); + return; + } + orc_lookup[lookup_num_blocks-1] =3D orc - __start_orc_unwind; + + orc_init =3D true; +} + +__weak struct orc_entry *arch_orc_find(unsigned long ip) +{ + return NULL; +} --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9619CC636D4 for ; Thu, 2 Feb 2023 07:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231926AbjBBHmz (ORCPT ); Thu, 2 Feb 2023 02:42:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231917AbjBBHmm (ORCPT ); Thu, 2 Feb 2023 02:42:42 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B34268494A; Wed, 1 Feb 2023 23:42:33 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id AD63B20B74FF; Wed, 1 Feb 2023 23:42:32 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com AD63B20B74FF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323753; bh=52SrY+gLkGQ+qY0RKHseR68Es2aziku7Iq+7Kj3IvpA=; h=From:To:Subject:Date:In-Reply-To:References:From; b=RUt80pjNp8C7trJI0sPTY1QI6sO7KgTQpWVd5pInOxNNd/XWFYJ1TUZS+69s9qN/6 OwruZ8I1N1PDYZXxQsgW6hzU2KESnVVqsn6j1IOeCcdNnEDAKF8o/BokI8HmeiETUG btkxQ9AB+SmHdZ4i2C8s/js9ajspwZApYakUgFK0= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 08/22] objtool: Introduce STATIC_CHECK Date: Thu, 2 Feb 2023 01:40:22 -0600 Message-Id: <20230202074036.507249-9-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Objtool currently implements static stack validation. Another method called dynamic validation can be supported for other architectures. Define STATIC_CHECK to select the files required for static validation in objtool build. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/Build | 6 +++--- tools/objtool/Makefile | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/objtool/Build b/tools/objtool/Build index c4666d0b40ba..974290dc4aac 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -2,13 +2,13 @@ objtool-y +=3D arch/$(SRCARCH)/ =20 objtool-y +=3D weak.o =20 -objtool-y +=3D check.o -objtool-y +=3D special.o +objtool-$(STATIC_CHECK) +=3D check.o +objtool-$(STATIC_CHECK) +=3D special.o objtool-y +=3D builtin-check.o objtool-y +=3D cfi.o objtool-y +=3D insn.o objtool-y +=3D decode.o -objtool-y +=3D unwind_hints.o +objtool-$(STATIC_CHECK) +=3D unwind_hints.o objtool-y +=3D elf.o objtool-y +=3D objtool.o =20 diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile index a3a9cc24e0e3..797d1ea02db0 100644 --- a/tools/objtool/Makefile +++ b/tools/objtool/Makefile @@ -43,9 +43,10 @@ BUILD_ORC :=3D n =20 ifeq ($(SRCARCH),x86) BUILD_ORC :=3D y + STATIC_CHECK :=3D y endif =20 -export BUILD_ORC +export BUILD_ORC STATIC_CHECK export srctree OUTPUT CFLAGS SRCARCH AWK include $(srctree)/tools/build/Makefile.include =20 --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7065CC61DA4 for ; Thu, 2 Feb 2023 07:43:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232011AbjBBHnA (ORCPT ); Thu, 2 Feb 2023 02:43:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231932AbjBBHmt (ORCPT ); Thu, 2 Feb 2023 02:42:49 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9F4F084950; Wed, 1 Feb 2023 23:42:34 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id AB75D20B74FE; Wed, 1 Feb 2023 23:42:33 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com AB75D20B74FE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323754; bh=KaIKlI0UKpV8EYektSfNHyuf2dAhVB2Dmflmbr+t46g=; h=From:To:Subject:Date:In-Reply-To:References:From; b=dKp0FGp8fcilc2cU+rUUiFilgQqSIYYrFFMq9N5fbTNWTC7F/drDb/6gGAtDUCuCf Qn+Lq1vuSFzmJcbGi6t5+hcEBStvg4oCrb3dtKp4PGv+ISFV/3SEfZzVgXOmMmjE2f DQthM+Pdi/aKJ1vmI7rWPPEk/CAli8A474Mdk15I= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 09/22] objtool: arm64: Add basic definitions and compile Date: Thu, 2 Feb 2023 01:40:23 -0600 Message-Id: <20230202074036.507249-10-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Add CFI definitions and Endianness for ARM64. Add DYNAMIC_CHECK option for ARM64. Provide stubs for arch_decode_instructions() and check() just to get Objtool to build on ARM64. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/Build | 1 + tools/objtool/Makefile | 6 +++++- tools/objtool/arch/arm64/Build | 1 + tools/objtool/arch/arm64/decode.c | 21 +++++++++++++++++++ .../arch/arm64/include/arch/cfi_regs.h | 13 ++++++++++++ .../arch/arm64/include/arch/endianness.h | 9 ++++++++ tools/objtool/dcheck.c | 16 ++++++++++++++ 7 files changed, 66 insertions(+), 1 deletion(-) create mode 100644 tools/objtool/arch/arm64/Build create mode 100644 tools/objtool/arch/arm64/decode.c create mode 100644 tools/objtool/arch/arm64/include/arch/cfi_regs.h create mode 100644 tools/objtool/arch/arm64/include/arch/endianness.h create mode 100644 tools/objtool/dcheck.c diff --git a/tools/objtool/Build b/tools/objtool/Build index 974290dc4aac..fb0846b7d95e 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -4,6 +4,7 @@ objtool-y +=3D weak.o =20 objtool-$(STATIC_CHECK) +=3D check.o objtool-$(STATIC_CHECK) +=3D special.o +objtool-$(DYNAMIC_CHECK) +=3D dcheck.o objtool-y +=3D builtin-check.o objtool-y +=3D cfi.o objtool-y +=3D insn.o diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile index 797d1ea02db0..92583b82eb78 100644 --- a/tools/objtool/Makefile +++ b/tools/objtool/Makefile @@ -46,7 +46,11 @@ ifeq ($(SRCARCH),x86) STATIC_CHECK :=3D y endif =20 -export BUILD_ORC STATIC_CHECK +ifeq ($(SRCARCH),arm64) + DYNAMIC_CHECK :=3D y +endif + +export BUILD_ORC STATIC_CHECK DYNAMIC_CHECK export srctree OUTPUT CFLAGS SRCARCH AWK include $(srctree)/tools/build/Makefile.include =20 diff --git a/tools/objtool/arch/arm64/Build b/tools/objtool/arch/arm64/Build new file mode 100644 index 000000000000..3ff1f00c6a47 --- /dev/null +++ b/tools/objtool/arch/arm64/Build @@ -0,0 +1 @@ +objtool-y +=3D decode.o diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/d= ecode.c new file mode 100644 index 000000000000..69f851337537 --- /dev/null +++ b/tools/objtool/arch/arm64/decode.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ + +#include +#include + +#include + +int arch_decode_instruction(struct objtool_file *file, + const struct section *sec, + unsigned long offset, unsigned int maxlen, + unsigned int *len, enum insn_type *type, + unsigned long *immediate, + struct list_head *ops_list) +{ + return 0; +} diff --git a/tools/objtool/arch/arm64/include/arch/cfi_regs.h b/tools/objto= ol/arch/arm64/include/arch/cfi_regs.h new file mode 100644 index 000000000000..cff3b04d7248 --- /dev/null +++ b/tools/objtool/arch/arm64/include/arch/cfi_regs.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef _OBJTOOL_CFI_REGS_H +#define _OBJTOOL_CFI_REGS_H + +#define CFI_FP 29 +#define CFI_BP CFI_FP +#define CFI_RA 30 +#define CFI_SP 31 + +#define CFI_NUM_REGS 32 + +#endif /* _OBJTOOL_CFI_REGS_H */ diff --git a/tools/objtool/arch/arm64/include/arch/endianness.h b/tools/obj= tool/arch/arm64/include/arch/endianness.h new file mode 100644 index 000000000000..7c362527da20 --- /dev/null +++ b/tools/objtool/arch/arm64/include/arch/endianness.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +#ifndef _ARCH_ENDIANNESS_H +#define _ARCH_ENDIANNESS_H + +#include + +#define __TARGET_BYTE_ORDER __LITTLE_ENDIAN + +#endif /* _ARCH_ENDIANNESS_H */ diff --git a/tools/objtool/dcheck.c b/tools/objtool/dcheck.c new file mode 100644 index 000000000000..e2098c9ad282 --- /dev/null +++ b/tools/objtool/dcheck.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2015-2017 Josh Poimboeuf + */ + +#include +#include +#include +#include + +#include + +int check(struct objtool_file *file) +{ + return 0; +} --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCA85C05027 for ; Thu, 2 Feb 2023 07:43:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232081AbjBBHnE (ORCPT ); Thu, 2 Feb 2023 02:43:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231964AbjBBHmu (ORCPT ); Thu, 2 Feb 2023 02:42:50 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9E348841A2; Wed, 1 Feb 2023 23:42:35 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id A987B2086203; Wed, 1 Feb 2023 23:42:34 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A987B2086203 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323755; bh=4fW0dt8uJ6yPubxoQOl47EhlJ/H7JCFSPjMi/D921fM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=mhzukO34LKeqP16XWJzkrtzw10Vu4jVQh4Ow/u5GFP36iXhsLeZI4pjRsvhpkfozh 2cV9upnaldaqgEYwf5xQjRxoy/VmR55R7tHHlKhKtMTRUOnQErGFX9iRDImUFCWgab 5X4LEd4KPZxGJ7WSqesDmwh8rOcrr2/0q23oCEQg= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 10/22] objtool: arm64: Implement decoder for Dynamic FP validation Date: Thu, 2 Feb 2023 01:40:24 -0600 Message-Id: <20230202074036.507249-11-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Implement arch_decode_instruction() for ARM64. For Dynamic FP validation, we need to walk each function's code and determine the stack and frame offsets at each instruction. So, the following instructions are completely decoded: Instructions that affect the SP and FP: - Load-Store instructions - Add/Sub/Mov instructions Instructions that affect control flow: - Branch instructions - Call instructions - Return instructions Miscellaneous instructions: - Break instruction used for bugs - Paciasp instruction that occurs at the beginning of the frame pointer prolog The rest of the instructions are either dont-care from an unwind perspective or unexpected from the compiler. Add checks for the unexpected ones to catch them if the compiler ever generates them. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/arch/arm64/decode.c | 506 ++++++++++++++++++++++++++- tools/objtool/include/objtool/arch.h | 2 + 2 files changed, 507 insertions(+), 1 deletion(-) diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/d= ecode.c index 69f851337537..aaae16791807 100644 --- a/tools/objtool/arch/arm64/decode.c +++ b/tools/objtool/arch/arm64/decode.c @@ -1,5 +1,9 @@ // SPDX-License-Identifier: GPL-2.0-or-later /* + * decode.c - ARM64 instruction decoder for dynamic FP validation. Only a + * small subset of the instructions need to be decoded. The rest + * only need to be sanity checked. + * * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) * * Copyright (C) 2022 Microsoft Corporation @@ -7,15 +11,515 @@ =20 #include #include +#include =20 #include +#include +#include + +/* ARM64 instructions are all 4 bytes wide. */ +#define INSN_SIZE 4 + +/* --------------------- instruction decode structs ----------------------= -- */ + +struct decode_var { + u32 insn; + enum insn_type type; + s64 imm; + unsigned int mode1; + unsigned int mode2; + unsigned int check_reg; + struct list_head *ops; +}; + +struct decode { + unsigned long opmask; + unsigned long op; + unsigned int width; + unsigned int shift; + unsigned int bits; + unsigned int sign_extend; + unsigned int mult; + unsigned int mode1; + unsigned int mode2; + void (*func)(struct decode *decode, struct decode_var *var); +}; + +struct class { + unsigned long opmask; + unsigned long op; + void (*check)(struct decode_var *var); +}; + +/* ------------------------ stack operations -----------------------------= -- */ + +static void add_stack_op(unsigned char src_reg, enum op_src_type src_type, + s64 src_offset, + unsigned char dest_reg, enum op_dest_type dest_type, + s64 dest_offset, + struct list_head *ops) +{ + struct stack_op *op; + + op =3D calloc(1, sizeof(*op)); + if (!op) { + WARN("calloc failed"); + return; + } + + op->src.reg =3D src_reg; + op->src.type =3D src_type; + op->src.offset =3D src_offset; + op->dest.reg =3D dest_reg; + op->dest.type =3D dest_type; + op->dest.offset =3D dest_offset; + + list_add_tail(&op->list, ops); +} + +static void add_op(struct decode_var *var, + unsigned char rn, s64 offset, unsigned char rd) +{ + add_stack_op(rn, OP_SRC_ADD, offset, rd, OP_DEST_REG, 0, var->ops); +} + +static void load_op(struct decode_var *var, s64 offset, unsigned char rd) +{ + add_stack_op(CFI_SP, OP_SRC_REG_INDIRECT, offset, rd, OP_DEST_REG, 0, + var->ops); +} + +static void store_op(struct decode_var *var, s64 offset, unsigned char rd) +{ + add_stack_op(CFI_SP, OP_SRC_REG, 0, rd, OP_DEST_REG_INDIRECT, offset, + var->ops); +} + +/* ------------------------ decode functions -----------------------------= -- */ + +#define is_saved_reg(rt) ((rt) =3D=3D CFI_FP || (rt) =3D=3D CFI_RA) +#define is_frame_reg(rt) ((rt) =3D=3D CFI_FP || (rt) =3D=3D CFI_SP) + +/* ----- Add/Subtract instructions. ----- */ + +#define CMN_OP 0x31000000 /* Alias of ADDS imm */ +#define CMP_OP 0x71000000 /* Alias of SUBS imm */ + +static void add(struct decode *decode, struct decode_var *var) +{ + unsigned int rd =3D var->insn & 0x1F; + unsigned int rn =3D (var->insn >> 5) & 0x1F; + unsigned int shift =3D (var->insn >> 22) & 1; + + if (decode->op =3D=3D CMN_OP || decode->op =3D=3D CMP_OP) + return; + + if (!is_frame_reg(rd)) + return; + + if (is_frame_reg(rn)) { + if (shift) + var->imm <<=3D 12; + add_op(var, rn, var->imm, rd); + } else { + var->type =3D INSN_UNRELIABLE; + } +} + +#define CMN_EXT_OP 0x2B200000 /* Alias of ADDS ext */ +#define CMP_EXT_OP 0x6B200000 /* Alias of SUBS ext */ + +static void addc(struct decode *decode, struct decode_var *var) +{ + unsigned int rd =3D var->insn & 0x1F; + + if (decode->op =3D=3D CMN_EXT_OP || decode->op =3D=3D CMP_EXT_OP) + return; + + if (is_frame_reg(rd)) + var->type =3D INSN_UNRELIABLE; +} + +static void sub(struct decode *decode, struct decode_var *var) +{ + var->imm =3D -var->imm; + return add(decode, var); +} + +/* ----- Load instructions. ----- */ + +/* + * For some instructions, the target register cannot be FP. There are 3 ca= ses: + * + * - The register width is 32 bits. FP cannot be 32 bits. + * - The register is loaded from one that is not the SP. We do not track + * the value of other registers in static analysis. + * - The instruction does not make sense for the FP to be the target. + */ +static void check_reg(unsigned int reg, struct decode_var *var) +{ + if (reg =3D=3D CFI_FP) + var->type =3D INSN_UNRELIABLE; +} + +static void ldp(struct decode *decode, struct decode_var *var) +{ + unsigned int rt1 =3D var->insn & 0x1F; + unsigned int rt2 =3D (var->insn >> 10) & 0x1F; + unsigned int rn =3D (var->insn >> 5) & 0x1F; + s64 imm; + + if (rn !=3D CFI_SP || var->check_reg) { + check_reg(rt1, var); + check_reg(rt2, var); + } + + if (rn =3D=3D CFI_SP) { + if (var->mode1 && var->mode2) /* Pre-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + + imm =3D var->mode1 ? 0 : var->imm; + if (is_saved_reg(rt1)) + load_op(var, imm, rt1); + if (is_saved_reg(rt2)) + load_op(var, imm + 8, rt2); + + if (var->mode1 && !var->mode2) /* Post-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + } +} + +static void ldpc(struct decode *decode, struct decode_var *var) +{ + var->check_reg =3D 1; + ldp(decode, var); +} + +static void ldr(struct decode *decode, struct decode_var *var) +{ + unsigned int rd =3D var->insn & 0x1F; + unsigned int rn =3D (var->insn >> 5) & 0x1F; + s64 imm; + + if (rn !=3D CFI_SP || var->check_reg) + check_reg(rd, var); + + if (rn =3D=3D CFI_SP) { + if (var->mode1 && var->mode2) /* Pre-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + + imm =3D var->mode1 ? 0 : var->imm; + if (is_saved_reg(rd)) + load_op(var, imm, rd); + + if (var->mode1 && !var->mode2) /* Post-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + } +} + +/* ----- Store instructions. ----- */ + +static void stp(struct decode *decode, struct decode_var *var) +{ + unsigned int rt1 =3D var->insn & 0x1F; + unsigned int rt2 =3D (var->insn >> 10) & 0x1F; + unsigned int rn =3D (var->insn >> 5) & 0x1F; + s64 imm; + + if (var->check_reg) { + check_reg(rt1, var); + check_reg(rt2, var); + } + + if (rn =3D=3D CFI_SP) { + if (var->mode1 && var->mode2) /* Pre-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + + imm =3D var->mode1 ? 0 : var->imm; + if (is_saved_reg(rt1)) + store_op(var, imm, rt1); + if (is_saved_reg(rt2)) + store_op(var, imm + 8, rt2); + + if (var->mode1 && !var->mode2) /* Post-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + } +} + +static void stpc(struct decode *decode, struct decode_var *var) +{ + var->check_reg =3D 1; + stp(decode, var); +} + +static void str(struct decode *decode, struct decode_var *var) +{ + unsigned int rd =3D var->insn & 0x1F; + unsigned int rn =3D (var->insn >> 5) & 0x1F; + s64 imm; + + if (var->check_reg) + check_reg(rd, var); + + if (rn =3D=3D CFI_SP) { + if (var->mode1 && var->mode2) /* Pre-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + + imm =3D var->mode1 ? 0 : var->imm; + if (is_saved_reg(rd)) + store_op(var, imm, rd); + + if (var->mode1 && !var->mode2) /* Post-index */ + add_op(var, CFI_SP, var->imm, CFI_SP); + } +} + +static void strc(struct decode *decode, struct decode_var *var) +{ + var->check_reg =3D 1; + str(decode, var); +} + +/* ----- Control transfer instructions. ----- */ + +#define BR_UNCONDITIONAL 0x14000000 + +static void bra(struct decode *decode, struct decode_var *var) +{ + if (var->imm) { + if (decode->op =3D=3D BR_UNCONDITIONAL) + var->type =3D INSN_JUMP_UNCONDITIONAL; + else + var->type =3D INSN_JUMP_CONDITIONAL; + } else { + var->type =3D INSN_JUMP_DYNAMIC; + } +} + +static void call(struct decode *decode, struct decode_var *var) +{ + var->type =3D var->imm ? INSN_CALL : INSN_CALL_DYNAMIC; +} + +static void ret(struct decode *decode, struct decode_var *var) +{ + var->type =3D INSN_RETURN; +} + +/* ----- Miscellaneous instructions. ----- */ + +static void bug(struct decode *decode, struct decode_var *var) +{ + var->type =3D INSN_BUG; +} + +static void pac(struct decode *decode, struct decode_var *var) +{ + var->type =3D INSN_START; +} + +/* ------------------------ Instruction decode ---------------------------= -- */ + +struct decode decode_array[] =3D { +/* + * mask OP code mask + * opcode OP code + * width Target register width. Values can be: + * 64 (64-bit) + * 32 (32-bit), + * X (64-bit if bit X in the instruction is set) + * -X (32-bit if bit X in the instruction is set) + * shift Shift for the immediate value + * bits Number of bits in the immediate value + * sign Sign extend the immediate value + * mult Multiplier for the immediate value + * am1 Addressing mode bit 1 + * am2 Addressing mode bit 2 + * func Decode function + * + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D INSTRUCTIONS =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * mask opcode width shift bits sign mult am1 am2 func + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + */ +{ 0x7E400000, 0x28400000, 31, 15, 7, 1, 0, 23, 24, ldp /* LDP = */}, +{ 0x7E400000, 0x68400000, 32, 15, 7, 1, 4, 23, 24, ldp /* LDPSW = */}, +{ 0x7FC00000, 0x28400000, 31, 15, 7, 1, 0, 0, 0, ldpc /* LDNP = */}, +{ 0xBFE00000, 0xB8400000, 30, 12, 9, 1, 1, 10, 11, ldr /* LDR = */}, +{ 0xBFC00000, 0xB9400000, 30, 10, 12, 0, 0, 0, 0, ldr /* LDR off = */}, +{ 0xFF200400, 0xF8200400, 64, 12, 9, 1, 8, 11, 11, ldr /* LDRA = */}, +{ 0xFFC00000, 0x39400000, 32, 10, 12, 0, 1, 0, 0, ldr /* LDRB off= */}, +{ 0xFFE00000, 0x38400000, 32, 12, 9, 1, 1, 10, 11, ldr /* LDRB = */}, +{ 0xFFC00000, 0x79400000, 32, 10, 12, 0, 2, 0, 0, ldr /* LDRH off= */}, +{ 0xFFE00000, 0x78400000, 32, 12, 9, 1, 1, 10, 11, ldr /* LDRH = */}, +{ 0xFF800000, 0x39800000, -22, 10, 12, 0, 1, 0, 0, ldr /* LDRSB of= f */}, +{ 0xFFA00000, 0x38800000, -22, 12, 9, 1, 1, 10, 11, ldr /* LDRSB = */}, +{ 0xFF800000, 0x79800000, -22, 10, 12, 0, 2, 0, 0, ldr /* LDRSH of= f */}, +{ 0xFFA00000, 0x78800000, -22, 12, 9, 1, 1, 10, 11, ldr /* LDRSH = */}, +{ 0xFFC00000, 0xB9800000, 32, 10, 12, 0, 4, 0, 0, ldr /* LDRSW of= f */}, +{ 0xFFE00000, 0xB8800000, 32, 12, 9, 1, 1, 10, 11, ldr /* LDRSW = */}, +{ 0x7E000000, 0x28000000, 31, 15, 7, 1, 0, 23, 24, stp /* STP = */}, +{ 0x7E400000, 0x28000000, 31, 15, 7, 1, 0, 23, 24, stp /* STG = */}, +{ 0xFE400000, 0x68000000, 64, 15, 7, 1, 16, 23, 24, stpc /* STGP = */}, +{ 0x7FC00000, 0x28000000, 31, 15, 7, 1, 0, 0, 0, stpc /* STNP = */}, +{ 0xBFC00000, 0xB9000000, 30, 10, 12, 0, 0, 0, 0, str /* STR off = */}, +{ 0xBFE00000, 0xB8000000, 30, 12, 9, 1, 1, 10, 11, str /* STR = */}, +{ 0xFFE00000, 0xD9200000, 64, 12, 9, 1, 16, 10, 11, strc /* STG = */}, +{ 0xFFE00000, 0xD9A00000, 64, 12, 9, 1, 16, 10, 11, strc /* ST2G = */}, +{ 0x7F800000, 0x11000000, 31, 10, 12, 0, 1, 0, 0, add /* ADD imm = */}, +{ 0x7FE00000, 0x0B200000, 31, 10, 3, 0, 1, 0, 0, addc /* ADD ext = */}, +{ 0x7F800000, 0x31000000, 31, 10, 12, 0, 1, 0, 0, add /* ADDS imm= */}, +{ 0x7FE00000, 0x2B200000, 31, 10, 3, 0, 1, 0, 0, addc /* ADDS ext= */}, +{ 0x7F800000, 0x51000000, 31, 10, 12, 0, 1, 0, 0, sub /* SUB imm = */}, +{ 0x7FE00000, 0x4B200000, 31, 10, 3, 0, 1, 0, 0, addc /* SUB ext = */}, +{ 0x7F800000, 0x71000000, 31, 10, 12, 0, 1, 0, 0, sub /* SUBS imm= */}, +{ 0x7FE00000, 0x6B200000, 31, 10, 3, 0, 1, 0, 0, addc /* SUBS ext= */}, +{ 0xFC000000, 0x14000000, 64, 0, 26, 1, 4, 0, 0, bra /* B = */}, +{ 0xFF000010, 0x54000000, 64, 5, 19, 1, 4, 0, 0, bra /* B.cond = */}, +{ 0xFF000010, 0x54000010, 64, 5, 19, 1, 4, 0, 0, bra /* BC.cond = */}, +{ 0xFFFFFC1F, 0xD61F0000, 64, 0, 0, 0, 0, 0, 0, bra /* BR = */}, +{ 0xFEFFF800, 0xD61F0800, 64, 0, 0, 0, 0, 0, 0, bra /* BRA = */}, +{ 0x7E000000, 0x34000000, 31, 5, 19, 1, 4, 0, 0, bra /* CBZ/CBNZ= */}, +{ 0x7E000000, 0x36000000, 31, 5, 14, 1, 4, 0, 0, bra /* TBZ/TBNZ= */}, +{ 0xFC000000, 0x94000000, 64, 0, 26, 1, 4, 0, 0, call /* BL = */}, +{ 0xFFFFFC1F, 0xD63F0000, 64, 0, 0, 0, 0, 0, 0, call /* BLR = */}, +{ 0xFEFFF800, 0xD63F0800, 64, 0, 0, 0, 0, 0, 0, call /* BLRA = */}, +{ 0xFFFFFC1F, 0xD65F0000, 64, 0, 0, 0, 0, 0, 0, ret /* RET = */}, +{ 0xFFFFFBFF, 0xD65F0BFF, 64, 0, 0, 0, 0, 0, 0, ret /* RETA = */}, +{ 0xFFFFFFFF, 0xD69F03E0, 64, 0, 0, 0, 0, 0, 0, ret /* ERET = */}, +{ 0xFFFFFBFF, 0xD69F0BFF, 64, 0, 0, 0, 0, 0, 0, ret /* ERETA = */}, +{ 0xFFE00000, 0xD4200000, 64, 5, 16, 0, 1, 0, 0, bug /* BRK = */}, +{ 0xFFFFFFFF, 0xD503233F, 64, 0, 0, 0, 1, 0, 0, pac /* PACIASP = */}, +}; +unsigned int ndecode =3D ARRAY_SIZE(decode_array); + +static void ignore(struct decode_var *var) +{ +} + +static void check_target(struct decode_var *var) +{ + unsigned int rd =3D var->insn & 0x1F; + + check_reg(rd, var); +} + +struct class class_array[] =3D { +/* + * mask Class OP mask + * opcode Class OP code + * check Function to perform checks + * + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D INSTRUCTION CLASSES =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * mask opcode check + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + */ +{ 0x1E000000, 0x00000000, ignore /* RSVD_00 */ }, +{ 0x1E000000, 0x02000000, ignore /* UNALLOC_01 */ }, +{ 0x1E000000, 0x04000000, ignore /* SVE_02 */ }, +{ 0x1E000000, 0x06000000, ignore /* UNALLOC_03 */ }, +{ 0x1E000000, 0x08000000, check_target /* LOAD_STORE_04 */ }, +{ 0x1E000000, 0x0A000000, check_target /* DP_REGISTER_05 */ }, +{ 0x1E000000, 0x0C000000, ignore /* LOAD_STORE_06 */ }, +{ 0x1E000000, 0x0E000000, ignore /* SIMD_FP_07 */ }, +{ 0x1E000000, 0x12000000, check_target /* DP_IMMEDIATE_09 */ }, +{ 0x1E000000, 0x10000000, check_target /* DP_IMMEDIATE_08 */ }, +{ 0x1E000000, 0x14000000, check_target /* BR_SYS_10 */ }, +{ 0x1E000000, 0x16000000, check_target /* BR_SYS_11 */ }, +{ 0x1E000000, 0x18000000, check_target /* LOAD_STORE_12 */ }, +{ 0x1E000000, 0x1A000000, ignore /* DP_REGISTER_13 */ }, +{ 0x1E000000, 0x1C000000, check_target /* LOAD_STORE_14 */ }, +{ 0x1E000000, 0x1E000000, ignore /* SIMD_FP_15 */ }, +}; +unsigned int nclass =3D ARRAY_SIZE(class_array); + +static inline s64 sign_extend(s64 imm, unsigned int bits) +{ + return (imm << (64 - bits)) >> (64 - bits); +} =20 int arch_decode_instruction(struct objtool_file *file, const struct section *sec, unsigned long offset, unsigned int maxlen, unsigned int *len, enum insn_type *type, unsigned long *immediate, - struct list_head *ops_list) + struct list_head *ops) { + struct decode *decode; + struct decode_var var; + struct class *class; + unsigned int width, mask, mult, i; + + if (maxlen < INSN_SIZE) + return -1; + *len =3D INSN_SIZE; + + var.insn =3D *(u32 *)(sec->data->d_buf + offset); + var.type =3D INSN_OTHER; + var.imm =3D 0; + var.ops =3D ops; + + *type =3D INSN_OTHER; + + /* Decode the instruction, if listed. */ + for (i =3D 0; i < ndecode; i++) { + decode =3D &decode_array[i]; + + if ((var.insn & decode->opmask) !=3D decode->op) + continue; + + /* Extract addressing mode (for some instructions). */ + var.mode1 =3D 0; + var.mode2 =3D 0; + if (decode->mode1) + var.mode1 =3D (var.insn >> decode->mode1) & 1; + if (decode->mode2) + var.mode2 =3D (var.insn >> decode->mode2) & 1; + + /* Determine target register width. */ + width =3D decode->width; + if (width < 0) + width =3D (var.insn & (1 << -width)) ? 32 : 64; + else if (width < 32) + width =3D (var.insn & (1 << width)) ? 64 : 32; + + /* + * If the target register width is 32 bits, set the check flag + * so that the target registers are checked to make sure they + * are not the FP or the RA. We should not be using 32-bit + * values in these registers. + */ + var.check_reg =3D (width =3D=3D 32); + + /* Extract the immediate value. */ + mask =3D (1 << decode->bits) - 1; + var.imm =3D (var.insn >> decode->shift) & mask; + if (decode->sign_extend) + var.imm =3D sign_extend(var.imm, decode->bits); + + /* Scale the immediate value. */ + mult =3D decode->mult; + if (!mult) + mult =3D (width =3D=3D 32) ? 4 : 8; + var.imm *=3D mult; + + /* Decode the instruction. */ + decode->func(decode, &var); + goto out; + } + + /* + * Sanity check to make sure that the compiler has not generated + * code that modifies the FP or the RA in an unexpected way. + */ + for (i =3D 0; i < nclass; i++) { + class =3D &class_array[i]; + if ((var.insn & class->opmask) =3D=3D class->op) { + class->check(&var); + goto out; + } + } +out: + *immediate =3D var.imm; + *type =3D var.type; return 0; } diff --git a/tools/objtool/include/objtool/arch.h b/tools/objtool/include/o= bjtool/arch.h index beb2f3aa94ff..3c2f8c1b8265 100644 --- a/tools/objtool/include/objtool/arch.h +++ b/tools/objtool/include/objtool/arch.h @@ -29,6 +29,8 @@ enum insn_type { INSN_TRAP, INSN_ENDBR, INSN_OTHER, + INSN_START, + INSN_UNRELIABLE, }; =20 enum op_dest_type { --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06D9FC05027 for ; Thu, 2 Feb 2023 07:43:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232090AbjBBHnH (ORCPT ); Thu, 2 Feb 2023 02:43:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231971AbjBBHmv (ORCPT ); Thu, 2 Feb 2023 02:42:51 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B3C688495D; Wed, 1 Feb 2023 23:42:36 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id A79A520B96E8; Wed, 1 Feb 2023 23:42:35 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A79A520B96E8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323756; bh=3qU35kR6nm9/qdtMkBRKO5MFQ8eSw6OI7xVYMG8qPdU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=hONvI3e0GsW7hJPLcxlcDOCGFQv9nEsAM58s8vmj+SktxBGS71H6LsGwW6rFfpKPF sxZ3btYfSM9XR2NeR07R7+VEqWKQsP10HiBErjXRDWnlujUAvocDiERW3Oi5K9fKc/ BmaFGuQS65bH+qQz9rwbDoYyISLXSIniZW+kuIg0= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 11/22] objtool: arm64: Invoke the decoder Date: Thu, 2 Feb 2023 01:40:25 -0600 Message-Id: <20230202074036.507249-12-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Invoke decode_instructions() from check(). For Dynamic Validation of the frame pointer, we only need the "-s" option for objtool. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/dcheck.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tools/objtool/dcheck.c b/tools/objtool/dcheck.c index e2098c9ad282..cd2700153408 100644 --- a/tools/objtool/dcheck.c +++ b/tools/objtool/dcheck.c @@ -9,8 +9,13 @@ #include =20 #include +#include +#include =20 int check(struct objtool_file *file) { - return 0; + if (!opts.stackval) + return 1; + + return decode_instructions(file); } --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 848C4C61DA4 for ; Thu, 2 Feb 2023 07:43:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232100AbjBBHnL (ORCPT ); Thu, 2 Feb 2023 02:43:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232014AbjBBHmw (ORCPT ); Thu, 2 Feb 2023 02:42:52 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E2E50841BE; Wed, 1 Feb 2023 23:42:37 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id A6A1320B9D4D; Wed, 1 Feb 2023 23:42:36 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A6A1320B9D4D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323757; bh=4Oz2+1ZYx7D5K7SknLE3Ah83SAKi4AO9HhOoR9/v50Y=; h=From:To:Subject:Date:In-Reply-To:References:From; b=ereOmhQVPKXkr6Q/wbie7EuTjecB1QvtMyAGoNimgY3RtObxfhUPlePkFXkuXUT0k 701f0VmYEOI0ApYjBEhQXQ8Mrjmmdfp3fiQZhxf7dlMeg1vqyw7PLO2b/f5ep3pBSx CC3U1QdNyRfpM+E0zs2igZ0BxDGco7KBj6reMcUI= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 12/22] objtool: arm64: Compute destinations for call and jump instructions Date: Thu, 2 Feb 2023 01:40:26 -0600 Message-Id: <20230202074036.507249-13-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Compute the destination address of each call and jump instruction after decoding all the instructions. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/arch/arm64/decode.c | 12 ++++++++ tools/objtool/dcheck.c | 47 ++++++++++++++++++++++++++++++- 2 files changed, 58 insertions(+), 1 deletion(-) diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/d= ecode.c index aaae16791807..81653ed3c323 100644 --- a/tools/objtool/arch/arm64/decode.c +++ b/tools/objtool/arch/arm64/decode.c @@ -20,6 +20,18 @@ /* ARM64 instructions are all 4 bytes wide. */ #define INSN_SIZE 4 =20 +/* --------------------- arch support functions ------------------------- = */ + +unsigned long arch_dest_reloc_offset(int addend) +{ + return addend; +} + +unsigned long arch_jump_destination(struct instruction *insn) +{ + return insn->offset + insn->immediate; +} + /* --------------------- instruction decode structs ----------------------= -- */ =20 struct decode_var { diff --git a/tools/objtool/dcheck.c b/tools/objtool/dcheck.c index cd2700153408..eb806a032a32 100644 --- a/tools/objtool/dcheck.c +++ b/tools/objtool/dcheck.c @@ -12,10 +12,55 @@ #include #include =20 +/* + * Find the destination instructions for all jumps. + */ +static void add_jump_destinations(struct objtool_file *file) +{ + struct instruction *insn; + struct reloc *reloc; + struct section *dest_sec; + unsigned long dest_off; + + for_each_insn(file, insn) { + if (insn->type !=3D INSN_CALL && + insn->type !=3D INSN_JUMP_CONDITIONAL && + insn->type !=3D INSN_JUMP_UNCONDITIONAL) { + continue; + } + + reloc =3D insn_reloc(file, insn); + if (!reloc) { + dest_sec =3D insn->sec; + dest_off =3D arch_jump_destination(insn); + } else if (reloc->sym->type =3D=3D STT_SECTION) { + dest_sec =3D reloc->sym->sec; + dest_off =3D arch_dest_reloc_offset(reloc->addend); + } else if (reloc->sym->sec->idx) { + dest_sec =3D reloc->sym->sec; + dest_off =3D reloc->sym->sym.st_value + + arch_dest_reloc_offset(reloc->addend); + } else { + /* non-func asm code jumping to another file */ + continue; + } + + insn->jump_dest =3D find_insn(file, dest_sec, dest_off); + } +} + int check(struct objtool_file *file) { + int ret; + if (!opts.stackval) return 1; =20 - return decode_instructions(file); + ret =3D decode_instructions(file); + if (ret) + return ret; + + add_jump_destinations(file); + + return 0; } --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97732C63797 for ; Thu, 2 Feb 2023 07:43:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231835AbjBBHnW (ORCPT ); Thu, 2 Feb 2023 02:43:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231956AbjBBHmx (ORCPT ); Thu, 2 Feb 2023 02:42:53 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C0FFD84952; Wed, 1 Feb 2023 23:42:38 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id A4A8720861F6; Wed, 1 Feb 2023 23:42:37 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A4A8720861F6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323758; bh=ZReeOzoTrtsb5LMSAI87dQm76FTp/hJZoPEJyZX88wA=; h=From:To:Subject:Date:In-Reply-To:References:From; b=LoHhNNqfKNphpcEW3Qd8TKXIdVNy37/Cs3LLnduT4IgGh3B7pNL/KWbIOyStRmTZi U94SYC1JoUa9b8XD8x6HHQygE5AhnhZWr747OMUR2UApiZ0yyL8dQr/Y7FXJmZa0KX BHDkLPunEUG48UuAFm/Z0i0FAT5kmcaKOsQ5IFE4= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 13/22] objtool: arm64: Walk instructions and compute CFI for each instruction Date: Thu, 2 Feb 2023 01:40:27 -0600 Message-Id: <20230202074036.507249-14-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Implement arch_initial_func_cfi_state() to initialize the CFI for a function. Add code to check() in dcheck.c to walk the instructions in every function and compute the CFI information for each instruction. Perform the following checks to validate the CFI: - Make sure that there is exactly one frame pointer prolog for an epilog. - Make sure that the frame pointer register is initialized to the location at which the previous frame pointer is stored on the stack. - Make sure that the frame pointer is restored in the epilog from the same location on stack where it was saved. - Make sure that the return address is restored in the epilog from the same location on stack where it was saved. - Make sure that the frame pointer and return address are saved on the stack adjacent to each other in the correct order as specified in the ABI. - If an instruction can be reached via two different code paths, make sure that the CFIs computed from traversing each path match for the instruction. - Every time the frame pointer or stack offset is changed, make sure the offsets have legal values. insn_cfi_match() is used to compare CFIs to see if they match. When there is a mismatch, the function emits error messages. With static checking, these errors result in failure. With dynamic checking, these errors only resulting in marking those instructions as unreliable for unwind. In the latter case, suppress the warning messages. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/arch/arm64/decode.c | 15 ++ tools/objtool/check.c | 2 +- tools/objtool/dcheck.c | 287 +++++++++++++++++++++++++++ tools/objtool/include/objtool/insn.h | 3 +- tools/objtool/insn.c | 39 ++-- 5 files changed, 329 insertions(+), 17 deletions(-) diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/d= ecode.c index 81653ed3c323..f723be80c09a 100644 --- a/tools/objtool/arch/arm64/decode.c +++ b/tools/objtool/arch/arm64/decode.c @@ -22,6 +22,21 @@ =20 /* --------------------- arch support functions ------------------------- = */ =20 +void arch_initial_func_cfi_state(struct cfi_init_state *state) +{ + int i; + + for (i =3D 0; i < CFI_NUM_REGS; i++) { + state->regs[i].base =3D CFI_UNDEFINED; + state->regs[i].offset =3D 0; + } + state->regs[CFI_FP].base =3D CFI_CFA; + + /* initial CFA (call frame address) */ + state->cfa.base =3D CFI_SP; + state->cfa.offset =3D 0; +} + unsigned long arch_dest_reloc_offset(int addend) { return addend; diff --git a/tools/objtool/check.c b/tools/objtool/check.c index d14a2b7b8b37..94efe94a566e 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2863,7 +2863,7 @@ static int validate_branch(struct objtool_file *file,= struct symbol *func, =20 visited =3D VISITED_BRANCH << state.uaccess; if (insn->visited & VISITED_BRANCH_MASK) { - if (!insn->hint && !insn_cfi_match(insn, &state.cfi)) + if (!insn->hint && !insn_cfi_match(insn, &state.cfi, true)) return 1; =20 if (insn->visited & visited) diff --git a/tools/objtool/dcheck.c b/tools/objtool/dcheck.c index eb806a032a32..8b78cb608528 100644 --- a/tools/objtool/dcheck.c +++ b/tools/objtool/dcheck.c @@ -49,6 +49,283 @@ static void add_jump_destinations(struct objtool_file *= file) } } =20 +static bool update_cfi_state(struct cfi_state *cfi, struct stack_op *op) +{ + struct cfi_reg *cfa =3D &cfi->cfa; + struct cfi_reg *fp_reg =3D &cfi->regs[CFI_FP]; + struct cfi_reg *fp_val =3D &cfi->vals[CFI_FP]; + struct cfi_reg *ra_val =3D &cfi->vals[CFI_RA]; + enum op_src_type src_type =3D op->src.type; + enum op_dest_type dest_type =3D op->dest.type; + unsigned char dest_reg =3D op->dest.reg; + int offset; + + if (src_type =3D=3D OP_SRC_ADD && dest_type =3D=3D OP_DEST_REG) { + + if (op->src.reg =3D=3D CFI_SP) { + if (op->dest.reg =3D=3D CFI_SP) { + cfa->offset -=3D op->src.offset; + } else { + if (fp_reg->offset) { + /* FP is already set. */ + return false; + } + fp_reg->offset =3D -cfa->offset + op->src.offset; + if (fp_reg->offset !=3D fp_val->offset) { + /* + * FP does not match the location + * where FP is stored on stack. + */ + return false; + } + } + } else { + if (op->dest.reg =3D=3D CFI_SP) { + cfa->offset =3D + -(fp_reg->offset + op->src.offset); + } else { + /* Setting the FP from itself is unreliable. */ + return false; + } + } + /* + * When the stack pointer is restored in the frame pointer + * epilog, forget where the FP and RA were stored. + */ + if (cfa->offset < -fp_val->offset) + fp_val->offset =3D 0; + if (cfa->offset < -ra_val->offset) + ra_val->offset =3D 0; + goto out; + } + + if (src_type =3D=3D OP_SRC_REG_INDIRECT && dest_type =3D=3D OP_DEST_REG) { + offset =3D -cfa->offset + op->src.offset; + if (dest_reg =3D=3D CFI_FP) { + if (!fp_val->offset || fp_val->offset !=3D offset) { + /* + * Loading the FP from a different place than + * where it is stored. + */ + return false; + } + if (!ra_val->offset || + (ra_val->offset - fp_val->offset) !=3D 8) { + /* FP and RA must be adjacent in a frame. */ + return false; + } + fp_reg->offset =3D 0; + } + goto out; + } + + if (src_type =3D=3D OP_SRC_REG && dest_type =3D=3D OP_DEST_REG_INDIRECT) { + offset =3D -cfa->offset + op->dest.offset; + if (dest_reg =3D=3D CFI_FP) { + /* Record where the FP is stored on the stack. */ + fp_val->offset =3D offset; + } else { + /* Record where the RA is stored on the stack. */ + if (fp_val->offset && (offset - fp_val->offset) =3D=3D 8) + ra_val->offset =3D offset; + } + goto out; + } + return false; +out: + if (cfa->offset < 0 || fp_reg->offset > 0 || + fp_val->offset > 0 || ra_val->offset > 0) { + /* Unexpected SP and FP offset values. */ + return false; + } + return true; +} + +static bool do_stack_ops(struct instruction *insn, struct insn_state *stat= e) +{ + struct stack_op *op; + + list_for_each_entry(op, &insn->stack_ops, list) { + if (!update_cfi_state(&state->cfi, op)) + return false; + } + return true; +} + +static bool validate_branch(struct objtool_file *file, struct section *sec, + struct symbol *func, struct instruction *insn, + struct insn_state *state) +{ + struct symbol *insn_func =3D insn->func; + struct instruction *dest; + struct cfi_state save_cfi; + struct cfi_reg *cfa; + struct cfi_reg *regs; + unsigned long start, end; + + for (; insn; insn =3D next_insn_same_sec(file, insn)) { + + if (insn->func !=3D insn_func) + return true; + + if (insn->cfi) + return insn_cfi_match(insn, &state->cfi, false); + + insn->cfi =3D cfi_hash_find_or_add(&state->cfi); + dest =3D insn->jump_dest; + + if (!do_stack_ops(insn, state)) + return false; + + switch (insn->type) { + case INSN_BUG: + return true; + + case INSN_UNRELIABLE: + return false; + + case INSN_RETURN: + cfa =3D &state->cfi.cfa; + regs =3D state->cfi.regs; + if (cfa->offset || regs[CFI_FP].offset) { + /* SP and FP offsets should be 0 on return. */ + return false; + } + return true; + + case INSN_CALL: + case INSN_CALL_DYNAMIC: + start =3D func->offset; + end =3D start + func->len; + /* Treat intra-function calls as jumps. */ + if (!dest || dest->sec !=3D sec || + dest->offset <=3D start || dest->offset >=3D end) { + break; + } + + case INSN_JUMP_UNCONDITIONAL: + case INSN_JUMP_CONDITIONAL: + case INSN_JUMP_DYNAMIC: + if (dest) { + save_cfi =3D state->cfi; + if (!validate_branch(file, sec, func, dest, + state)) { + return false; + } + state->cfi =3D save_cfi; + } + if (insn->type =3D=3D INSN_JUMP_UNCONDITIONAL || + insn->type =3D=3D INSN_JUMP_DYNAMIC) { + return true; + } + break; + + default: + break; + } + } + return true; +} + +static bool walk_reachable(struct objtool_file *file, struct section *sec, + struct symbol *func) +{ + struct instruction *insn =3D find_insn(file, sec, func->offset); + struct insn_state state; + + func_for_each_insn(file, func, insn) { + + if (insn->offset !=3D func->offset && + (insn->type !=3D INSN_START || insn->cfi)) { + continue; + } + + init_insn_state(file, &state, sec); + set_func_state(&state.cfi); + + if (!validate_branch(file, sec, func, insn, &state)) + return false; + } + return true; +} + +static void remove_cfi(struct objtool_file *file, struct symbol *func) +{ + struct instruction *insn; + + func_for_each_insn(file, func, insn) { + insn->cfi =3D NULL; + } +} + +/* + * Instructions that were not visited by walk_reachable() would not have a + * CFI. Try to initialize their CFI. For instance, there could be a table = of + * unconditional branches like for a switch statement. Or, code can be pat= ched + * by the kernel at runtime. After patching, some of the previously unreac= hable + * code may become reachable. + * + * This follows the same pattern as the DWARF info generated by the compil= er. + */ +static bool walk_unreachable(struct objtool_file *file, struct section *se= c, + struct symbol *func) +{ + struct instruction *insn, *prev; + struct insn_state state; + + func_for_each_insn(file, func, insn) { + + if (insn->cfi) + continue; + + prev =3D list_prev_entry(insn, list); + if (!prev || prev->func !=3D insn->func || !prev->cfi) + continue; + + if (prev->type !=3D INSN_JUMP_UNCONDITIONAL && + prev->type !=3D INSN_JUMP_DYNAMIC && + prev->type !=3D INSN_BUG) { + continue; + } + + /* Propagate the CFI. */ + state.cfi =3D *prev->cfi; + if (!validate_branch(file, sec, func, insn, &state)) + return false; + } + return true; +} + +static void walk_section(struct objtool_file *file, struct section *sec) +{ + struct symbol *func; + + list_for_each_entry(func, &sec->symbol_list, list) { + + if (func->type !=3D STT_FUNC || !func->len || + func->pfunc !=3D func || func->alias !=3D func) { + /* No CFI generated for this function. */ + continue; + } + + if (!walk_reachable(file, sec, func) || + !walk_unreachable(file, sec, func)) { + remove_cfi(file, func); + continue; + } + } +} + +static void walk_sections(struct objtool_file *file) +{ + struct section *sec; + + for_each_sec(file, sec) { + if (sec->sh.sh_flags & SHF_EXECINSTR) + walk_section(file, sec); + } +} + int check(struct objtool_file *file) { int ret; @@ -56,11 +333,21 @@ int check(struct objtool_file *file) if (!opts.stackval) return 1; =20 + arch_initial_func_cfi_state(&initial_func_cfi); + + if (!cfi_hash_alloc(1UL << (file->elf->symbol_bits - 3))) + return -1; + ret =3D decode_instructions(file); if (ret) return ret; =20 add_jump_destinations(file); =20 + if (list_empty(&file->insn_list)) + return 0; + + walk_sections(file); + return 0; } diff --git a/tools/objtool/include/objtool/insn.h b/tools/objtool/include/o= bjtool/insn.h index cfd1ae7e2e8e..3a43a591b318 100644 --- a/tools/objtool/include/objtool/insn.h +++ b/tools/objtool/include/objtool/insn.h @@ -84,7 +84,8 @@ struct instruction *next_insn_same_sec(struct objtool_fil= e *file, struct instruction *next_insn_same_func(struct objtool_file *file, struct instruction *insn); struct reloc *insn_reloc(struct objtool_file *file, struct instruction *in= sn); -bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi2); +bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi2, + bool print); bool same_function(struct instruction *insn1, struct instruction *insn2); bool is_first_func_insn(struct objtool_file *file, struct instruction *ins= n); =20 diff --git a/tools/objtool/insn.c b/tools/objtool/insn.c index e570b46ad39e..be3617d55aea 100644 --- a/tools/objtool/insn.c +++ b/tools/objtool/insn.c @@ -135,7 +135,8 @@ bool is_first_func_insn(struct objtool_file *file, stru= ct instruction *insn) return false; } =20 -bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi2) +bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi2, + bool print) { struct cfi_state *cfi1 =3D insn->cfi; int i; @@ -147,10 +148,12 @@ bool insn_cfi_match(struct instruction *insn, struct = cfi_state *cfi2) =20 if (memcmp(&cfi1->cfa, &cfi2->cfa, sizeof(cfi1->cfa))) { =20 - WARN_FUNC("stack state mismatch: cfa1=3D%d%+d cfa2=3D%d%+d", - insn->sec, insn->offset, - cfi1->cfa.base, cfi1->cfa.offset, - cfi2->cfa.base, cfi2->cfa.offset); + if (print) { + WARN_FUNC("stack state mismatch: cfa1=3D%d%+d cfa2=3D%d%+d", + insn->sec, insn->offset, + cfi1->cfa.base, cfi1->cfa.offset, + cfi2->cfa.base, cfi2->cfa.offset); + } =20 } else if (memcmp(&cfi1->regs, &cfi2->regs, sizeof(cfi1->regs))) { for (i =3D 0; i < CFI_NUM_REGS; i++) { @@ -158,26 +161,32 @@ bool insn_cfi_match(struct instruction *insn, struct = cfi_state *cfi2) sizeof(struct cfi_reg))) continue; =20 - WARN_FUNC("stack state mismatch: reg1[%d]=3D%d%+d reg2[%d]=3D%d%+d", - insn->sec, insn->offset, - i, cfi1->regs[i].base, cfi1->regs[i].offset, - i, cfi2->regs[i].base, cfi2->regs[i].offset); + if (print) { + WARN_FUNC("stack state mismatch: reg1[%d]=3D%d%+d reg2[%d]=3D%d%+d", + insn->sec, insn->offset, + i, cfi1->regs[i].base, cfi1->regs[i].offset, + i, cfi2->regs[i].base, cfi2->regs[i].offset); + } break; } =20 } else if (cfi1->type !=3D cfi2->type) { =20 - WARN_FUNC("stack state mismatch: type1=3D%d type2=3D%d", - insn->sec, insn->offset, cfi1->type, cfi2->type); + if (print) { + WARN_FUNC("stack state mismatch: type1=3D%d type2=3D%d", + insn->sec, insn->offset, cfi1->type, cfi2->type); + } =20 } else if (cfi1->drap !=3D cfi2->drap || (cfi1->drap && cfi1->drap_reg !=3D cfi2->drap_reg) || (cfi1->drap && cfi1->drap_offset !=3D cfi2->drap_offset)) { =20 - WARN_FUNC("stack state mismatch: drap1=3D%d(%d,%d) drap2=3D%d(%d,%d)", - insn->sec, insn->offset, - cfi1->drap, cfi1->drap_reg, cfi1->drap_offset, - cfi2->drap, cfi2->drap_reg, cfi2->drap_offset); + if (print) { + WARN_FUNC("stack state mismatch: drap1=3D%d(%d,%d) drap2=3D%d(%d,%d)", + insn->sec, insn->offset, + cfi1->drap, cfi1->drap_reg, cfi1->drap_offset, + cfi2->drap, cfi2->drap_reg, cfi2->drap_offset); + } =20 } else return true; --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A9BFC05027 for ; Thu, 2 Feb 2023 07:43:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232022AbjBBHnY (ORCPT ); Thu, 2 Feb 2023 02:43:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232049AbjBBHmx (ORCPT ); Thu, 2 Feb 2023 02:42:53 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0ED1E84B45; Wed, 1 Feb 2023 23:42:39 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id A2D4620B2EE0; Wed, 1 Feb 2023 23:42:38 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A2D4620B2EE0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323759; bh=jNAeObYr1g94vg9HIxaKJNneDImCayRVcc8/ZI8nYbk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=E9qmkBC171byJFAPg/JByrrx9hcL68uTd/qai9oaoDmLm0tCmvAFBq2TX6t/yicvC rtKPgYcBIXzeXXLw/9Oj5oe1//Fukn/nHcR/X+2aYF9a6I2RCDSvJdTU//O7x8VmQK vh1Q6ZkqmZQqeezBJsKuqZyuOHwM5rXCTH/g7MEk= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 14/22] objtool: arm64: Generate ORC data from CFI for object files Date: Thu, 2 Feb 2023 01:40:28 -0600 Message-Id: <20230202074036.507249-15-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Enable ORC data for ARM64. Call orc_create() from check() in dcheck.c to generate the ORC sections in object files for dynamic frame pointer validation. Define support functions for ORC data creation. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/orc_types.h | 35 +++++++++ tools/arch/arm64/include/asm/orc_types.h | 35 +++++++++ tools/objtool/Makefile | 1 + tools/objtool/arch/arm64/Build | 1 + tools/objtool/arch/arm64/include/arch/elf.h | 9 +++ tools/objtool/arch/arm64/orc.c | 86 +++++++++++++++++++++ tools/objtool/dcheck.c | 5 +- tools/objtool/include/objtool/insn.h | 1 + tools/objtool/include/objtool/objtool.h | 1 + tools/objtool/insn.c | 20 +++++ tools/objtool/orc_gen.c | 12 ++- tools/objtool/sync-check.sh | 7 ++ 12 files changed, 210 insertions(+), 3 deletions(-) create mode 100644 arch/arm64/include/asm/orc_types.h create mode 100644 tools/arch/arm64/include/asm/orc_types.h create mode 100644 tools/objtool/arch/arm64/include/arch/elf.h create mode 100644 tools/objtool/arch/arm64/orc.c diff --git a/arch/arm64/include/asm/orc_types.h b/arch/arm64/include/asm/or= c_types.h new file mode 100644 index 000000000000..c7bb690ca7d9 --- /dev/null +++ b/arch/arm64/include/asm/orc_types.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ + +#ifndef _ORC_TYPES_H +#define _ORC_TYPES_H + +#include +#include +#include + +/* + * The ORC_REG_* registers are base registers which are used to find other + * registers on the stack. + * + * ORC_REG_PREV_SP, also known as DWARF Call Frame Address (CFA), is the + * address of the previous frame: the caller's SP before it called the cur= rent + * function. + * + * ORC_REG_UNDEFINED means the corresponding register's value didn't chang= e in + * the current frame. + * + * We only use base registers SP and FP -- which the previous SP is based = on -- + * and PREV_SP and UNDEFINED -- which the previous FP is based on. + */ +#define ORC_REG_UNDEFINED 0 +#define ORC_REG_PREV_SP 1 +#define ORC_REG_SP 2 +#define ORC_REG_FP 3 +#define ORC_REG_MAX 4 + +#endif /* _ORC_TYPES_H */ diff --git a/tools/arch/arm64/include/asm/orc_types.h b/tools/arch/arm64/in= clude/asm/orc_types.h new file mode 100644 index 000000000000..c7bb690ca7d9 --- /dev/null +++ b/tools/arch/arm64/include/asm/orc_types.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ + +#ifndef _ORC_TYPES_H +#define _ORC_TYPES_H + +#include +#include +#include + +/* + * The ORC_REG_* registers are base registers which are used to find other + * registers on the stack. + * + * ORC_REG_PREV_SP, also known as DWARF Call Frame Address (CFA), is the + * address of the previous frame: the caller's SP before it called the cur= rent + * function. + * + * ORC_REG_UNDEFINED means the corresponding register's value didn't chang= e in + * the current frame. + * + * We only use base registers SP and FP -- which the previous SP is based = on -- + * and PREV_SP and UNDEFINED -- which the previous FP is based on. + */ +#define ORC_REG_UNDEFINED 0 +#define ORC_REG_PREV_SP 1 +#define ORC_REG_SP 2 +#define ORC_REG_FP 3 +#define ORC_REG_MAX 4 + +#endif /* _ORC_TYPES_H */ diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile index 92583b82eb78..14bb324d9385 100644 --- a/tools/objtool/Makefile +++ b/tools/objtool/Makefile @@ -47,6 +47,7 @@ ifeq ($(SRCARCH),x86) endif =20 ifeq ($(SRCARCH),arm64) + BUILD_ORC :=3D y DYNAMIC_CHECK :=3D y endif =20 diff --git a/tools/objtool/arch/arm64/Build b/tools/objtool/arch/arm64/Build index 3ff1f00c6a47..8615abfb12cf 100644 --- a/tools/objtool/arch/arm64/Build +++ b/tools/objtool/arch/arm64/Build @@ -1 +1,2 @@ objtool-y +=3D decode.o +objtool-y +=3D orc.o diff --git a/tools/objtool/arch/arm64/include/arch/elf.h b/tools/objtool/ar= ch/arm64/include/arch/elf.h new file mode 100644 index 000000000000..4ae6df2bd90c --- /dev/null +++ b/tools/objtool/arch/arm64/include/arch/elf.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 */ + +#ifndef _OBJTOOL_ARCH_ELF +#define _OBJTOOL_ARCH_ELF + +#define R_NONE R_AARCH64_NONE +#define R_PCREL R_AARCH64_PREL32 + +#endif /* _OBJTOOL_ARCH_ELF */ diff --git a/tools/objtool/arch/arm64/orc.c b/tools/objtool/arch/arm64/orc.c new file mode 100644 index 000000000000..cef14114e1ec --- /dev/null +++ b/tools/objtool/arch/arm64/orc.c @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ +#include + +#include + +#include +#include + +int init_orc_entry(struct orc_entry *orc, struct cfi_state *cfi, + struct instruction *insn) +{ + struct cfi_reg *fp =3D &cfi->regs[CFI_FP]; + + memset(orc, 0, sizeof(*orc)); + + orc->sp_reg =3D ORC_REG_SP; + orc->fp_reg =3D ORC_REG_PREV_SP; + + if (!cfi || cfi->cfa.base =3D=3D CFI_UNDEFINED || + (cfi->type =3D=3D UNWIND_HINT_TYPE_CALL && !fp->offset)) { + /* + * The frame pointer has not been set up. This instruction is + * unreliable from an unwind perspective. + */ + return 0; + } + + orc->sp_offset =3D cfi->cfa.offset; + orc->fp_offset =3D fp->offset; + orc->type =3D cfi->type; + orc->end =3D cfi->end; + + return 0; +} + +static const char *reg_name(unsigned int reg) +{ + switch (reg) { + case ORC_REG_PREV_SP: + return "cfa"; + case ORC_REG_FP: + return "x29"; + case ORC_REG_SP: + return "sp"; + default: + return "?"; + } +} + +const char *orc_type_name(unsigned int type) +{ + switch (type) { + case UNWIND_HINT_TYPE_CALL: + return "call"; + default: + return "?"; + } +} + +void orc_print_reg(unsigned int reg, int offset) +{ + if (reg =3D=3D ORC_REG_UNDEFINED) + printf("(und)"); + else + printf("%s%+d", reg_name(reg), offset); +} + +void orc_print_sp(void) +{ + printf(" cfa:"); +} + +void orc_print_fp(void) +{ + printf(" x29:"); +} + +bool orc_ignore_section(struct section *sec) +{ + return !strcmp(sec->name, ".head.text"); +} diff --git a/tools/objtool/dcheck.c b/tools/objtool/dcheck.c index 8b78cb608528..57499752c523 100644 --- a/tools/objtool/dcheck.c +++ b/tools/objtool/dcheck.c @@ -349,5 +349,8 @@ int check(struct objtool_file *file) =20 walk_sections(file); =20 - return 0; + if (opts.orc) + ret =3D orc_create(file); + + return ret; } diff --git a/tools/objtool/include/objtool/insn.h b/tools/objtool/include/o= bjtool/insn.h index 3a43a591b318..ac718f1e2d2f 100644 --- a/tools/objtool/include/objtool/insn.h +++ b/tools/objtool/include/objtool/insn.h @@ -84,6 +84,7 @@ struct instruction *next_insn_same_sec(struct objtool_fil= e *file, struct instruction *next_insn_same_func(struct objtool_file *file, struct instruction *insn); struct reloc *insn_reloc(struct objtool_file *file, struct instruction *in= sn); +bool insn_can_reloc(struct instruction *insn); bool insn_cfi_match(struct instruction *insn, struct cfi_state *cfi2, bool print); bool same_function(struct instruction *insn1, struct instruction *insn2); diff --git a/tools/objtool/include/objtool/objtool.h b/tools/objtool/includ= e/objtool/objtool.h index 7f2d1b095333..b7655ad3e402 100644 --- a/tools/objtool/include/objtool/objtool.h +++ b/tools/objtool/include/objtool/objtool.h @@ -46,5 +46,6 @@ void objtool_pv_add(struct objtool_file *file, int idx, s= truct symbol *func); int check(struct objtool_file *file); int orc_dump(const char *objname); int orc_create(struct objtool_file *file); +bool orc_ignore_section(struct section *sec); =20 #endif /* _OBJTOOL_H */ diff --git a/tools/objtool/insn.c b/tools/objtool/insn.c index be3617d55aea..af48319f2225 100644 --- a/tools/objtool/insn.c +++ b/tools/objtool/insn.c @@ -193,3 +193,23 @@ bool insn_cfi_match(struct instruction *insn, struct c= fi_state *cfi2, =20 return false; } + +/* + * This is a hack for Clang. Clang is aggressive about removing section + * symbols and then some. If we cannot find something to relocate an + * instruction against, we must not generate CFI for it or the ORC + * generation will fail later. + */ +bool insn_can_reloc(struct instruction *insn) +{ + struct section *insn_sec =3D insn->sec; + unsigned long insn_off =3D insn->offset; + + if (insn_sec->sym || + find_symbol_containing(insn_sec, insn_off) || + find_symbol_containing(insn_sec, insn_off - 1)) { + /* See elf_add_reloc_to_insn(). */ + return true; + } + return false; +} diff --git a/tools/objtool/orc_gen.c b/tools/objtool/orc_gen.c index ea2e361ff7bc..bddf5889466f 100644 --- a/tools/objtool/orc_gen.c +++ b/tools/objtool/orc_gen.c @@ -14,6 +14,11 @@ #include #include =20 +bool __weak orc_ignore_section(struct section *sec) +{ + return false; +} + static int write_orc_entry(struct elf *elf, struct section *orc_sec, struct section *ip_sec, unsigned int idx, struct section *insn_sec, unsigned long insn_off, @@ -87,13 +92,16 @@ int orc_create(struct objtool_file *file) struct instruction *insn; bool empty =3D true; =20 - if (!sec->text) + if (!sec->text || orc_ignore_section(sec)) continue; =20 sec_for_each_insn(file, sec, insn) { struct alt_group *alt_group =3D insn->alt_group; int i; =20 + if (!insn_can_reloc(insn)) + continue; + if (!alt_group) { if (init_orc_entry(&orc, insn->cfi, insn)) return -1; @@ -137,7 +145,7 @@ int orc_create(struct objtool_file *file) } =20 /* Add a section terminator */ - if (!empty) { + if (!empty && sec->sym) { orc_list_add(&orc_list, &null, sec, sec->sh.sh_size); nr++; } diff --git a/tools/objtool/sync-check.sh b/tools/objtool/sync-check.sh index ef1acb064605..0d0656f6ce4a 100755 --- a/tools/objtool/sync-check.sh +++ b/tools/objtool/sync-check.sh @@ -29,6 +29,13 @@ arch/x86/lib/insn.c ' fi =20 +if [ "$SRCARCH" =3D "arm64" ]; then +FILES=3D"$FILES +arch/arm64/include/asm/orc_types.h +include/linux/orc_entry.h +" +fi + check_2 () { file1=3D$1 file2=3D$2 --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26F94C61DA4 for ; Thu, 2 Feb 2023 07:43:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232147AbjBBHn2 (ORCPT ); Thu, 2 Feb 2023 02:43:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232058AbjBBHm5 (ORCPT ); Thu, 2 Feb 2023 02:42:57 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id BD2E684B5D; Wed, 1 Feb 2023 23:42:40 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id A0C5E20B2EE1; Wed, 1 Feb 2023 23:42:39 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A0C5E20B2EE1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323760; bh=y3qsADNio9+dY4UdeX2rS9vahgQ8I1RdPk9wpVWAKKk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=QbCigbfUQUdYrfpoaBfqtidMu56nuRH8VwMDzxOyNicKedC4AWx9LpldEKyQxKxYE mrudZPVymG6n0BhuyUinr0jgeJ1ftU9C8w3mCRm4yOaWtLQR6/rfCt+hB9JiQclgrU OTmcCV9iRRFgQ6KhBHfjiw3/FbGTxWUucTHWyd7U= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 15/22] objtool: arm64: Add unwind hint support Date: Thu, 2 Feb 2023 01:40:29 -0600 Message-Id: <20230202074036.507249-16-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Implement the unwind hint macros for ARM64. Define the unwind hint types as well. Process the unwind hints section for dynamic FP validation for ARM64. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/unwind_hints.h | 104 ++++++++++++++++++++ include/linux/objtool.h | 3 + tools/arch/arm64/include/asm/unwind_hints.h | 104 ++++++++++++++++++++ tools/include/linux/objtool.h | 3 + tools/objtool/Build | 2 +- tools/objtool/arch/arm64/decode.c | 21 ++++ tools/objtool/arch/arm64/orc.c | 4 + tools/objtool/dcheck.c | 4 + tools/objtool/include/objtool/endianness.h | 1 + tools/objtool/sync-check.sh | 1 + tools/objtool/unwind_hints.c | 24 +++-- 11 files changed, 260 insertions(+), 11 deletions(-) create mode 100644 arch/arm64/include/asm/unwind_hints.h create mode 100644 tools/arch/arm64/include/asm/unwind_hints.h diff --git a/arch/arm64/include/asm/unwind_hints.h b/arch/arm64/include/asm= /unwind_hints.h new file mode 100644 index 000000000000..fb1b924d85bc --- /dev/null +++ b/arch/arm64/include/asm/unwind_hints.h @@ -0,0 +1,104 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _ASM_ARM64_UNWIND_HINTS_H +#define _ASM_ARM64_UNWIND_HINTS_H + +#ifndef __ASSEMBLY__ + +#include + +/* + * This struct is used by asm and inline asm code to manually annotate the + * CFI for an instruction. We have to use s16 instead of s8 for some of th= ese + * fields as 8-bit fields are not relocated by some assemblers. + */ +struct unwind_hint { + u32 ip; + s16 sp_offset; + s16 sp_reg; + s16 type; + s16 end; +}; + +#endif + +#include + +#include "orc_types.h" + +#ifdef CONFIG_STACK_VALIDATION + +#ifndef __ASSEMBLY__ + +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "987: \n\t" \ + ".pushsection .discard.unwind_hints\n\t" \ + /* struct unwind_hint */ \ + ".long 987b - .\n\t" \ + ".short " __stringify(sp_offset) "\n\t" \ + ".short " __stringify(sp_reg) "\n\t" \ + ".short " __stringify(type) "\n\t" \ + ".short " __stringify(end) "\n\t" \ + ".popsection\n\t" + +#else /* __ASSEMBLY__ */ + +/* + * There are points in ASM code where it is useful to unwind through even + * though the ASM code itself may be unreliable from an unwind perspective. + * E.g., interrupt and exception handlers. + * + * These macros provide hints to objtool to compute the CFI information at + * such instructions. + */ +.macro UNWIND_HINT sp_reg:req sp_offset=3D0 type:req end=3D0 +.Lunwind_hint_pc_\@: + .pushsection .discard.unwind_hints + /* struct unwind_hint */ + .long .Lunwind_hint_pc_\@ - . + .short \sp_offset + .short \sp_reg + .short \type + .short \end + .popsection +.endm + +#endif /* __ASSEMBLY__ */ + +#else /* !CONFIG_STACK_VALIDATION */ + +#ifndef __ASSEMBLY__ + +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "\n\t" +#else +.macro UNWIND_HINT sp_reg:req sp_offset=3D0 type:req end=3D0 +.endm +#endif + +#endif /* CONFIG_STACK_VALIDATION */ +#ifdef __ASSEMBLY__ + +.macro UNWIND_HINT_FTRACE, offset + .set sp_reg, ORC_REG_SP + .set sp_offset, \offset + .set type, UNWIND_HINT_TYPE_FTRACE + UNWIND_HINT sp_reg=3Dsp_reg sp_offset=3Dsp_offset type=3Dtype +.endm + +.macro UNWIND_HINT_REGS, offset + .set sp_reg, ORC_REG_SP + .set sp_offset, \offset + .set type, UNWIND_HINT_TYPE_REGS + UNWIND_HINT sp_reg=3Dsp_reg sp_offset=3Dsp_offset type=3Dtype +.endm + +.macro UNWIND_HINT_IRQ, offset + .set sp_reg, ORC_REG_SP + .set sp_offset, \offset + .set type, UNWIND_HINT_TYPE_IRQ_STACK + UNWIND_HINT sp_reg=3Dsp_reg sp_offset=3Dsp_offset type=3Dtype +.endm + +#endif /* __ASSEMBLY__ */ + +#endif /* _ASM_ARM64_UNWIND_HINTS_H */ diff --git a/include/linux/objtool.h b/include/linux/objtool.h index 1af295efc12c..dcbd365944f6 100644 --- a/include/linux/objtool.h +++ b/include/linux/objtool.h @@ -17,6 +17,8 @@ * Useful for code which doesn't have an ELF function annotation. * * UNWIND_HINT_ENTRY: machine entry without stack, SYSCALL/SYSENTER etc. + * + * UNWIND_HINT_TYPE_IRQ_STACK: Used to unwind through the IRQ stack. */ #define UNWIND_HINT_TYPE_CALL 0 #define UNWIND_HINT_TYPE_REGS 1 @@ -25,6 +27,7 @@ #define UNWIND_HINT_TYPE_ENTRY 4 #define UNWIND_HINT_TYPE_SAVE 5 #define UNWIND_HINT_TYPE_RESTORE 6 +#define UNWIND_HINT_TYPE_IRQ_STACK 7 =20 #ifdef CONFIG_OBJTOOL =20 diff --git a/tools/arch/arm64/include/asm/unwind_hints.h b/tools/arch/arm64= /include/asm/unwind_hints.h new file mode 100644 index 000000000000..fb1b924d85bc --- /dev/null +++ b/tools/arch/arm64/include/asm/unwind_hints.h @@ -0,0 +1,104 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _ASM_ARM64_UNWIND_HINTS_H +#define _ASM_ARM64_UNWIND_HINTS_H + +#ifndef __ASSEMBLY__ + +#include + +/* + * This struct is used by asm and inline asm code to manually annotate the + * CFI for an instruction. We have to use s16 instead of s8 for some of th= ese + * fields as 8-bit fields are not relocated by some assemblers. + */ +struct unwind_hint { + u32 ip; + s16 sp_offset; + s16 sp_reg; + s16 type; + s16 end; +}; + +#endif + +#include + +#include "orc_types.h" + +#ifdef CONFIG_STACK_VALIDATION + +#ifndef __ASSEMBLY__ + +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "987: \n\t" \ + ".pushsection .discard.unwind_hints\n\t" \ + /* struct unwind_hint */ \ + ".long 987b - .\n\t" \ + ".short " __stringify(sp_offset) "\n\t" \ + ".short " __stringify(sp_reg) "\n\t" \ + ".short " __stringify(type) "\n\t" \ + ".short " __stringify(end) "\n\t" \ + ".popsection\n\t" + +#else /* __ASSEMBLY__ */ + +/* + * There are points in ASM code where it is useful to unwind through even + * though the ASM code itself may be unreliable from an unwind perspective. + * E.g., interrupt and exception handlers. + * + * These macros provide hints to objtool to compute the CFI information at + * such instructions. + */ +.macro UNWIND_HINT sp_reg:req sp_offset=3D0 type:req end=3D0 +.Lunwind_hint_pc_\@: + .pushsection .discard.unwind_hints + /* struct unwind_hint */ + .long .Lunwind_hint_pc_\@ - . + .short \sp_offset + .short \sp_reg + .short \type + .short \end + .popsection +.endm + +#endif /* __ASSEMBLY__ */ + +#else /* !CONFIG_STACK_VALIDATION */ + +#ifndef __ASSEMBLY__ + +#define UNWIND_HINT(sp_reg, sp_offset, type, end) \ + "\n\t" +#else +.macro UNWIND_HINT sp_reg:req sp_offset=3D0 type:req end=3D0 +.endm +#endif + +#endif /* CONFIG_STACK_VALIDATION */ +#ifdef __ASSEMBLY__ + +.macro UNWIND_HINT_FTRACE, offset + .set sp_reg, ORC_REG_SP + .set sp_offset, \offset + .set type, UNWIND_HINT_TYPE_FTRACE + UNWIND_HINT sp_reg=3Dsp_reg sp_offset=3Dsp_offset type=3Dtype +.endm + +.macro UNWIND_HINT_REGS, offset + .set sp_reg, ORC_REG_SP + .set sp_offset, \offset + .set type, UNWIND_HINT_TYPE_REGS + UNWIND_HINT sp_reg=3Dsp_reg sp_offset=3Dsp_offset type=3Dtype +.endm + +.macro UNWIND_HINT_IRQ, offset + .set sp_reg, ORC_REG_SP + .set sp_offset, \offset + .set type, UNWIND_HINT_TYPE_IRQ_STACK + UNWIND_HINT sp_reg=3Dsp_reg sp_offset=3Dsp_offset type=3Dtype +.endm + +#endif /* __ASSEMBLY__ */ + +#endif /* _ASM_ARM64_UNWIND_HINTS_H */ diff --git a/tools/include/linux/objtool.h b/tools/include/linux/objtool.h index 1af295efc12c..dcbd365944f6 100644 --- a/tools/include/linux/objtool.h +++ b/tools/include/linux/objtool.h @@ -17,6 +17,8 @@ * Useful for code which doesn't have an ELF function annotation. * * UNWIND_HINT_ENTRY: machine entry without stack, SYSCALL/SYSENTER etc. + * + * UNWIND_HINT_TYPE_IRQ_STACK: Used to unwind through the IRQ stack. */ #define UNWIND_HINT_TYPE_CALL 0 #define UNWIND_HINT_TYPE_REGS 1 @@ -25,6 +27,7 @@ #define UNWIND_HINT_TYPE_ENTRY 4 #define UNWIND_HINT_TYPE_SAVE 5 #define UNWIND_HINT_TYPE_RESTORE 6 +#define UNWIND_HINT_TYPE_IRQ_STACK 7 =20 #ifdef CONFIG_OBJTOOL =20 diff --git a/tools/objtool/Build b/tools/objtool/Build index fb0846b7d95e..2780e402babb 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -9,7 +9,7 @@ objtool-y +=3D builtin-check.o objtool-y +=3D cfi.o objtool-y +=3D insn.o objtool-y +=3D decode.o -objtool-$(STATIC_CHECK) +=3D unwind_hints.o +objtool-y +=3D unwind_hints.o objtool-y +=3D elf.o objtool-y +=3D objtool.o =20 diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/d= ecode.c index f723be80c09a..570069ac68ae 100644 --- a/tools/objtool/arch/arm64/decode.c +++ b/tools/objtool/arch/arm64/decode.c @@ -17,6 +17,8 @@ #include #include =20 +#include + /* ARM64 instructions are all 4 bytes wide. */ #define INSN_SIZE 4 =20 @@ -47,6 +49,25 @@ unsigned long arch_jump_destination(struct instruction *= insn) return insn->offset + insn->immediate; } =20 +int arch_decode_hint_reg(u8 sp_reg, int *base) +{ + switch (sp_reg) { + case ORC_REG_UNDEFINED: + *base =3D CFI_UNDEFINED; + break; + case ORC_REG_SP: + *base =3D CFI_SP; + break; + case ORC_REG_FP: + *base =3D CFI_FP; + break; + default: + return -1; + } + + return 0; +} + /* --------------------- instruction decode structs ----------------------= -- */ =20 struct decode_var { diff --git a/tools/objtool/arch/arm64/orc.c b/tools/objtool/arch/arm64/orc.c index cef14114e1ec..5b155585258a 100644 --- a/tools/objtool/arch/arm64/orc.c +++ b/tools/objtool/arch/arm64/orc.c @@ -57,6 +57,10 @@ const char *orc_type_name(unsigned int type) switch (type) { case UNWIND_HINT_TYPE_CALL: return "call"; + case UNWIND_HINT_TYPE_REGS: + return "regs"; + case UNWIND_HINT_TYPE_IRQ_STACK: + return "irqstack"; default: return "?"; } diff --git a/tools/objtool/dcheck.c b/tools/objtool/dcheck.c index 57499752c523..567f492b0e3e 100644 --- a/tools/objtool/dcheck.c +++ b/tools/objtool/dcheck.c @@ -349,6 +349,10 @@ int check(struct objtool_file *file) =20 walk_sections(file); =20 + ret =3D read_unwind_hints(file); + if (ret) + return ret; + if (opts.orc) ret =3D orc_create(file); =20 diff --git a/tools/objtool/include/objtool/endianness.h b/tools/objtool/inc= lude/objtool/endianness.h index 10241341eff3..9a53ab421a19 100644 --- a/tools/objtool/include/objtool/endianness.h +++ b/tools/objtool/include/objtool/endianness.h @@ -29,6 +29,7 @@ case 8: __ret =3D __NEED_BSWAP ? bswap_64(val) : (val); break; \ case 4: __ret =3D __NEED_BSWAP ? bswap_32(val) : (val); break; \ case 2: __ret =3D __NEED_BSWAP ? bswap_16(val) : (val); break; \ + case 1: __ret =3D (val); break; \ default: \ BUILD_BUG(); break; \ } \ diff --git a/tools/objtool/sync-check.sh b/tools/objtool/sync-check.sh index 0d0656f6ce4a..3742d1e2585c 100755 --- a/tools/objtool/sync-check.sh +++ b/tools/objtool/sync-check.sh @@ -31,6 +31,7 @@ fi =20 if [ "$SRCARCH" =3D "arm64" ]; then FILES=3D"$FILES +arch/arm64/include/asm/unwind_hints.h arch/arm64/include/asm/orc_types.h include/linux/orc_entry.h " diff --git a/tools/objtool/unwind_hints.c b/tools/objtool/unwind_hints.c index f2521659bae5..c51013c5d0b3 100644 --- a/tools/objtool/unwind_hints.c +++ b/tools/objtool/unwind_hints.c @@ -16,6 +16,7 @@ int read_unwind_hints(struct objtool_file *file) struct unwind_hint *hint; struct instruction *insn; struct reloc *reloc; + u8 sp_reg, type; int i; =20 sec =3D find_section_by_name(file->elf, ".discard.unwind_hints"); @@ -38,6 +39,9 @@ int read_unwind_hints(struct objtool_file *file) for (i =3D 0; i < sec->sh.sh_size / sizeof(struct unwind_hint); i++) { hint =3D (struct unwind_hint *)sec->data->d_buf + i; =20 + sp_reg =3D bswap_if_needed(hint->sp_reg); + type =3D bswap_if_needed(hint->type); + reloc =3D find_reloc_by_dest(file->elf, sec, i * sizeof(*hint)); if (!reloc) { WARN("can't find reloc for unwind_hints[%d]", i); @@ -52,18 +56,18 @@ int read_unwind_hints(struct objtool_file *file) =20 insn->hint =3D true; =20 - if (hint->type =3D=3D UNWIND_HINT_TYPE_SAVE) { + if (type =3D=3D UNWIND_HINT_TYPE_SAVE) { insn->hint =3D false; insn->save =3D true; continue; } =20 - if (hint->type =3D=3D UNWIND_HINT_TYPE_RESTORE) { + if (type =3D=3D UNWIND_HINT_TYPE_RESTORE) { insn->restore =3D true; continue; } =20 - if (hint->type =3D=3D UNWIND_HINT_TYPE_REGS_PARTIAL) { + if (type =3D=3D UNWIND_HINT_TYPE_REGS_PARTIAL) { struct symbol *sym =3D find_symbol_by_offset(insn->sec, insn->offset); =20 if (sym && sym->bind =3D=3D STB_GLOBAL) { @@ -76,12 +80,12 @@ int read_unwind_hints(struct objtool_file *file) } } =20 - if (hint->type =3D=3D UNWIND_HINT_TYPE_ENTRY) { - hint->type =3D UNWIND_HINT_TYPE_CALL; + if (type =3D=3D UNWIND_HINT_TYPE_ENTRY) { + type =3D UNWIND_HINT_TYPE_CALL; insn->entry =3D 1; } =20 - if (hint->type =3D=3D UNWIND_HINT_TYPE_FUNC) { + if (type =3D=3D UNWIND_HINT_TYPE_FUNC) { insn->cfi =3D &func_cfi; continue; } @@ -89,15 +93,15 @@ int read_unwind_hints(struct objtool_file *file) if (insn->cfi) cfi =3D *(insn->cfi); =20 - if (arch_decode_hint_reg(hint->sp_reg, &cfi.cfa.base)) { + if (arch_decode_hint_reg(sp_reg, &cfi.cfa.base)) { WARN_FUNC("unsupported unwind_hint sp base reg %d", - insn->sec, insn->offset, hint->sp_reg); + insn->sec, insn->offset, sp_reg); return -1; } =20 cfi.cfa.offset =3D bswap_if_needed(hint->sp_offset); - cfi.type =3D hint->type; - cfi.end =3D hint->end; + cfi.type =3D type; + cfi.end =3D bswap_if_needed(hint->end); =20 insn->cfi =3D cfi_hash_find_or_add(&cfi); } --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 820FEC61DA4 for ; Thu, 2 Feb 2023 07:43:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232176AbjBBHnc (ORCPT ); Thu, 2 Feb 2023 02:43:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232075AbjBBHnD (ORCPT ); Thu, 2 Feb 2023 02:43:03 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id BC5E78418F; Wed, 1 Feb 2023 23:42:41 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id 9DD1420B2EE2; Wed, 1 Feb 2023 23:42:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9DD1420B2EE2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323761; bh=TU6iA6jKusJnO70H+H53EmwcF0iJW0CBgTtKff3JI6k=; h=From:To:Subject:Date:In-Reply-To:References:From; b=QC4nujwZtIv/QBJ9ymn/CgxJwUDnI+3uphc5T+Gu1JhzZ8DsZDrdFS6ekuXTYImNI 3hhJvzVlkgZtNG3y+q0Z+Er2prT0XsuyEtZPzyM7Y095qVEUMRdK9OZBd92AZdLUn7 ozFI8y1kgFhsn1M6zu0K6KGUmH7ot/1xROXsuhlg= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 16/22] arm64: Add unwind hints to exception handlers Date: Thu, 2 Feb 2023 01:40:30 -0600 Message-Id: <20230202074036.507249-17-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Add unwind hints to Interrupt and Exception handlers. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/kernel/entry.S | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index e28137d64b76..d73bed56f0e6 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -28,6 +28,7 @@ #include #include #include +#include =20 .macro clear_gp_regs .irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,2= 5,26,27,28,29 @@ -560,6 +561,7 @@ SYM_CODE_START_LOCAL(el\el\ht\()_\regsize\()_\label) .if \el =3D=3D 0 b ret_to_user .else + UNWIND_HINT_REGS PT_REGS_SIZE b ret_to_kernel .endif SYM_CODE_END(el\el\ht\()_\regsize\()_\label) @@ -887,6 +889,7 @@ SYM_FUNC_START(call_on_irq_stack) /* Move to the new stack and call the function there */ mov sp, x16 blr x1 + UNWIND_HINT_IRQ 16 =20 /* * Restore the SP from the FP, and restore the FP and LR from the frame --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B5EEC05027 for ; Thu, 2 Feb 2023 07:43:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231947AbjBBHnl (ORCPT ); Thu, 2 Feb 2023 02:43:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232034AbjBBHnR (ORCPT ); Thu, 2 Feb 2023 02:43:17 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C5DAB757BC; Wed, 1 Feb 2023 23:42:42 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id 9BD7020B2EE3; Wed, 1 Feb 2023 23:42:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9BD7020B2EE3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323762; bh=ja7iCXZAYSBOw6CRsMiBOuCjJ6dS1/OpL4Jd5Usm8GU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=AxPvm8rrHwawA3/KKIY7ZxCMnGqba6n07XjZ9NbXNZbZs3YZAyo4jdIfvP/Z9+TaN NQljocc3i94+iHdRznNOca41Vm/vOTmQeQ95hKo9MIe3KwdU5c9dBEDoUOyRkyXXFr eDWGS+0t8/WS7HFhWMHvZRSRkBDpcoLMdCCRk1u4= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 17/22] arm64: Add kernel and module support for ORC Date: Thu, 2 Feb 2023 01:40:31 -0600 Message-Id: <20230202074036.507249-18-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Call orc_lookup_init() from setup_arch() to perform ORC lookup initialization for vmlinux. Call orc_lookup_module_init() in module load to perform ORC lookup initialization for modules. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/kernel/module.c | 13 ++++++++++++- arch/arm64/kernel/setup.c | 2 ++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 76b41e4ca9fa..71264a181f61 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -19,6 +19,7 @@ #include #include #include +#include =20 void *module_alloc(unsigned long size) { @@ -509,10 +510,20 @@ int module_finalize(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, struct module *me) { - const Elf_Shdr *s; + const Elf_Shdr *s, *orc, *orc_ip; + s =3D find_section(hdr, sechdrs, ".altinstructions"); if (s) apply_alternatives_module((void *)s->sh_addr, s->sh_size); =20 + orc =3D find_section(hdr, sechdrs, ".orc_unwind"); + orc_ip =3D find_section(hdr, sechdrs, ".orc_unwind_ip"); + + if (orc && orc_ip) { + orc_lookup_module_init(me, + (void *)orc_ip->sh_addr, orc_ip->sh_size, + (void *)orc->sh_addr, orc->sh_size); + } + return module_init_ftrace_plt(hdr, sechdrs, me); } diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index fea3223704b6..360304dcd8c2 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -51,6 +51,7 @@ #include #include #include +#include =20 static int num_standard_resources; static struct resource *standard_resources; @@ -378,6 +379,7 @@ void __init __no_sanitize_address setup_arch(char **cmd= line_p) "This indicates a broken bootloader or old kernel\n", boot_args[1], boot_args[2], boot_args[3]); } + orc_lookup_init(); } =20 static inline bool cpu_can_disable(unsigned int cpu) --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FA88C61DA4 for ; Thu, 2 Feb 2023 07:43:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230456AbjBBHno (ORCPT ); Thu, 2 Feb 2023 02:43:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232111AbjBBHnR (ORCPT ); Thu, 2 Feb 2023 02:43:17 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A1E0E84B59; Wed, 1 Feb 2023 23:42:43 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id 99EFB20B2EE4; Wed, 1 Feb 2023 23:42:42 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 99EFB20B2EE4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323763; bh=CDc7R+oUQPek3aA1dFgOorPmCowQj3RJsL4UXFb2DgY=; h=From:To:Subject:Date:In-Reply-To:References:From; b=goYLJgwf1YQT2czqCuodcUPHAsKPIdaqYhBPR9ytjVVj8gaYPMeDhCVeBL8t2owJr JV228E1OZrBX/eS1A9h4wSLkcoKdieF6ov+xakMKTyBr36GEj/40Xj6Zl62UTYi5DJ UYLoQukcEM4pHnPt2pQSqW8ZdBOWuMxcl7Plm5AU= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 18/22] arm64: Build the kernel with ORC information Date: Thu, 2 Feb 2023 01:40:32 -0600 Message-Id: <20230202074036.507249-19-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Add code to scripts/Makefile.lib to define objtool options to generate ORC data for frame pointer validation. Define kernel configs: - to enable dynamic FRAME_POINTER_VALIDATION - to enable the generation of ORC data using objtool When these configs are enabled, objtool is invoked on relocatable files during kernel build with the following command: objtool --stackval --orc Objtool creates special sections in the object files: .orc_unwind_ip PC array. .orc_unwind ORC structure table. .orc_lookup ORC lookup table. Change arch/arm64/kernel/vmlinux.lds.S to include ORC_UNWIND_TABLE in the data section so that the special sections get included there. For modules, these sections will be added to the kernel during module load. In the future, the kernel can use these sections to find the ORC for a given instruction address. The unwinder can then compute the FP at an instruction address and validate the actual FP with that. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/Kconfig | 2 ++ arch/arm64/Kconfig.debug | 32 ++++++++++++++++++++++++++++++++ arch/arm64/include/asm/module.h | 12 +++++++++++- arch/arm64/kernel/vmlinux.lds.S | 3 +++ include/linux/objtool.h | 2 ++ scripts/Makefile | 4 +++- scripts/Makefile.lib | 9 +++++++++ tools/include/linux/objtool.h | 2 ++ 8 files changed, 64 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 505c8a1ccbe0..73c3f30a37c7 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -230,6 +230,8 @@ config ARM64 select TRACE_IRQFLAGS_SUPPORT select TRACE_IRQFLAGS_NMI_SUPPORT select HAVE_SOFTIRQ_ON_OWN_STACK + select HAVE_STACK_VALIDATION if FRAME_POINTER_VALIDATION + select STACK_VALIDATION if HAVE_STACK_VALIDATION help ARM 64-bit (AArch64) Linux support. =20 diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug index 265c4461031f..a50caabdb18e 100644 --- a/arch/arm64/Kconfig.debug +++ b/arch/arm64/Kconfig.debug @@ -20,4 +20,36 @@ config ARM64_RELOC_TEST depends on m tristate "Relocation testing module" =20 +config UNWINDER_ORC + bool "ORC unwinder" + depends on FRAME_POINTER_VALIDATION + select HAVE_MOD_ARCH_SPECIFIC + select OBJTOOL + help + This option enables ORC (Oops Rewind Capability) for ARM64. This + allows the unwinder to look up ORC data for an instruction address + and compute the frame pointer at that address. The computed frame + pointer is used to validate the actual frame pointer. + +config UNWINDER_FRAME_POINTER + bool "Frame pointer unwinder" + depends on FRAME_POINTER_VALIDATION + select FRAME_POINTER + help + ARM64 already uses the frame pointer for unwinding kernel stack + traces. We need to enable this config to enable STACK_VALIDATION. + STACK_VALIDATION is needed to get objtool to do static analysis + of kernel code. + +config FRAME_POINTER_VALIDATION + bool "Dynamic Frame pointer validation" + select UNWINDER_FRAME_POINTER + select UNWINDER_ORC + help + This invokes objtool on every object file causing it to + generate ORC data for the object file. ORC data is in a custom + data format which is a simplified version of the DWARF + Call Frame Information standard. See UNWINDER_ORC for more + details. + source "drivers/hwtracing/coresight/Kconfig" diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/modul= e.h index 18734fed3bdd..4362f44aae61 100644 --- a/arch/arm64/include/asm/module.h +++ b/arch/arm64/include/asm/module.h @@ -6,6 +6,7 @@ #define __ASM_MODULE_H =20 #include +#include =20 #ifdef CONFIG_ARM64_MODULE_PLTS struct mod_plt_sec { @@ -13,15 +14,24 @@ struct mod_plt_sec { int plt_num_entries; int plt_max_entries; }; +#endif =20 +#ifdef CONFIG_HAVE_MOD_ARCH_SPECIFIC struct mod_arch_specific { +#ifdef CONFIG_ARM64_MODULE_PLTS struct mod_plt_sec core; struct mod_plt_sec init; =20 /* for CONFIG_DYNAMIC_FTRACE */ struct plt_entry *ftrace_trampolines; -}; #endif +#ifdef CONFIG_UNWINDER_ORC + unsigned int num_orcs; + int *orc_unwind_ip; + struct orc_entry *orc_unwind; +#endif +}; +#endif /* CONFIG_HAVE_MOD_ARCH_SPECIFIC */ =20 u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs, void *loc, const Elf64_Rela *rela, diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 45131e354e27..bf7b55ae10ee 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -61,6 +61,7 @@ #define RUNTIME_DISCARD_EXIT =20 #include +#include #include #include #include @@ -294,6 +295,8 @@ SECTIONS __mmuoff_data_end =3D .; } =20 + ORC_UNWIND_TABLE + PECOFF_EDATA_PADDING __pecoff_data_rawsize =3D ABSOLUTE(. - __initdata_begin); _edata =3D .; diff --git a/include/linux/objtool.h b/include/linux/objtool.h index dcbd365944f6..c980522190f7 100644 --- a/include/linux/objtool.h +++ b/include/linux/objtool.h @@ -31,7 +31,9 @@ =20 #ifdef CONFIG_OBJTOOL =20 +#ifndef CONFIG_ARM64 #include +#endif =20 #ifndef __ASSEMBLY__ =20 diff --git a/scripts/Makefile b/scripts/Makefile index 1575af84d557..df3e4d90f195 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -23,8 +23,10 @@ HOSTLDLIBS_sign-file =3D $(shell $(HOSTPKG_CONFIG) --lib= s libcrypto 2> /dev/null | ifdef CONFIG_UNWINDER_ORC ifeq ($(ARCH),x86_64) ARCH :=3D x86 -endif HOSTCFLAGS_sorttable.o +=3D -I$(srctree)/tools/arch/x86/include +else +HOSTCFLAGS_sorttable.o +=3D -I$(srctree)/tools/arch/$(ARCH)/include +endif HOSTCFLAGS_sorttable.o +=3D -DUNWINDER_ORC_ENABLED endif =20 diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 3aa384cec76b..d364871a1046 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -252,6 +252,13 @@ ifdef CONFIG_OBJTOOL =20 objtool :=3D $(objtree)/tools/objtool/objtool =20 +ifdef CONFIG_FRAME_POINTER_VALIDATION + +objtool-args-$(CONFIG_STACK_VALIDATION) +=3D --stackval +objtool-args-$(CONFIG_UNWINDER_ORC) +=3D --orc + +else + objtool-args-$(CONFIG_HAVE_JUMP_LABEL_HACK) +=3D --hacks=3Djump_label objtool-args-$(CONFIG_HAVE_NOINSTR_HACK) +=3D --hacks=3Dnoinstr objtool-args-$(CONFIG_X86_KERNEL_IBT) +=3D --ibt @@ -265,6 +272,8 @@ objtool-args-$(CONFIG_HAVE_STATIC_CALL_INLINE) +=3D --= static-call objtool-args-$(CONFIG_HAVE_UACCESS_VALIDATION) +=3D --uaccess objtool-args-$(CONFIG_GCOV_KERNEL) +=3D --no-unreachable =20 +endif + objtool-args =3D $(objtool-args-y) \ $(if $(delay-objtool), --link) \ $(if $(part-of-module), --module) diff --git a/tools/include/linux/objtool.h b/tools/include/linux/objtool.h index dcbd365944f6..c980522190f7 100644 --- a/tools/include/linux/objtool.h +++ b/tools/include/linux/objtool.h @@ -31,7 +31,9 @@ =20 #ifdef CONFIG_OBJTOOL =20 +#ifndef CONFIG_ARM64 #include +#endif =20 #ifndef __ASSEMBLY__ =20 --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78D59C636D4 for ; Thu, 2 Feb 2023 07:43:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232144AbjBBHnr (ORCPT ); Thu, 2 Feb 2023 02:43:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232119AbjBBHnS (ORCPT ); Thu, 2 Feb 2023 02:43:18 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 894DF84B41; Wed, 1 Feb 2023 23:42:44 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id 96F9F2086204; Wed, 1 Feb 2023 23:42:43 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 96F9F2086204 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323764; bh=syyTIiCMmSq1Y7SegNUoz2WcQt0ayamimN+gUTnents=; h=From:To:Subject:Date:In-Reply-To:References:From; b=WNHp0Jg1XbGogtJLrp5HyBNcrB4/SAW3s5VtOmHsMAQedVtxDSEwAcbSowjtwErlN GSCZM2JHP1+z7YnZ1dgrIIFYPPSuaZg/Lq4TQIZN3cFCowOpO2kthYr3V0zV+vKMhP OHvRhwArCvYdD2JlUCNArkstF0j4ElJXLhrwIm5U= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 19/22] arm64: unwinder: Add a reliability check in the unwinder based on ORC Date: Thu, 2 Feb 2023 01:40:33 -0600 Message-Id: <20230202074036.507249-20-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Introduce a reliability flag in struct unwind_state. This will be set to false if the PC does not have a valid ORC or if the frame pointer computed from the ORC does not match the actual frame pointer. Now that the unwinder can validate the frame pointer, introduce arch_stack_walk_reliable(). Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace/common.h | 15 ++ arch/arm64/kernel/stacktrace.c | 167 ++++++++++++++++++++- 2 files changed, 175 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/includ= e/asm/stacktrace/common.h index 508f734de46e..064aaf5dc3a0 100644 --- a/arch/arm64/include/asm/stacktrace/common.h +++ b/arch/arm64/include/asm/stacktrace/common.h @@ -11,6 +11,7 @@ =20 #include #include +#include =20 struct stack_info { unsigned long low; @@ -23,6 +24,7 @@ struct stack_info { * @fp: The fp value in the frame record (or the real fp) * @pc: The lr value in the frame record (or the real lr) * + * @prev_pc: The lr value in the previous frame record. * @kr_cur: When KRETPROBES is selected, holds the kretprobe instance * associated with the most recently encountered replacement= lr * value. @@ -32,10 +34,15 @@ struct stack_info { * @stack: The stack currently being unwound. * @stacks: An array of stacks which can be unwound. * @nr_stacks: The number of stacks in @stacks. + * + * @cfa: The sp value at the call site of the current function. + * @unwind_type The previous frame's unwind type. + * @reliable: Stack trace is reliable. */ struct unwind_state { unsigned long fp; unsigned long pc; + unsigned long prev_pc; #ifdef CONFIG_KRETPROBES struct llist_node *kr_cur; #endif @@ -44,6 +51,9 @@ struct unwind_state { struct stack_info stack; struct stack_info *stacks; int nr_stacks; + unsigned long cfa; + int unwind_type; + bool reliable; }; =20 static inline struct stack_info stackinfo_get_unknown(void) @@ -70,11 +80,15 @@ static inline void unwind_init_common(struct unwind_sta= te *state, struct task_struct *task) { state->task =3D task; + state->prev_pc =3D 0; #ifdef CONFIG_KRETPROBES state->kr_cur =3D NULL; #endif =20 state->stack =3D stackinfo_get_unknown(); + state->reliable =3D true; + state->cfa =3D 0; + state->unwind_type =3D UNWIND_HINT_TYPE_CALL; } =20 static struct stack_info *unwind_find_next_stack(const struct unwind_state= *state, @@ -167,6 +181,7 @@ unwind_next_frame_record(struct unwind_state *state) /* * Record this frame record's values. */ + state->prev_pc =3D state->pc; state->fp =3D READ_ONCE(*(unsigned long *)(fp)); state->pc =3D READ_ONCE(*(unsigned long *)(fp + 8)); =20 diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 634279b3b03d..fbcb14539816 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -5,6 +5,8 @@ * Copyright (C) 2012 ARM Ltd. */ #include +#include +#include #include #include #include @@ -16,6 +18,122 @@ #include #include =20 +static inline bool unwind_completed(struct unwind_state *state) +{ + if (state->fp =3D=3D (unsigned long)task_pt_regs(state->task)->stackframe= ) { + /* Final frame; nothing to unwind */ + return true; + } + return false; +} + +#ifdef CONFIG_FRAME_POINTER_VALIDATION + +static void unwind_check_reliable(struct unwind_state *state) +{ + unsigned long pc, fp; + struct orc_entry *orc; + bool adjust_pc =3D false; + + if (unwind_completed(state)) + return; + + /* + * If a previous frame was unreliable, the CFA cannot be reliably + * computed anymore. + */ + if (!state->reliable) + return; + + pc =3D state->pc; + + /* Don't let modules unload while we're reading their ORC data. */ + preempt_disable(); + + orc =3D orc_find(pc); + if (!orc || (!orc->fp_offset && orc->type =3D=3D UNWIND_HINT_TYPE_CALL)) { + /* + * If the final instruction in a function happens to be a call + * instruction, the return address would fall outside of the + * function. That could be the case here. This can happen, for + * instance, if the called function is a "noreturn" function. + * The compiler can optimize away the instructions after the + * call. So, adjust the PC so it falls inside the function and + * retry. + * + * We only do this if the current and the previous frames + * are call frames and not hint frames. + */ + if (state->unwind_type =3D=3D UNWIND_HINT_TYPE_CALL) { + pc -=3D 4; + adjust_pc =3D true; + orc =3D orc_find(pc); + } + } + if (!orc) { + state->reliable =3D false; + goto out; + } + state->unwind_type =3D orc->type; + + if (!state->cfa) { + /* Set up the initial CFA and return. */ + state->cfa =3D state->fp - orc->fp_offset; + goto out; + } + + /* Compute the next CFA and FP. */ + switch (orc->type) { + case UNWIND_HINT_TYPE_CALL: + /* Normal call */ + state->cfa +=3D orc->sp_offset; + fp =3D state->cfa + orc->fp_offset; + break; + + case UNWIND_HINT_TYPE_REGS: + /* + * pt_regs hint: The frame pointer points to either the + * synthetic frame within pt_regs or to the place where + * x29 and x30 are saved in the register save area in + * pt_regs. + */ + state->cfa +=3D orc->sp_offset; + fp =3D state->cfa + offsetof(struct pt_regs, stackframe) - + sizeof(struct pt_regs); + if (state->fp !=3D fp) { + fp =3D state->cfa + offsetof(struct pt_regs, regs[29]) - + sizeof(struct pt_regs); + } + break; + + case UNWIND_HINT_TYPE_IRQ_STACK: + /* Hint to unwind from the IRQ stack to the task stack. */ + state->cfa =3D state->fp + orc->sp_offset; + fp =3D state->fp; + break; + + default: + fp =3D 0; + break; + } + + /* Validate the actual FP with the computed one. */ + if (state->fp !=3D fp) + state->reliable =3D false; +out: + if (state->reliable && adjust_pc) + state->pc =3D pc; + preempt_enable(); +} + +#else /* !CONFIG_FRAME_POINTER_VALIDATION */ + +static void unwind_check_reliable(struct unwind_state *state) +{ +} + +#endif /* CONFIG_FRAME_POINTER_VALIDATION */ + /* * Start an unwind from a pt_regs. * @@ -77,11 +195,9 @@ static inline void unwind_init_from_task(struct unwind_= state *state, static int notrace unwind_next(struct unwind_state *state) { struct task_struct *tsk =3D state->task; - unsigned long fp =3D state->fp; int err; =20 - /* Final frame; nothing to unwind */ - if (fp =3D=3D (unsigned long)task_pt_regs(tsk)->stackframe) + if (unwind_completed(state)) return -ENOENT; =20 err =3D unwind_next_frame_record(state); @@ -116,18 +232,23 @@ static int notrace unwind_next(struct unwind_state *s= tate) } NOKPROBE_SYMBOL(unwind_next); =20 -static void notrace unwind(struct unwind_state *state, +static int notrace unwind(struct unwind_state *state, bool need_reliable, stack_trace_consume_fn consume_entry, void *cookie) { - while (1) { - int ret; + int ret =3D 0; =20 + while (1) { + if (need_reliable && !state->reliable) + return -EINVAL; if (!consume_entry(cookie, state->pc)) break; ret =3D unwind_next(state); + if (need_reliable && !ret) + unwind_check_reliable(state); if (ret < 0) break; } + return ret; } NOKPROBE_SYMBOL(unwind); =20 @@ -216,5 +337,37 @@ noinline notrace void arch_stack_walk(stack_trace_cons= ume_fn consume_entry, unwind_init_from_task(&state, task); } =20 - unwind(&state, consume_entry, cookie); + unwind(&state, false, consume_entry, cookie); +} + +noinline notrace int arch_stack_walk_reliable( + stack_trace_consume_fn consume_entry, + void *cookie, struct task_struct *task) +{ + struct stack_info stacks[] =3D { + stackinfo_get_task(task), + STACKINFO_CPU(irq), +#if defined(CONFIG_VMAP_STACK) + STACKINFO_CPU(overflow), +#endif +#if defined(CONFIG_VMAP_STACK) && defined(CONFIG_ARM_SDE_INTERFACE) + STACKINFO_SDEI(normal), + STACKINFO_SDEI(critical), +#endif + }; + struct unwind_state state =3D { + .stacks =3D stacks, + .nr_stacks =3D ARRAY_SIZE(stacks), + }; + int ret; + + if (task =3D=3D current) + unwind_init_from_caller(&state); + else + unwind_init_from_task(&state, task); + unwind_check_reliable(&state); + + ret =3D unwind(&state, true, consume_entry, cookie); + + return ret =3D=3D -ENOENT ? 0 : -EINVAL; } --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35506C61DA4 for ; Thu, 2 Feb 2023 07:43:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232241AbjBBHnt (ORCPT ); Thu, 2 Feb 2023 02:43:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232073AbjBBHnZ (ORCPT ); Thu, 2 Feb 2023 02:43:25 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 3A61C8496B; Wed, 1 Feb 2023 23:42:46 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id 9414B2086206; Wed, 1 Feb 2023 23:42:44 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9414B2086206 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323765; bh=Z5kdOOhYeHggthgMW9m8t9VXx+iuiwxiRYiTtxW8O8M=; h=From:To:Subject:Date:In-Reply-To:References:From; b=kcj3J5nb4s9m/mJpYcx+0xmHpuI0hmsAgZY7rDqHp4KBk9UntvO5DWZIY4Ba3OYzy w/5FolkML12UdtH3P82p7qTJ15h1CR7M44uMBLjDqdF4EKs85SaU1Amd2AvNxeoqYy sQ1P4XB3jXlRU53jMkGHziCHKr87eMymH/O7LmTY= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 20/22] arm64: Define HAVE_DYNAMIC_FTRACE_WITH_ARGS Date: Thu, 2 Feb 2023 01:40:34 -0600 Message-Id: <20230202074036.507249-21-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" - Define HAVE_DYNAMIC_FTRACE_WITH_ARGS to support livepatch. - Supply the arch code for HAVE_DYNAMIC_FTRACE_WITH_ARGS. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/Kconfig.debug | 1 + arch/arm64/include/asm/ftrace.h | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug index a50caabdb18e..6d5dc90a0a52 100644 --- a/arch/arm64/Kconfig.debug +++ b/arch/arm64/Kconfig.debug @@ -45,6 +45,7 @@ config FRAME_POINTER_VALIDATION bool "Dynamic Frame pointer validation" select UNWINDER_FRAME_POINTER select UNWINDER_ORC + select HAVE_DYNAMIC_FTRACE_WITH_ARGS help This invokes objtool on every object file causing it to generate ORC data for the object file. ORC data is in a custom diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrac= e.h index 329dbbd4d50b..0bc03ecfb257 100644 --- a/arch/arm64/include/asm/ftrace.h +++ b/arch/arm64/include/asm/ftrace.h @@ -78,6 +78,26 @@ static inline unsigned long ftrace_call_adjust(unsigned = long addr) return addr; } =20 +#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS + +struct ftrace_regs { + struct pt_regs regs; +}; + +static __always_inline struct pt_regs * +arch_ftrace_get_regs(struct ftrace_regs *fregs) +{ + return &fregs->regs; +} + +static __always_inline void ftrace_instruction_pointer_set( + struct ftrace_regs *fregs, unsigned long pc) +{ + fregs->regs.pc =3D pc; +} + +#endif + #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS struct dyn_ftrace; struct ftrace_ops; --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79CC5C636D4 for ; Thu, 2 Feb 2023 07:44:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232052AbjBBHoB (ORCPT ); Thu, 2 Feb 2023 02:44:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232103AbjBBHnh (ORCPT ); Thu, 2 Feb 2023 02:43:37 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5FD1D7F691; Wed, 1 Feb 2023 23:42:55 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id 922F72086208; Wed, 1 Feb 2023 23:42:45 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 922F72086208 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323766; bh=euhGTDtLx3URmh7Pid/Q1K/+Ij8uzX8WupPSBlEC+MM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=jV2nJsZcYY6mR/g/T69RktV3BbEnpPSKo2+FrCEYesrufO3nmRg13tKNVnkI0+5/n mEOR9caZeKBzSFy3xIP1gwBvxPDndmiORpictD2xG5BJi2ZohCcK/IIadwlqOPGYO0 Z4V5bbhuj7EhbWO006GSQcejku5I8f8AfvKpdVIE= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 21/22] arm64: Define TIF_PATCH_PENDING for livepatch Date: Thu, 2 Feb 2023 01:40:35 -0600 Message-Id: <20230202074036.507249-22-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" - Define TIF_PATCH_PENDING in arch/arm64/include/asm/thread_info.h for livepatch. - Check TIF_PATCH_PENDING in do_notify_resume() to patch the current task for livepatch. Signed-off-by: Suraj Jitindar Singh Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/thread_info.h | 4 +++- arch/arm64/kernel/signal.c | 4 ++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/= thread_info.h index 848739c15de8..42ba9d37e8d8 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -68,6 +68,7 @@ int arch_dup_task_struct(struct task_struct *dst, #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_MTE_ASYNC_FAULT 5 /* MTE Asynchronous Tag Check Fault */ #define TIF_NOTIFY_SIGNAL 6 /* signal notifications exist */ +#define TIF_PATCH_PENDING 7 /* pending live patching update */ #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ @@ -100,11 +101,12 @@ int arch_dup_task_struct(struct task_struct *dst, #define _TIF_SVE (1 << TIF_SVE) #define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) +#define _TIF_PATCH_PENDING (1 << TIF_PATCH_PENDING) =20 #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ _TIF_UPROBE | _TIF_MTE_ASYNC_FAULT | \ - _TIF_NOTIFY_SIGNAL) + _TIF_NOTIFY_SIGNAL | _TIF_PATCH_PENDING) =20 #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 9ad911f1647c..dea21ba60ff1 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include =20 @@ -1120,6 +1121,9 @@ void do_notify_resume(struct pt_regs *regs, unsigned = long thread_flags) (void __user *)NULL, current); } =20 + if (thread_flags & _TIF_PATCH_PENDING) + klp_update_patch_state(current); + if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) do_signal(regs); =20 --=20 2.25.1 From nobody Sun May 5 23:03:06 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62D59C61DA4 for ; Thu, 2 Feb 2023 07:44:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231778AbjBBHoE (ORCPT ); Thu, 2 Feb 2023 02:44:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232195AbjBBHnj (ORCPT ); Thu, 2 Feb 2023 02:43:39 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7A2A45FD3; Wed, 1 Feb 2023 23:43:00 -0800 (PST) Received: from x64host.home (unknown [47.187.213.40]) by linux.microsoft.com (Postfix) with ESMTPSA id 902872086209; Wed, 1 Feb 2023 23:42:46 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 902872086209 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675323767; bh=rKRdvIoFhmLu94J10M1hjhJkCmPu9SVBqnqj3sarP8g=; h=From:To:Subject:Date:In-Reply-To:References:From; b=AwuokGIE5/2zTLGGDfA54LPomEEh7qwCRkhTmJIYot2YL+G8Pxg6t00zBSx9ozfwn XAIDCY11tyypgPR9qacDC3jkRNwqdF08iZzegu5yEpvWZIN+T5IQuLMTIpB3RPDkqR AQkghR7HK3d+Ht9bmmE8omlP23da//sjAl2DHYkc= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 22/22] arm64: Enable livepatch for ARM64 Date: Thu, 2 Feb 2023 01:40:36 -0600 Message-Id: <20230202074036.507249-23-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202074036.507249-1-madvenka@linux.microsoft.com> References: <0337266cf19f4c98388e3f6d09f590d9de258dc7> <20230202074036.507249-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Madhavan T. Venkataraman" Enable livepatch in arch/arm64/Kconfig. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/Kconfig | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 73c3f30a37c7..01f802935dda 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -232,6 +232,8 @@ config ARM64 select HAVE_SOFTIRQ_ON_OWN_STACK select HAVE_STACK_VALIDATION if FRAME_POINTER_VALIDATION select STACK_VALIDATION if HAVE_STACK_VALIDATION + select HAVE_RELIABLE_STACKTRACE if STACK_VALIDATION + select HAVE_LIVEPATCH if HAVE_DYNAMIC_FTRACE_WITH_REGS && HAVE_RELIABLE_= STACKTRACE help ARM 64-bit (AArch64) Linux support. =20 @@ -2269,3 +2271,4 @@ source "drivers/acpi/Kconfig" =20 source "arch/arm64/kvm/Kconfig" =20 +source "kernel/livepatch/Kconfig" --=20 2.25.1