From nobody Mon Apr 6 08:13:35 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6676C54EE9 for ; Thu, 8 Sep 2022 15:01:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231622AbiIHPBi (ORCPT ); Thu, 8 Sep 2022 11:01:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232766AbiIHPBb (ORCPT ); Thu, 8 Sep 2022 11:01:31 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2474ED9B7; Thu, 8 Sep 2022 08:01:28 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C4CF561D38; Thu, 8 Sep 2022 15:01:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D03C4C433C1; Thu, 8 Sep 2022 15:01:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662649287; bh=pCqMEF+hRGXhMV+bGw+tm59DrO7QZ5nD5ZCcllSjM/g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dY1apBZg3O93aJIBk19Yd3jF5YPM60PkzpYA8YGXIUnNpQJlj5a36B7Xdd77fqQdb MOy1Ol7wuY3KPZRUFAQuZEjznAilft0+6d3a3smt4kFWExvbpX0WjFGfhTvEFT0cHP tq9AfEcf/DgxaZq7ZE47yOTlkXq8bCLhpn0sFagS8cygr2uR7J7rPHUModY7FIoOmY zgIjtQt2PvzU8VPrCrilXnySvPtls+ul1/7Ff9dMCJDFKRvBBp6JfFNlCS7uOV6myb aIBfhcATkRf72jVv/BoUoIk12Xedsr8paI9IAh7kNFi0jg8lhKDXOHGFnia/iaSbE+ O7u4EU3GoecHw== From: "Masami Hiramatsu (Google)" To: Peter Zijlstra , Josh Poimboeuf Cc: Steven Rostedt , Masami Hiramatsu , Ingo Molnar , Suleiman Souhlal , bpf , linux-kernel@vger.kernel.org, Borislav Petkov , x86@kernel.org Subject: [PATCH v3 1/2] x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK Date: Fri, 9 Sep 2022 00:01:22 +0900 Message-Id: <166264928214.775585.6657611968575138295.stgit@devnote2> X-Mailer: git-send-email 2.25.1 In-Reply-To: <166264927154.775585.16570756675363838701.stgit@devnote2> References: <166264927154.775585.16570756675363838701.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Masami Hiramatsu (Google) Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after RET instruction, kprobes always failes to check the probed instruction boundary by decoding the function body if the probed address is after such sequence. (Note that some conditional code blocks will be placed after function return, if compiler decides it is not on the hot path.) This is because kprobes expects kgdb puts the INT3 as a software breakpoint and it will replace the original instruction. But these INT3 are not such purpose, it doesn't need to recover the original instruction. To avoid this issue, kprobes checks whether the INT3 is owned by kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction. Signed-off-by: Masami Hiramatsu (Google) Suggested-by: Peter Zijlstra Fixes: e463a09af2f0 ("x86: Add straight-line-speculation mitigation") Cc: stable@vger.kernel.org Reviewed-by: Josh Poimboeuf --- arch/x86/kernel/kprobes/core.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 4c3c27b6aea3..c6dd7ae68c8f 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -283,12 +284,15 @@ static int can_probe(unsigned long paddr) if (ret < 0) return 0; =20 +#ifdef CONFIG_KGDB /* - * Another debugging subsystem might insert this breakpoint. - * In that case, we can't recover it. + * If there is a dynamically installed kgdb sw breakpoint, + * this function should not be probed. */ - if (insn.opcode.bytes[0] =3D=3D INT3_INSN_OPCODE) + if (insn.opcode.bytes[0] =3D=3D INT3_INSN_OPCODE && + kgdb_has_hit_break(addr)) return 0; +#endif addr +=3D insn.length; } From nobody Mon Apr 6 08:13:35 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B370C54EE9 for ; Thu, 8 Sep 2022 15:01:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232775AbiIHPBy (ORCPT ); Thu, 8 Sep 2022 11:01:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232767AbiIHPBo (ORCPT ); Thu, 8 Sep 2022 11:01:44 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6F8EED9B1; Thu, 8 Sep 2022 08:01:40 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 13B29B82124; Thu, 8 Sep 2022 15:01:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 56845C433C1; Thu, 8 Sep 2022 15:01:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662649297; bh=EnQDCn6+DxnGRVKzzZhsuaX7OEoKlrHh00hQOt+anfY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P9X9G4T/J2ADevyxRvIJG1T96e7j6dgfn8qxN2GB1PYwDKwtN84K7hCup+P58Bevp FSHBLBpnc9NYt1Yxer9GMApEhxsIo4/YGvTmZRTYcatFt0sivsI3Nx+aUUjTQ/6Kyw L2z94AKYSwoUAxTCimgVdPfyUeRY5GcMv2ggaNaizbFFPbEo9L/ELezbW6JZFOpMIT hPTCtpvZMkhgu96qiFLGpCprJbrrr8v+vmE95WUyHuOJuBN0QbNXd0pUnFFLfdMHTA Nxuoomze5klrf0CoBaafjk53BO4rEnVsXrU+1PNirDJg6HgGmeyuJs5i+dR3n5J0aH a3y/40uqZdwsg== From: "Masami Hiramatsu (Google)" To: Peter Zijlstra , Josh Poimboeuf Cc: Steven Rostedt , Masami Hiramatsu , Ingo Molnar , Suleiman Souhlal , bpf , linux-kernel@vger.kernel.org, Borislav Petkov , x86@kernel.org Subject: [PATCH v3 2/2] x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK Date: Fri, 9 Sep 2022 00:01:32 +0900 Message-Id: <166264929259.775585.14768855667710290362.stgit@devnote2> X-Mailer: git-send-email 2.25.1 In-Reply-To: <166264927154.775585.16570756675363838701.stgit@devnote2> References: <166264927154.775585.16570756675363838701.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Masami Hiramatsu (Google) Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after function return, kprobe jump optimization always fails on the functions with such INT3 inside the function body. (It already checks the INT3 padding between functions, but not inside the function) To avoid this issue, as same as kprobes, check whether the INT3 comes from kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction. Signed-off-by: Masami Hiramatsu (Google) Suggested-by: Peter Zijlstra Fixes: e463a09af2f0 ("x86: Add straight-line-speculation mitigation") Cc: stable@vger.kernel.org Reviewed-by: Josh Poimboeuf --- arch/x86/kernel/kprobes/opt.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c index e6b8c5362b94..e57e07b0edb6 100644 --- a/arch/x86/kernel/kprobes/opt.c +++ b/arch/x86/kernel/kprobes/opt.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -279,19 +280,6 @@ static int insn_is_indirect_jump(struct insn *insn) return ret; } =20 -static bool is_padding_int3(unsigned long addr, unsigned long eaddr) -{ - unsigned char ops; - - for (; addr < eaddr; addr++) { - if (get_kernel_nofault(ops, (void *)addr) < 0 || - ops !=3D INT3_INSN_OPCODE) - return false; - } - - return true; -} - /* Decode whole function to ensure any instructions don't jump into target= */ static int can_optimize(unsigned long paddr) { @@ -334,15 +322,15 @@ static int can_optimize(unsigned long paddr) ret =3D insn_decode_kernel(&insn, (void *)recovered_insn); if (ret < 0) return 0; - +#ifdef CONFIG_KGDB /* - * In the case of detecting unknown breakpoint, this could be - * a padding INT3 between functions. Let's check that all the - * rest of the bytes are also INT3. + * If there is a dynamically installed kgdb sw breakpoint, + * this function should not be probed. */ - if (insn.opcode.bytes[0] =3D=3D INT3_INSN_OPCODE) - return is_padding_int3(addr, paddr - offset + size) ? 1 : 0; - + if (insn.opcode.bytes[0] =3D=3D INT3_INSN_OPCODE && + kgdb_has_hit_break(addr)) + return 0; +#endif /* Recover address */ insn.kaddr =3D (void *)addr; insn.next_byte =3D (void *)(addr + insn.length);