From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 429C53DEFF9 for ; Tue, 24 Mar 2026 09:49:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345750; cv=none; b=GQbNmESPRIi0MLKfkKXV1VM5aWI/YATeqURcYtvMGuRjhnoKqlSYMxcvebRm0Us4AcoHR6JsD1OQkb5VfDuD0UFI9EhX/MYYmT6gXTpch5gwQoyBsYtaVORc9V7cx+ob8MLGFLZmshyHb6IPTQPpbZrwkvR5M2UFvfBt9WHN2Uo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345750; c=relaxed/simple; bh=cu41BU69+aqmQZMD2uuVADqUugzltgEfWdbVsXmZ/WQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mSehxeiQqCGrVAynbhxzYST36VCnwa3OkQVkmw6ql5A+PiIV08110qdFC0viarF5Y1Woc24+EMC5AtTJ2wObug004XspHRf5+6wPTdYo7DusMF8m5YjsSl2BMs6vFYyt/oHpp6+2u8dJ+bVQtRJHY5DmnB5VrSkqWZLV/h7XPhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KU/bGIP/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KU/bGIP/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KUuxkh0IxxlR1KOoFNjZCYbUauFoeQmbuz1JYSeJkVA=; b=KU/bGIP/a1y2lrZhehwlcDhCUdF5cqPB/agchNfn4eraOGxZkaqy7+dPqYwtfbXD8OZtSQ HHHpF0N0cOCZv+whaAWz5qi3m+qDMF98vG1gDMn4qE4S8hS/W8E6p0PxgNqkn6ilY8/q8Y cjRs7o0D9u/9FTU2AoMdNtBkmdGge3A= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-528-cC3e26PvOy-Zp4rfC0OIjg-1; Tue, 24 Mar 2026 05:49:01 -0400 X-MC-Unique: cC3e26PvOy-Zp4rfC0OIjg-1 X-Mimecast-MFC-AGG-ID: cC3e26PvOy-Zp4rfC0OIjg_1774345738 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EBA601956089; Tue, 24 Mar 2026 09:48:56 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 08FBD300019F; Tue, 24 Mar 2026 09:48:42 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 01/10] objtool: Make validate_call() recognize indirect calls to pv_ops[] Date: Tue, 24 Mar 2026 10:47:52 +0100 Message-ID: <20260324094801.3092968-2-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" call_dest_name() does not get passed the file pointer of validate_call(), which means its invocation of insn_reloc() will always return NULL. Make it take a file pointer. While at it, make sure call_dest_name() uses arch_dest_reloc_offset(), otherwise it gets the pv_ops[] offset wrong. Fabricating an intentional warning shows the change; previously: vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to {dynami= c}() leaves .noinstr.text section now: vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to pv_ops[= 1]() leaves .noinstr.text section Signed-off-by: Valentin Schneider Acked-by: Josh Poimboeuf --- tools/objtool/check.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 3fd98c5b6e1a8..7c82934247484 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -3388,7 +3388,7 @@ static inline bool func_uaccess_safe(struct symbol *f= unc) return false; } =20 -static inline const char *call_dest_name(struct instruction *insn) +static inline const char *call_dest_name(struct objtool_file *file, struct= instruction *insn) { static char pvname[19]; struct reloc *reloc; @@ -3397,9 +3397,9 @@ static inline const char *call_dest_name(struct instr= uction *insn) if (insn_call_dest(insn)) return insn_call_dest(insn)->name; =20 - reloc =3D insn_reloc(NULL, insn); + reloc =3D insn_reloc(file, insn); if (reloc && !strcmp(reloc->sym->name, "pv_ops")) { - idx =3D (reloc_addend(reloc) / sizeof(void *)); + idx =3D arch_insn_adjusted_addend(insn, reloc) / sizeof(void *); snprintf(pvname, sizeof(pvname), "pv_ops[%d]", idx); return pvname; } @@ -3478,17 +3478,19 @@ static int validate_call(struct objtool_file *file, { if (state->noinstr && state->instr <=3D 0 && !noinstr_call_dest(file, insn, insn_call_dest(insn))) { - WARN_INSN(insn, "call to %s() leaves .noinstr.text section", call_dest_n= ame(insn)); + WARN_INSN(insn, "call to %s() leaves .noinstr.text section", call_dest_n= ame(file, insn)); return 1; } =20 if (state->uaccess && !func_uaccess_safe(insn_call_dest(insn))) { - WARN_INSN(insn, "call to %s() with UACCESS enabled", call_dest_name(insn= )); + WARN_INSN(insn, "call to %s() with UACCESS enabled", + call_dest_name(file, insn)); return 1; } =20 if (state->df) { - WARN_INSN(insn, "call to %s() with DF set", call_dest_name(insn)); + WARN_INSN(insn, "call to %s() with DF set", + call_dest_name(file, insn)); return 1; } =20 --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A24AA3DE437 for ; Tue, 24 Mar 2026 09:49:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345766; cv=none; b=Mx09rXzUL5TEsSP/L73rnr+dhPQru0kyjwfbR+xzeGhXkIpXUu+ADdPrbq3mOSZ6ljA1YijJBqr9uwOGqKsaDnF9nMadWIVQK3mZNoMu7nGyATzmcw8YFVjQtugRqvNm+qYPJKoOYWGvvxflyWJ7IzmHyCWa5oXKXboMPaNhPP4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345766; c=relaxed/simple; bh=qS5/565FkL/wKoGjJROAjvoaC615kAvpOTQWdysypuw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dVRd6NCEIQ7MgSn9liXKkOrXW22Qi3a0A1xdvwzkbIkcdT8SSwVa80VG5fetw+/Vpbc2h0o0XcMHrYtHXFIsMppTNJMxVUYR5Ws6O4jgLoSJmieX8bENM5NsJYO4S61/AWf61de3lPl/Rhil1mgoZjEz25QWD91WOxGIHCA9C6U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=i3swGJ55; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="i3swGJ55" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345762; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RTf6sg/Whunq8XuiYT0VZnNsDFxZypJnLbapPFQuudQ=; b=i3swGJ55c4TJNFL51IvtyBgwV+ueL9cMGiMT5oxG9jflQfrc6VS+8x8+mj3YqesXyMyf3b VE/zAMdwcbxegqY6DlqdBFrk3/DSpRDxiSGwr5Y95AltWekMeXyJat0GhAM9Q9I+t1pgVl gR77BQABCWzcQYjgcrNr9AZj9y/FDqw= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-193-7QOl47OWNtCgy7nQumHfow-1; Tue, 24 Mar 2026 05:49:16 -0400 X-MC-Unique: 7QOl47OWNtCgy7nQumHfow-1 X-Mimecast-MFC-AGG-ID: 7QOl47OWNtCgy7nQumHfow_1774345752 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 22F0D18002C7; Tue, 24 Mar 2026 09:49:12 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6D35E3000223; Tue, 24 Mar 2026 09:48:57 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 02/10] objtool: Flesh out warning related to pv_ops[] calls Date: Tue, 24 Mar 2026 10:47:53 +0100 Message-ID: <20260324094801.3092968-3-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" I had to look into objtool itself to understand what this warning was about; make it more explicit. Signed-off-by: Valentin Schneider Acked-by: Josh Poimboeuf --- tools/objtool/check.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 7c82934247484..418dce921e48d 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -3426,7 +3426,7 @@ static bool pv_call_dest(struct objtool_file *file, s= truct instruction *insn) =20 list_for_each_entry(target, &file->pv_ops[idx].targets, pv_target) { if (!target->sec->noinstr) { - WARN("pv_ops[%d]: %s", idx, target->name); + WARN("pv_ops[%d]: indirect call to %s() leaves .noinstr.text section", = idx, target->name); file->pv_ops[idx].clean =3D false; } } --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 069D637187C for ; Tue, 24 Mar 2026 09:49:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345775; cv=none; b=GUv17WH9YT+m9RzNn98o73PWLiekIQBi775ECeAcLcB5VxIHdr4VQNw9H39WbDlKaih1cjGftNfDarn8ATFPa8p/+n++6LKf383YK0NIhAPCZORcc/y141tltMk4pbT8ZGnnRkCqOuVlPQi5iS8DL9rUyLiek8mxnYRVKf5fMjI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345775; c=relaxed/simple; bh=Sv5tdIqdM9tfRNVlMP90CjpmuQ5qdZ9fHuZUzU14wrk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DgS2NuGqdGsDSc5PjOFZ7a6tLzOor5bE90j3ttJ+GtOzwbNW86tQAHnmtDo/+1hLfaLjsQd8VL0E88XUEN+tdCyu/BxPdfalzvL9iaB7gDbGaj6E2dVB6wOMlLjNEaLNCHWnXRZRKcIFG49ISs45J0fs0HhlRJpEBnGfoGC1G2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bLfcN7Lc; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bLfcN7Lc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345773; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GVAdBf/4VOwMJGhw64JQ9TCrIsoGFX1EbfapY7OUslA=; b=bLfcN7LcNrM3Pgv4lH4mjzDNIYOhGLvpmc3h69qAho/zGPM3opwy3WurqvNuC3l5mKAYIJ 09IPEt/mjRLGWl3u20tev2ROTswYE0Ck6xxpx4GuSk1qbglIlhNvTO4kV3yQdki1H6TfMX B0s6SRs+V4G9GBmuZhgVnWDGWWxRc3A= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-615-jysL9BghPfWrhe26ZbG_ag-1; Tue, 24 Mar 2026 05:49:29 -0400 X-MC-Unique: jysL9BghPfWrhe26ZbG_ag-1 X-Mimecast-MFC-AGG-ID: jysL9BghPfWrhe26ZbG_ag_1774345765 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 297341800281; Tue, 24 Mar 2026 09:49:25 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B0EDC30001BB; Tue, 24 Mar 2026 09:49:12 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 03/10] objtool: Always pass a section to validate_unwind_hints() Date: Tue, 24 Mar 2026 10:47:54 +0100 Message-ID: <20260324094801.3092968-4-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" When passing a NULL @sec to validate_unwind_hints(), it is unable to properly initialize the insn_state->noinstr passed down during validation. This means we lose noinstr validation of the hints. That validation currently happens when 'opts.noinstr' is true but 'validate_branch_enabled()' isn't. In other words, this will run noinstr validation of hints: $ objtool --noinstr --link [...] but this won't: $ objtool --noinstr --link --uaccess [...] Always pass a valid section to validate_unwind_hints(), so that noinstr validation of hints happens regardless of the value of validate_branch_enabled(). Signed-off-by: Valentin Schneider --- tools/objtool/check.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 418dce921e48d..b6e63d5beecc3 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -4064,13 +4064,8 @@ static int validate_unwind_hints(struct objtool_file= *file, struct section *sec) =20 init_insn_state(file, &state, sec); =20 - if (sec) { - sec_for_each_insn(file, sec, insn) - warnings +=3D validate_unwind_hint(file, insn, &state); - } else { - for_each_insn(file, insn) - warnings +=3D validate_unwind_hint(file, insn, &state); - } + sec_for_each_insn(file, sec, insn) + warnings +=3D validate_unwind_hint(file, insn, &state); =20 return warnings; } @@ -4567,6 +4562,21 @@ static int validate_functions(struct objtool_file *f= ile) return warnings; } =20 +static int validate_file_unwind_hints(struct objtool_file *file) +{ + struct section *sec; + int warnings =3D 0; + + for_each_sec(file->elf, sec) { + if (!is_text_sec(sec)) + continue; + + warnings +=3D validate_unwind_hints(file, sec); + } + + return warnings; +} + static void mark_endbr_used(struct instruction *insn) { if (!list_empty(&insn->call_node)) @@ -4976,7 +4986,8 @@ int check(struct objtool_file *file) int w =3D 0; =20 w +=3D validate_functions(file); - w +=3D validate_unwind_hints(file, NULL); + w +=3D validate_file_unwind_hints(file); + if (!w) w +=3D validate_reachable_instructions(file); =20 --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 407E13DD528 for ; Tue, 24 Mar 2026 09:49:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345791; cv=none; b=JSwFQn+d54S4ScUgJQPPzbOACjOUMFOZyc1qX/5kcdrsJ6j980UYnzA20bVXxPFrH+WHUrONVRQMalPZjWFj+tZiw67lJa/JDzCxm90leRYg89E6G1eOOIrNAADdbx6hDMNKOaLYNy9cIsfzTZ/YqUiNRPeBzx2T9CctwroCato= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345791; c=relaxed/simple; bh=PydM+q7fKSo44l/v6IFdJAmQkeutCSBP5jnsFNy+4PU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BnalW8XE/gM2GVn9KHAjvYtgV7UICcFOZ/9NsUBQpX8NJTHYoOw/A8Kc4r2nHvZA/Eyrnn8BPGFHG05ITXshP6rTwek/7V62leeHBAB4nHGpQ2B/aI5DPRqtLRaW2GL/T8Gy7ROUwpfeDfpGiKd98xeYObrWxpKWwx9sQERUOs8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Qr25MtKX; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Qr25MtKX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345789; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E6la/YGRpfCEF4wUUEPjSzD94dCfNlkWphwgGjyKscE=; b=Qr25MtKXGa9b5n3VQuylZ0M3/qVrtNDUCvzMJ4pW4qL0zCZD4ZpCTXaXkuCniFOvoPTRuf u2CKrGkbf6FZaqaeZvZd4PB5b6jna7E9o5MMLQQ6hrTS8G1EieDoX/ob6wzxbnP8zJSkC+ GW2up/ov7THsa8KWdnvfr2R4hwfLw2k= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-371-q8xavQyQO_WXuiyIPfHRjw-1; Tue, 24 Mar 2026 05:49:44 -0400 X-MC-Unique: q8xavQyQO_WXuiyIPfHRjw-1 X-Mimecast-MFC-AGG-ID: q8xavQyQO_WXuiyIPfHRjw_1774345779 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 44362180044D; Tue, 24 Mar 2026 09:49:39 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C073930001BB; Tue, 24 Mar 2026 09:49:25 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 04/10] x86/retpoline: Make warn_thunk_thunk .noinstr Date: Tue, 24 Mar 2026 10:47:55 +0100 Message-ID: <20260324094801.3092968-5-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Objtool now warns about it: vmlinux.o: warning: objtool: .altinstr_replacement+0x28e1: call to warn_t= hunk_thunk() leaves .noinstr.text section Mark it noinstr. Signed-off-by: Valentin Schneider --- arch/x86/entry/entry.S | 3 ++- arch/x86/kernel/cpu/bugs.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/entry/entry.S b/arch/x86/entry/entry.S index 6ba2b3adcef0f..e76560f86b332 100644 --- a/arch/x86/entry/entry.S +++ b/arch/x86/entry/entry.S @@ -40,6 +40,8 @@ SYM_FUNC_START(__WARN_trap) SYM_FUNC_END(__WARN_trap) EXPORT_SYMBOL(__WARN_trap) =20 +THUNK warn_thunk_thunk, __warn_thunk + .popsection =20 /* @@ -60,7 +62,6 @@ EXPORT_SYMBOL_FOR_KVM(x86_verw_sel); =20 .popsection =20 -THUNK warn_thunk_thunk, __warn_thunk =20 /* * Clang's implementation of TLS stack cookies requires the variable in diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index d0a2847a4bb05..1ddf9355a37af 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -3732,7 +3732,7 @@ ssize_t cpu_show_vmscape(struct device *dev, struct d= evice_attribute *attr, char } #endif =20 -void __warn_thunk(void) +void noinstr __warn_thunk(void) { WARN_ONCE(1, "Unpatched return thunk in use. This should not happen!\n"); } --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9551139022B for ; Tue, 24 Mar 2026 09:50:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345803; cv=none; b=cS9nzB7oFs9TGstJxNf+9L6oWNaoGsSZMADYs7QJOPgovqscQtc9KgMTZGGldu2sp4peuLnvSlMWd+PCiWZNIoAZdaSzGmyFFJTXqLlh3/tvzWWc3xdJ5RZphtut+e+5AzHar9+mSqtFUA3TWSlb6MBF85dqMHT5TKSZVVW5HJU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345803; c=relaxed/simple; bh=T9kQUcLEzgnq+9wUjyQRoFnUMCb6dHQz4GedubhV2ZM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ISGlsPthqkICrttXYujXXPAJ/gqZQEw7Bt0toCdtpn+xL89N8B7xjBaoVMFU62PNZnP4TKXuQxZ6fFvFrUSt9YtvKV3M7CgpeQ9HYQhO3r7fTLAYaVEN7FD52g09qVIbr53x55UlwoW+HmLiSfiExpt64eDOb3oc2EPMnsQniQo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=epxWMKPk; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="epxWMKPk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=33jZYy+kVgo9I3jnxoYq18WEqNxEKHpsQgdAp+R82tQ=; b=epxWMKPkZNzNMMX8+MSH5kCOFZMVSaUtDeNxUklMvXh9REx2/QnJp8Epzbz0+YKJnto8B3 0Q33DM2QyFCJUUD+s52cCN6hQG3JHXA0nFNv+SV6Bu0l2MnbyeWKcAylXD9hLNauq7Uk4X ICnzFTzw3djwPEgfe8upZ1ti11LlZjc= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-647-fmmzxXonPomwp4_ey5VEAg-1; Tue, 24 Mar 2026 05:49:57 -0400 X-MC-Unique: fmmzxXonPomwp4_ey5VEAg-1 X-Mimecast-MFC-AGG-ID: fmmzxXonPomwp4_ey5VEAg_1774345793 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 191C418005B3; Tue, 24 Mar 2026 09:49:53 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AA8B8300019F; Tue, 24 Mar 2026 09:49:39 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 05/10] sched/isolation: Mark housekeeping_overridden key as __ro_after_init Date: Tue, 24 Mar 2026 10:47:56 +0100 Message-ID: <20260324094801.3092968-6-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" housekeeping_overridden is only ever enabled in the __init function housekeeping_init(), and is never disabled. Mark it __ro_after_init. Signed-off-by: Valentin Schneider --- kernel/sched/isolation.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 3ad0d6df6a0a2..54d1d93cdeea5 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -16,7 +16,7 @@ enum hk_flags { HK_FLAG_KERNEL_NOISE =3D BIT(HK_TYPE_KERNEL_NOISE), }; =20 -DEFINE_STATIC_KEY_FALSE(housekeeping_overridden); +DEFINE_STATIC_KEY_FALSE_RO(housekeeping_overridden); EXPORT_SYMBOL_GPL(housekeeping_overridden); =20 struct housekeeping { --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35E733D3CED for ; Tue, 24 Mar 2026 09:50:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345819; cv=none; b=iGMXFMDfJPzYKYimvHEo5lZK+OCPblyNCGHfBMDvp59kIeek5Dea0snOu4fqh7TQ38Q5rAmE7CQkcdM94KQd0AWnD4sCkTa5c/hSoYxrKQSb1p+iOPIiBekFh4ElvXAA35z/g5wH2uxwnRdzoxqasEthnYXNqZEN0PNuP8/b9gE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345819; c=relaxed/simple; bh=c7XvVbjC282ucWO+aOFGc5mr/TuyX1G9IwGKdqjctWA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GxC3U/lG9XiYx2uUdQ6zjqne0aoKMhXRbUnRi9w4wYNllIeJ4ySv7Okt/f9F28qV2doKbJczvsheG02PUJyGECt1JN9EKjV9a815tvLbsqiwvvCxoV8UhLXMOkPEvCswJyi8U77T4MDjFzGiG0ERlJENJIDqRlIQg0d9DjnvbnQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=EjfSWwsj; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EjfSWwsj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TVByJqRI+8ML79p9W+br/aYbG0M8y+9HOuZPjVZSsqs=; b=EjfSWwsjUDntBX413NW07Gzcs0Rh5b4bylPguGNbbgBoJQPDRcKaO2kXlalRLTSGHxBpHM SsixwS21vnJhW+hiuOKl4x2izHG47UdKK4rXF//EXhdlKKp3j9kvXgF7adBCca7AW+82nl tKvmMpkAk3OHJ+56wJxxLTYFW7rIe3w= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-465-A-mACfHtOXaT3WLgbCoRDw-1; Tue, 24 Mar 2026 05:50:13 -0400 X-MC-Unique: A-mACfHtOXaT3WLgbCoRDw-1 X-Mimecast-MFC-AGG-ID: A-mACfHtOXaT3WLgbCoRDw_1774345808 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7388218002CB; Tue, 24 Mar 2026 09:50:08 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9BDC730001BB; Tue, 24 Mar 2026 09:49:53 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 06/10] objtool: Add .entry.text validation for static branches Date: Tue, 24 Mar 2026 10:47:57 +0100 Message-ID: <20260324094801.3092968-7-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Josh Poimboeuf Warn about static branches in entry text, unless the corresponding key is RO-after-init. Signed-off-by: Josh Poimboeuf [Reduced to only .entry.text rather than .noinstr] Signed-off-by: Valentin Schneider --- include/linux/jump_label.h | 11 +++-- include/linux/objtool.h | 16 ++++++ tools/objtool/Documentation/objtool.txt | 12 +++++ tools/objtool/check.c | 65 ++++++++++++++++++++++++- tools/objtool/include/objtool/check.h | 2 + tools/objtool/include/objtool/elf.h | 3 +- tools/objtool/include/objtool/special.h | 1 + tools/objtool/special.c | 15 +++++- 8 files changed, 118 insertions(+), 7 deletions(-) diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index fdb79dd1ebd8c..9f05338a2f798 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -76,6 +76,7 @@ #include #include #include +#include =20 extern bool static_key_initialized; =20 @@ -376,8 +377,9 @@ struct static_key_false { #define DEFINE_STATIC_KEY_TRUE(name) \ struct static_key_true name =3D STATIC_KEY_TRUE_INIT =20 -#define DEFINE_STATIC_KEY_TRUE_RO(name) \ - struct static_key_true name __ro_after_init =3D STATIC_KEY_TRUE_INIT +#define DEFINE_STATIC_KEY_TRUE_RO(name) \ + struct static_key_true name __ro_after_init =3D STATIC_KEY_TRUE_INIT; \ + ANNOTATE_ENTRY_ALLOWED(name) =20 #define DECLARE_STATIC_KEY_TRUE(name) \ extern struct static_key_true name @@ -385,8 +387,9 @@ struct static_key_false { #define DEFINE_STATIC_KEY_FALSE(name) \ struct static_key_false name =3D STATIC_KEY_FALSE_INIT =20 -#define DEFINE_STATIC_KEY_FALSE_RO(name) \ - struct static_key_false name __ro_after_init =3D STATIC_KEY_FALSE_INIT +#define DEFINE_STATIC_KEY_FALSE_RO(name) \ + struct static_key_false name __ro_after_init =3D STATIC_KEY_FALSE_INIT; \ + ANNOTATE_ENTRY_ALLOWED(name) =20 #define DECLARE_STATIC_KEY_FALSE(name) \ extern struct static_key_false name diff --git a/include/linux/objtool.h b/include/linux/objtool.h index 9a00e701454c5..d738450897b3b 100644 --- a/include/linux/objtool.h +++ b/include/linux/objtool.h @@ -34,6 +34,19 @@ static void __used __section(".discard.func_stack_frame_non_standard") \ *__func_stack_frame_non_standard_##func =3D func =20 +#define __ANNOTATE_ENTRY_ALLOWED(key) \ + static void __used __section(".discard.entry_allowed") \ + *__annotate_entry_allowed_##key =3D &key + +/* + * This is used to tell objtool that a given static key is safe to be used + * within .noinstr code, and it doesn't need to generate a warning about i= t. + * + * For more information, see tools/objtool/Documentation/objtool.txt, + * "non-RO static key usage in entry code" + */ +#define ANNOTATE_ENTRY_ALLOWED(key) __ANNOTATE_ENTRY_ALLOWED(key) + /* * STACK_FRAME_NON_STANDARD_FP() is a frame-pointer-specific function igno= re * for the case where a function is intentionally missing frame pointer se= tup, @@ -111,6 +124,9 @@ #define UNWIND_HINT(type, sp_reg, sp_offset, signal) "\n\t" #define STACK_FRAME_NON_STANDARD(func) #define STACK_FRAME_NON_STANDARD_FP(func) +#define __ASM_ANNOTATE(label, type) "" +#define ASM_ANNOTATE(type) +#define ANNOTATE_ENTRY_ALLOWED(key) #else .macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 signal=3D0 .endm diff --git a/tools/objtool/Documentation/objtool.txt b/tools/objtool/Docume= ntation/objtool.txt index 9e97fc25b2d8a..72fd8cbf56abc 100644 --- a/tools/objtool/Documentation/objtool.txt +++ b/tools/objtool/Documentation/objtool.txt @@ -456,6 +456,18 @@ the objtool maintainers. these special names and does not use module_init() / module_exit() macros to create them. =20 +vmlinux.o: warning: objtool: entry_SYSCALL_64+0x108: housekeeping_overridd= en: non-RO static key usage in entry code + +13. file.o: warning: func()+0x2a: key: non-RO static key usage in entry co= de + + This means that .entry.text function func() uses a static key named 'key' + which can be modified at runtime. This is discouraged because the jump + location may be accessed before a serializating operation has been + executed. + + Check whether the static key/call in question is only modified + during init. If so, define it as read-only-after-init with + DEFINE_STATIC_KEY_*_RO(). =20 If the error doesn't seem to make sense, it could be a bug in objtool. Feel free to ask objtool maintainers for help. diff --git a/tools/objtool/check.c b/tools/objtool/check.c index b6e63d5beecc3..a76364eb8a4f5 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -327,8 +327,10 @@ static void init_insn_state(struct objtool_file *file,= struct insn_state *state, memset(state, 0, sizeof(*state)); init_cfi_state(&state->cfi); =20 - if (opts.noinstr && sec) + if (opts.noinstr && sec) { state->noinstr =3D sec->noinstr; + state->entry =3D sec->entry; + } } =20 static struct cfi_state *cfi_alloc(void) @@ -433,6 +435,9 @@ static int decode_instructions(struct objtool_file *fil= e) !strncmp(sec->name, ".text..__x86.", 13)) sec->noinstr =3D true; =20 + if (!strcmp(sec->name, ".entry.text")) + sec->entry=3D true; + /* * .init.text code is ran before userspace and thus doesn't * strictly need retpolines, except for modules which are @@ -1035,6 +1040,45 @@ static int create_sym_checksum_section(struct objtoo= l_file *file) static int create_sym_checksum_section(struct objtool_file *file) { return= -EINVAL; } #endif =20 +static int read_entry_allowed(struct objtool_file *file) +{ + struct section *rsec; + struct symbol *sym; + struct reloc *reloc; + + rsec =3D find_section_by_name(file->elf, ".rela.discard.entry_allowed"); + if (!rsec) + return 0; + + for_each_reloc(rsec, reloc) { + switch (reloc->sym->type) { + case STT_OBJECT: + case STT_FUNC: + sym =3D reloc->sym; + break; + + case STT_SECTION: + sym =3D find_symbol_by_offset(reloc->sym->sec, + reloc_addend(reloc)); + if (!sym) { + WARN_FUNC(reloc->sym->sec, reloc_addend(reloc), + "can't find static key/call symbol"); + return -1; + } + break; + + default: + WARN("unexpected relocation symbol type in %s: %d", + rsec->name, reloc->sym->type); + return -1; + } + + sym->entry_allowed =3D 1; + } + + return 0; +} + /* * Warnings shouldn't be reported for ignored functions. */ @@ -1878,6 +1922,8 @@ static int handle_jump_alt(struct objtool_file *file, return -1; } =20 + orig_insn->key =3D special_alt->key; + if (opts.hack_jump_label && special_alt->key_addend & 2) { struct reloc *reloc =3D insn_reloc(file, orig_insn); =20 @@ -2660,6 +2706,9 @@ static int decode_sections(struct objtool_file *file) if (read_annotate(file, __annotate_late)) return -1; =20 + if (read_entry_allowed(file)) + return -1; + return 0; } =20 @@ -3544,6 +3593,17 @@ static int validate_return(struct symbol *func, stru= ct instruction *insn, struct return 0; } =20 +static int validate_static_key(struct instruction *insn, struct insn_state= *state) +{ + if (state->entry && !insn->key->entry_allowed) { + WARN_INSN(insn, "%s: non-RO static key usage in entry code", + insn->key->name); + return 1; + } + + return 0; +} + static struct instruction *next_insn_to_validate(struct objtool_file *file, struct instruction *insn) { @@ -3807,6 +3867,9 @@ static int validate_insn(struct objtool_file *file, s= truct symbol *func, if (handle_insn_ops(insn, next_insn, statep)) return 1; =20 + if (insn->key) + validate_static_key(insn, statep); + switch (insn->type) { =20 case INSN_RETURN: diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/= objtool/check.h index 2e1346ad5e926..78bf8191be18d 100644 --- a/tools/objtool/include/objtool/check.h +++ b/tools/objtool/include/objtool/check.h @@ -16,6 +16,7 @@ struct insn_state { bool uaccess; bool df; bool noinstr; + bool entry; s8 instr; }; =20 @@ -97,6 +98,7 @@ struct instruction { struct symbol *sym; struct stack_op *stack_ops; struct cfi_state *cfi; + struct symbol *key; }; =20 static inline struct symbol *insn_func(struct instruction *insn) diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/ob= jtool/elf.h index e12c516bd3200..9d12f7132311a 100644 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -51,7 +51,7 @@ struct section { Elf_Data *data; const char *name; int idx; - bool _changed, text, rodata, noinstr, init, truncate; + bool _changed, text, rodata, noinstr, init, truncate, entry; struct reloc *relocs; unsigned long nr_alloc_relocs; struct section *twin; @@ -89,6 +89,7 @@ struct symbol { u8 changed : 1; u8 included : 1; u8 klp : 1; + u8 entry_allowed : 1; struct list_head pv_target; struct reloc *relocs; struct section *group_sec; diff --git a/tools/objtool/include/objtool/special.h b/tools/objtool/includ= e/objtool/special.h index 121c3761899c1..2298586a75479 100644 --- a/tools/objtool/include/objtool/special.h +++ b/tools/objtool/include/objtool/special.h @@ -18,6 +18,7 @@ struct special_alt { bool group; bool jump_or_nop; u8 key_addend; + struct symbol *key; =20 struct section *orig_sec; unsigned long orig_off; diff --git a/tools/objtool/special.c b/tools/objtool/special.c index 2a533afbc69aa..adec1d0d8a5fe 100644 --- a/tools/objtool/special.c +++ b/tools/objtool/special.c @@ -111,13 +111,26 @@ static int get_alt_entry(struct elf *elf, const struc= t special_entry *entry, =20 if (entry->key) { struct reloc *key_reloc; + struct symbol *key; + s64 key_addend; =20 key_reloc =3D find_reloc_by_dest(elf, sec, offset + entry->key); if (!key_reloc) { ERROR_FUNC(sec, offset + entry->key, "can't find key reloc"); return -1; } - alt->key_addend =3D reloc_addend(key_reloc); + + key =3D key_reloc->sym; + key_addend =3D reloc_addend(key_reloc); + + if (key->type =3D=3D STT_SECTION) + key =3D find_symbol_by_offset(key->sec, key_addend & ~3); + + /* embedded keys not supported */ + if (key) { + alt->key =3D key; + alt->key_addend =3D key_addend; + } } =20 return 0; --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83D523DFC82 for ; Tue, 24 Mar 2026 09:50:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345833; cv=none; b=MmHWi12CvV9qsbmrBQEreQ/9B/0swVCv0m3rDcsE7KORFesQh2J7EqeZ2pReFrgK5f1TJvrd+8XWfwPr4W8h1CA6LSJzJLa1I6Wh8sMqJcvZL4Rox21SUwL0T8HWNBDA6R6pEXqBB3BAY6nfmjIcN3XieCNnqLkgxEXM5m0/LNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345833; c=relaxed/simple; bh=hDR6aj4CEOylmPGqbtIXro21hvw+cK0laBn2ySWqW3Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DHksXV3K7dDR4c3tw2zO5NZ+BOlRagZyIeldNWluX/4ZVzooVOSmifLBB06z1a8N3Epsld1R9pBCzVdPlCrjztSIpNZQgjlJyHvNTkVgfYjZRhHqNPvqhNDbUeWW/NJcgqoYSKjxAF6ItZY8yNCAuIk5Gj4xgFirJQGMnuPzars= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YplABxfn; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YplABxfn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gs9VmOHHSiH2vTwCJERGvoDC9R184iNrGHNb0XwJL0A=; b=YplABxfnh1215omD8VmL5ZFu6x3JjkbDSHvLEdKWnOgYFDGF6U2RNf/jkW8GegMZqyJL5e OqZvuau+PaWjy6ZyVbZhsKH9exWrLxtwf5e2vE1LGdofrdqnqmYSv8yn+JS/1Qqi9ypA/2 5h5StEQFeXQCgPtSk+7ktWGeCgx/NmU= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-445-V_v494rdMsmtuLvusHWzzw-1; Tue, 24 Mar 2026 05:50:25 -0400 X-MC-Unique: V_v494rdMsmtuLvusHWzzw-1 X-Mimecast-MFC-AGG-ID: V_v494rdMsmtuLvusHWzzw_1774345821 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 72B261800281; Tue, 24 Mar 2026 09:50:21 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0B28C300019F; Tue, 24 Mar 2026 09:50:08 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 07/10] x86/jump_label: Add ASM support for static_branch_likely() Date: Tue, 24 Mar 2026 10:47:58 +0100 Message-ID: <20260324094801.3092968-8-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" A later commit will add some early entry code that only needs to be executed if nohz_full is present on the cmdline, not just if CONFIG_NO_HZ_FULL is compiled in. Add an ASM-callable static branch macro. Note that I haven't found a way to express unlikely (i.e. out-of-line) static branches in ASM macros without using extra jumps, which kind of defeats the purpose. Consider: .macro FOOBAR // Key enabled: JMP .Ldostuff_\@ // Key disabled: NOP STATIC_BRANCH_UNLIKELY key, .Ldostuff_\@ // Patched to JMP if enabled jmp .Lend_\@ .Ldostuff_\@: .Lend_\@: .endm Instead, this should be expressed as a likely (i.e. in-line) static key: .macro FOOBAR // Key enabled: NOP // Key disabled: JMP .Lend_\@ STATIC_BRANCH_LIKELY key, .Lend\@ // Patched to NOP if enabled .Lend_\@: .endm Suggested-by: Frederic Weisbecker Signed-off-by: Valentin Schneider --- arch/x86/include/asm/jump_label.h | 33 ++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_= label.h index 05b16299588d5..ea587598abe7c 100644 --- a/arch/x86/include/asm/jump_label.h +++ b/arch/x86/include/asm/jump_label.h @@ -7,7 +7,38 @@ #include #include =20 -#ifndef __ASSEMBLER__ +#ifdef __ASSEMBLER__ + +/* + * There isn't a neat way to craft unlikely static branches in ASM, so they + * all have to be expressed as likely (inline) static branches. This macro + * thus assumes a "likely" usage. + */ +.macro ARCH_STATIC_BRANCH_LIKELY_ASM key, label, jump, hack +1: +.if \jump || \hack + jmp \label +.else + .byte BYTES_NOP5 +.endif + .pushsection __jump_table, "aw" + _ASM_ALIGN + .long 1b - . + .long \label - . + /* LIKELY so bit0=3D1, bit1=3Dhack */ + _ASM_PTR \key + 1 + (\hack << 1) - . + .popsection +.endm + +.macro STATIC_BRANCH_TRUE_LIKELY key, label + ARCH_STATIC_BRANCH_LIKELY_ASM \key, \label, 0, IS_ENABLED(CONFIG_HAVE_JUM= P_LABEL_HACK) +.endm + +.macro STATIC_BRANCH_FALSE_LIKELY key, label + ARCH_STATIC_BRANCH_LIKELY_ASM \key, \label, 1, 0 +.endm + +#else /* !__ASSEMBLER__ */ =20 #include #include --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38F3F3DE45B for ; Tue, 24 Mar 2026 09:50:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345847; cv=none; b=LncivhbHGDN2tRlWvcwy7vhO5ds2T6vgjlua6sZ2rk+fMRFOIiUh+kMvlJ/91sxcBb6lRtdRJFD0pX4juhebpiT/CzhlFNgpVwTZIQueuBpEtBVULlNfaOv/3LrNcdmv5TAHjJjks7j0WtI4tQDXRIcbECVyZt89wjtBBw+j5+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345847; c=relaxed/simple; bh=b7ZNZCzOPKjGbwMlzoQoUzql2SUqhZEqKDX76D3sxUg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PEBl/rs/YapXL19QYuM5dUQ1A9wVyLMV4aH0tD5IzTVTuZghK/wVibkxmyq7af8XXpJUtABdTV/D9VGnuhWtiw82bRNKgazTh2gn9kXhvHPTX4WpZ9s6BXUF6cXrBA7NAhNGWOSBuCi8B4uBs9LlF3QZ3HkB/Aqlhpn5kxS2vUA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=CZmKTVzl; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CZmKTVzl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tg6n/oqixejy7VhwIIBWEvPzLaP7Ximx0xbVbwmAmuE=; b=CZmKTVzlzHIs2o4WTWOvpgN+Ccxk9Is3FydnypCGh5itCDsrR9+Ug7c6quu/CEXeJ+6pTZ wI3DDdJeOsAAbw2eGEndyZmm8l4i6ha7gZbantFZS8j7q878l+4bNCrHyvCvBHrivgLh6L LyrOvUrXdZRC/wHvM4TSKwi6Bz8MtxY= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-663-Svp_4I3WOleNg3hwdwqHoA-1; Tue, 24 Mar 2026 05:50:39 -0400 X-MC-Unique: Svp_4I3WOleNg3hwdwqHoA-1 X-Mimecast-MFC-AGG-ID: Svp_4I3WOleNg3hwdwqHoA_1774345835 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 63F90180044D; Tue, 24 Mar 2026 09:50:35 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E7E0B300019F; Tue, 24 Mar 2026 09:50:21 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 08/10] x86/mm/pti: Introduce a kernel/user CR3 software signal Date: Tue, 24 Mar 2026 10:47:59 +0100 Message-ID: <20260324094801.3092968-9-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Later commits will rely on being able to check whether a remote CPU is using the kernel or the user CR3. This software signal needs to be updated before the actual CR3 write, IOW it always immediately precedes it: KERNEL_CR3_LOADED :=3D 1 SWITCH_TO_KERNEL_CR3 [...] KERNEL_CR3_LOADED :=3D 0 SWITCH_TO_USER_CR3 The variable also gets mapped into the user space visible pages. I tried really hard not to do that, and at some point had something mostly working with having an alias to it through the cpu_entry_area accessed like so before the switch to the kernel CR3: subq $10, %rsp sgdt (%rsp) movq 2(%rsp), \scratch_reg /* GDT address */ addq $10, %rsp movl $1, CPU_ENTRY_AREA_kernel_cr3(\scratch_reg) however this explodes when running 64-bit user code that invokes SYSCALL, since the scratch reg is %rsp itself, and I figured this was enough headach= es. This will only be really useful for NOHZ_FULL CPUs, but it should be cheaper to unconditionally update a never-used per-CPU variable living in its own cacheline than to check a shared cpumask such as housekeeping_cpumask(HK_TYPE_KERNEL_NOISE) at every entry. Signed-off-by: Valentin Schneider --- arch/x86/Kconfig | 14 +++++++++++++ arch/x86/entry/calling.h | 13 ++++++++++++ arch/x86/entry/syscall_64.c | 4 ++++ arch/x86/include/asm/tlbflush.h | 3 +++ arch/x86/mm/pti.c | 36 ++++++++++++++++++++++----------- 5 files changed, 58 insertions(+), 12 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 80527299f859a..f680e83cd5962 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2192,6 +2192,20 @@ config ADDRESS_MASKING The capability can be used for efficient address sanitizers (ASAN) implementation and for optimizations in JITs. =20 +config TRACK_CR3 + def_bool n + prompt "Track which CR3 is in use" + depends on X86_64 && MITIGATION_PAGE_TABLE_ISOLATION && NO_HZ_FULL + help + This option adds a software signal that allows checking remotely + whether a CPU is using the user or the kernel page table. + + This allows further optimizations for NOHZ_FULL CPUs. + + This obviously makes the user<->kernel transition overhead even worse. + + If unsure, say N. + config HOTPLUG_CPU def_bool y depends on SMP diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 77e2d920a6407..4099b7d86efd9 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -9,6 +9,7 @@ #include #include #include +#include =20 /* =20 @@ -170,8 +171,17 @@ For 32-bit we have the following conventions - kernel = is built with andq $(~PTI_USER_PGTABLE_AND_PCID_MASK), \reg .endm =20 +.macro NOTE_CR3_SWITCH scratch_reg:req in_kernel:req +#ifdef CONFIG_TRACK_CR3 + STATIC_BRANCH_FALSE_LIKELY housekeeping_overridden, .Lend_\@ + movl \in_kernel, PER_CPU_VAR(kernel_cr3_loaded) +.Lend_\@: +#endif // CONFIG_TRACK_CR3 +.endm + .macro SWITCH_TO_KERNEL_CR3 scratch_reg:req ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI + NOTE_CR3_SWITCH \scratch_reg $1 mov %cr3, \scratch_reg ADJUST_KERNEL_CR3 \scratch_reg mov \scratch_reg, %cr3 @@ -182,6 +192,7 @@ For 32-bit we have the following conventions - kernel i= s built with PER_CPU_VAR(cpu_tlbstate + TLB_STATE_user_pcid_flush_mask) =20 .macro SWITCH_TO_USER_CR3 scratch_reg:req scratch_reg2:req + NOTE_CR3_SWITCH \scratch_reg $0 mov %cr3, \scratch_reg =20 ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID @@ -229,6 +240,7 @@ For 32-bit we have the following conventions - kernel i= s built with =20 .macro SAVE_AND_SWITCH_TO_KERNEL_CR3 scratch_reg:req save_reg:req ALTERNATIVE "jmp .Ldone_\@", "", X86_FEATURE_PTI + NOTE_CR3_SWITCH \scratch_reg $1 movq %cr3, \scratch_reg movq \scratch_reg, \save_reg /* @@ -257,6 +269,7 @@ For 32-bit we have the following conventions - kernel i= s built with bt $PTI_USER_PGTABLE_BIT, \save_reg jnc .Lend_\@ =20 + NOTE_CR3_SWITCH \scratch_reg $0 ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID =20 /* diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c index b6e68ea98b839..7583f71978856 100644 --- a/arch/x86/entry/syscall_64.c +++ b/arch/x86/entry/syscall_64.c @@ -83,6 +83,10 @@ static __always_inline bool do_syscall_x32(struct pt_reg= s *regs, int nr) return false; } =20 +#ifdef CONFIG_TRACK_CR3 +DEFINE_PER_CPU_PAGE_ALIGNED(bool, kernel_cr3_loaded) =3D true; +#endif + /* Returns true to return using SYSRET, or false to use IRET */ __visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr) { diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflus= h.h index 00daedfefc1b0..3b3aceee701e6 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -17,6 +17,9 @@ #include =20 DECLARE_PER_CPU(u64, tlbstate_untag_mask); +#ifdef CONFIG_TRACK_CR3 +DECLARE_PER_CPU_PAGE_ALIGNED(bool, kernel_cr3_loaded); +#endif =20 void __flush_tlb_all(void); =20 diff --git a/arch/x86/mm/pti.c b/arch/x86/mm/pti.c index f7546e9e8e896..e75450cabd3a6 100644 --- a/arch/x86/mm/pti.c +++ b/arch/x86/mm/pti.c @@ -440,6 +440,18 @@ static void __init pti_clone_p4d(unsigned long addr) *user_p4d =3D *kernel_p4d; } =20 +static void __init pti_clone_percpu(unsigned long va) +{ + phys_addr_t pa =3D per_cpu_ptr_to_phys((void *)va); + pte_t *target_pte; + + target_pte =3D pti_user_pagetable_walk_pte(va, false); + if (WARN_ON(!target_pte)) + return; + + *target_pte =3D pfn_pte(pa >> PAGE_SHIFT, PAGE_KERNEL); +} + /* * Clone the CPU_ENTRY_AREA and associated data into the user space visible * page table. @@ -450,25 +462,25 @@ static void __init pti_clone_user_shared(void) =20 pti_clone_p4d(CPU_ENTRY_AREA_BASE); =20 + /* + * This is done for all possible CPUs during boot to ensure that it's + * propagated to all mms. + */ for_each_possible_cpu(cpu) { /* * The SYSCALL64 entry code needs one word of scratch space * in which to spill a register. It lives in the sp2 slot * of the CPU's TSS. - * - * This is done for all possible CPUs during boot to ensure - * that it's propagated to all mms. */ + pti_clone_percpu((unsigned long)&per_cpu(cpu_tss_rw, cpu)); =20 - unsigned long va =3D (unsigned long)&per_cpu(cpu_tss_rw, cpu); - phys_addr_t pa =3D per_cpu_ptr_to_phys((void *)va); - pte_t *target_pte; - - target_pte =3D pti_user_pagetable_walk_pte(va, false); - if (WARN_ON(!target_pte)) - return; - - *target_pte =3D pfn_pte(pa >> PAGE_SHIFT, PAGE_KERNEL); +#ifdef CONFIG_TRACK_CR3 + /* + * The entry code needs access to the @kernel_cr3_loaded percpu + * variable before the kernel CR3 is loaded. + */ + pti_clone_percpu((unsigned long)&per_cpu(kernel_cr3_loaded, cpu)); +#endif } } =20 --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13BCF3D902F for ; Tue, 24 Mar 2026 09:51:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345863; cv=none; b=uCUzWnNh2czsYUP3AyDvcrOeS/jIyEH9rdyPMY3/QcX/3eQjV5BvyuOiflslZcgwaR6JyXqBAD1STI4ODSFWhmHiW86NTizx/6Wy7oPonrxnIgA48zfRk5V0sClk0YU4vObDdVZ/WdDqbgssWMGnlHRW+VzwK+i3V0bpx7o0JHo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345863; c=relaxed/simple; bh=/kAYgUotWkTkab98yJocoAzcptjnDZjvJSvELvTC5gc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P2eOXNdDh6Drh7mXqoGru+/tqalZ1tooi83ygpUYGEVY5Kz4p1L0dYSoopi/sbFLBBTF7z6nCVgbRUGfAhkBK6R2BWC8e1m5zWRdU+xvzj3sauOUnyuCVXLpKYP92rZHySkJAOtjOYaqSVpzGEnHzmGjExLUnioNAmc5W2V95WI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=hGqCQ9UK; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hGqCQ9UK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gat9n9rqQs56/qB2UIuZaMkqQi+a1E0dBrxpCkg7CqE=; b=hGqCQ9UKXa6EkncmoJaWq2onfEtnDy1j3MYmiaSzqSqbI0EMh5D8nW53zUtLZfV2j8W5FP wBo2PzyhFrJqvzhrpV1qLzH05mPiiRpPBwW1ZHDjyu+zHinzcAe08a6SNenmWsK7uzp+jL z5hcOhz7JtYCRew+j3qN9HE/li8Wwdo= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-131-B5zXNt9rNpyEt3OiBpBtfg-1; Tue, 24 Mar 2026 05:50:55 -0400 X-MC-Unique: B5zXNt9rNpyEt3OiBpBtfg-1 X-Mimecast-MFC-AGG-ID: B5zXNt9rNpyEt3OiBpBtfg_1774345850 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A6F7F1800281; Tue, 24 Mar 2026 09:50:50 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D57C830001BB; Tue, 24 Mar 2026 09:50:35 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: "Peter Zijlstra (Intel)" , Nicolas Saenz Julienne , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 09/10] context_tracking,x86: Defer kernel text patching IPIs when tracking CR3 switches Date: Tue, 24 Mar 2026 10:48:00 +0100 Message-ID: <20260324094801.3092968-10-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" text_poke_bp_batch() sends IPIs to all online CPUs to synchronize them vs the newly patched instruction. CPUs that are executing in userspace do not need this synchronization to happen immediately, and this is actually harmful interference for NOHZ_FULL CPUs. As the synchronization IPIs are sent using a blocking call, returning from text_poke_bp_batch() implies all CPUs will observe the patched instruction(s), and this should be preserved even if the IPI is deferred. In other words, to safely defer this synchronization, any kernel instruction leading to the execution of the deferred instruction sync must *not* be mutable (patchable) at runtime. This means we must pay attention to mutable instructions in the early entry code: - alternatives - static keys - static calls - all sorts of probes (kprobes/ftrace/bpf/???) The early entry code is noinstr, which gets rid of the probes. Alternatives are safe, because it's boot-time patching (before SMP is even brought up) which is before any IPI deferral can happen. This leaves us with static keys and static calls. Any static key used in early entry code should be only forever-enabled at boot time, IOW __ro_after_init (pretty much like alternatives). Exceptions to that will now be caught by objtool. The deferred instruction sync is the CR3 RMW done as part of kPTI when switching to the kernel page table: SDM vol2 chapter 4.3 - Move to/from control registers: ``` MOV CR* instructions, except for MOV CR8, are serializing instructions. ``` Leverage the new kernel_cr3_loaded signal and the kPTI CR3 RMW to defer sync_core() IPIs targeting NOHZ_FULL CPUs. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Valentin Schneider --- arch/x86/include/asm/text-patching.h | 5 ++++ arch/x86/kernel/alternative.c | 34 +++++++++++++++++++++++----- arch/x86/kernel/kprobes/core.c | 4 ++-- arch/x86/kernel/kprobes/opt.c | 4 ++-- arch/x86/kernel/module.c | 2 +- 5 files changed, 38 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/te= xt-patching.h index f2d142a0a862e..628e80f8318cd 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -33,6 +33,11 @@ extern void text_poke_apply_relocation(u8 *buf, const u8= * const instr, size_t i */ extern void *text_poke(void *addr, const void *opcode, size_t len); extern void smp_text_poke_sync_each_cpu(void); +#ifdef CONFIG_TRACK_CR3 +extern void smp_text_poke_sync_each_cpu_deferrable(void); +#else +#define smp_text_poke_sync_each_cpu_deferrable smp_text_poke_sync_each_cpu +#endif extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern void *text_poke_copy(void *addr, const void *opcode, size_t len); #define text_poke_copy text_poke_copy diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 28518371d8bf3..f3af77d7c533c 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -6,6 +6,7 @@ #include #include #include +#include =20 #include #include @@ -13,6 +14,7 @@ #include #include #include +#include =20 int __read_mostly alternatives_patched; =20 @@ -2706,11 +2708,29 @@ static void do_sync_core(void *info) sync_core(); } =20 +static void __smp_text_poke_sync_each_cpu(smp_cond_func_t cond_func) +{ + on_each_cpu_cond(cond_func, do_sync_core, NULL, 1); +} + void smp_text_poke_sync_each_cpu(void) { - on_each_cpu(do_sync_core, NULL, 1); + __smp_text_poke_sync_each_cpu(NULL); +} + +#ifdef CONFIG_TRACK_CR3 +static bool do_sync_core_defer_cond(int cpu, void *info) +{ + return housekeeping_cpu(cpu, HK_TYPE_KERNEL_NOISE) || + per_cpu(kernel_cr3_loaded, cpu); } =20 +void smp_text_poke_sync_each_cpu_deferrable(void) +{ + __smp_text_poke_sync_each_cpu(do_sync_core_defer_cond); +} +#endif + /* * NOTE: crazy scheme to allow patching Jcc.d32 but not increase the size = of * this thing. When len =3D=3D 6 everything is prefixed with 0x0f and we m= ap @@ -2914,11 +2934,13 @@ void smp_text_poke_batch_finish(void) * First step: add a INT3 trap to the address that will be patched. */ for (i =3D 0; i < text_poke_array.nr_entries; i++) { - text_poke_array.vec[i].old =3D *(u8 *)text_poke_addr(&text_poke_array.ve= c[i]); - text_poke(text_poke_addr(&text_poke_array.vec[i]), &int3, INT3_INSN_SIZE= ); + void *addr =3D text_poke_addr(&text_poke_array.vec[i]); + + text_poke_array.vec[i].old =3D *((u8 *)addr); + text_poke(addr, &int3, INT3_INSN_SIZE); } =20 - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); =20 /* * Second step: update all but the first byte of the patched range. @@ -2980,7 +3002,7 @@ void smp_text_poke_batch_finish(void) * not necessary and we'd be safe even without it. But * better safe than sorry (plus there's not only Intel). */ - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); } =20 /* @@ -3001,7 +3023,7 @@ void smp_text_poke_batch_finish(void) } =20 if (do_sync) - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); =20 /* * Remove and wait for refs to be zero. diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index c1fac3a9fecc2..61a93ba30f255 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -789,7 +789,7 @@ void arch_arm_kprobe(struct kprobe *p) u8 int3 =3D INT3_INSN_OPCODE; =20 text_poke(p->addr, &int3, 1); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); perf_event_text_poke(p->addr, &p->opcode, 1, &int3, 1); } =20 @@ -799,7 +799,7 @@ void arch_disarm_kprobe(struct kprobe *p) =20 perf_event_text_poke(p->addr, &int3, 1, &p->opcode, 1); text_poke(p->addr, &p->opcode, 1); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); } =20 void arch_remove_kprobe(struct kprobe *p) diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c index 6f826a00eca29..3b3be66da320c 100644 --- a/arch/x86/kernel/kprobes/opt.c +++ b/arch/x86/kernel/kprobes/opt.c @@ -509,11 +509,11 @@ void arch_unoptimize_kprobe(struct optimized_kprobe *= op) JMP32_INSN_SIZE - INT3_INSN_SIZE); =20 text_poke(addr, new, INT3_INSN_SIZE); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); text_poke(addr + INT3_INSN_SIZE, new + INT3_INSN_SIZE, JMP32_INSN_SIZE - INT3_INSN_SIZE); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); =20 perf_event_text_poke(op->kp.addr, old, JMP32_INSN_SIZE, new, JMP32_INSN_S= IZE); } diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 11c45ce42694c..0894b1f38de77 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -209,7 +209,7 @@ static int write_relocate_add(Elf64_Shdr *sechdrs, write, apply); =20 if (!early) { - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); mutex_unlock(&text_mutex); } =20 --=20 2.52.0 From nobody Fri Apr 3 10:17:43 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6E9735A399 for ; Tue, 24 Mar 2026 09:51:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345873; cv=none; b=Zkr0XfHCE0uVBeAfp6kGvclY/iZ4aQLtb6eT9FPTsa89/5fAHxObEroq0CUxkkWk1T+MHKMGuEZ0dT4DlbEW1s9k6OdcjsvOWMoFOew5+zqDsrSyRC+P2chGvBkiCHbUysEAEqa26WuOmo1QTNZyEC9dlZB3aLBgquVWNEjSQPA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774345873; c=relaxed/simple; bh=xopU9c0YD6Ruqe30rx3/tN0/jRrJ5uH0UC4uqScMrPo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Fck0pvQqRzSPBRMddSLN+hjAH9WNoJfDu5YLqbyT44qjzS9k3WaJbwOa43XpbmwKZwWDvZgqXwtAyrbTY/ln/r3BrilZ4bFxyFA0mojKzfqRO34VYxd0b8iOlUhl1+87YewGwxxX/lh8zg44VPSeqhVVydjaQsLZuZXJ3UD/NF4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=g0dH42pH; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="g0dH42pH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774345870; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rz+xHMp39eTKGjjZlITwEBRnQcSaPN2YegGdzhgHICw=; b=g0dH42pHB0L3CzzkuN8I1IOM2T5m84ETV/3u/d0NlZWUG1nkiPlNZhUVinIMFoYDoc0nbf KIUUcq4UrfVI1tRqoRkqA6YU16iS2ma0UIPYnX9D4VUkG7K9e3NZ2TL25se2UP1Ypz0E9J E97OXEeBz+dJtJ/6gPznKwBQT3FRRxY= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-622-wNsf_mUJN3imxGWVyKna5w-1; Tue, 24 Mar 2026 05:51:06 -0400 X-MC-Unique: wNsf_mUJN3imxGWVyKna5w-1 X-Mimecast-MFC-AGG-ID: wNsf_mUJN3imxGWVyKna5w_1774345862 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A46981955F18; Tue, 24 Mar 2026 09:51:02 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.44.34.246]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 20CBD300019F; Tue, 24 Mar 2026 09:50:50 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Tomas Glozar , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v8 10/10] x86/mm, mm/vmalloc: Defer kernel TLB flush IPIs when tracking CR3 switches Date: Tue, 24 Mar 2026 10:48:01 +0100 Message-ID: <20260324094801.3092968-11-vschneid@redhat.com> In-Reply-To: <20260324094801.3092968-1-vschneid@redhat.com> References: <20260324094801.3092968-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Previous commits have added a software signal that tracks which CR3 (kernel or user) is in use for any given CPU. Combined with: o the CR3 switch itself being a flush for non-global mappings o global mappings under kPTI being limited to the CEA and entry text we now have a way to safely defer (kernel) TLB flush IPIs targeting NOHZ_FULL CPUs executing in userspace (i.e. with the user CR3 loaded). When sending a kernel TLB flush IPI to a NOHZ_FULL CPU, check whether it is using the user CR3, and if it is, do not interrupt it and instead rely on the CR3 write that happens when switching to the kernel CR3. Signed-off-by: Valentin Schneider --- arch/x86/include/asm/tlbflush.h | 1 + arch/x86/mm/tlb.c | 34 ++++++++++++++++++++++++++------- mm/vmalloc.c | 30 ++++++++++++++++++++++++----- 3 files changed, 53 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflus= h.h index 3b3aceee701e6..8bae150206665 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -22,6 +22,7 @@ DECLARE_PER_CPU_PAGE_ALIGNED(bool, kernel_cr3_loaded); #endif =20 void __flush_tlb_all(void); +void flush_tlb_kernel_range_deferrable(unsigned long start, unsigned long = end); =20 #define TLB_FLUSH_ALL -1UL #define TLB_GENERATION_INVALID 0 diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index f5b93e01e3472..e08f16474f074 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -13,6 +13,7 @@ #include #include #include +#include =20 #include #include @@ -1530,23 +1531,24 @@ static void do_kernel_range_flush(void *info) flush_tlb_one_kernel(addr); } =20 -static void kernel_tlb_flush_all(struct flush_tlb_info *info) +static void kernel_tlb_flush_all(smp_cond_func_t cond, struct flush_tlb_in= fo *info) { if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) invlpgb_flush_all(); else - on_each_cpu(do_flush_tlb_all, NULL, 1); + on_each_cpu_cond(cond, do_flush_tlb_all, NULL, 1); } =20 -static void kernel_tlb_flush_range(struct flush_tlb_info *info) +static void kernel_tlb_flush_range(smp_cond_func_t cond, struct flush_tlb_= info *info) { if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) invlpgb_kernel_range_flush(info); else - on_each_cpu(do_kernel_range_flush, info, 1); + on_each_cpu_cond(cond, do_kernel_range_flush, info, 1); } =20 -void flush_tlb_kernel_range(unsigned long start, unsigned long end) +static inline void +__flush_tlb_kernel_range(smp_cond_func_t cond, unsigned long start, unsign= ed long end) { struct flush_tlb_info *info; =20 @@ -1556,13 +1558,31 @@ void flush_tlb_kernel_range(unsigned long start, un= signed long end) TLB_GENERATION_INVALID); =20 if (info->end =3D=3D TLB_FLUSH_ALL) - kernel_tlb_flush_all(info); + kernel_tlb_flush_all(cond, info); else - kernel_tlb_flush_range(info); + kernel_tlb_flush_range(cond, info); =20 put_flush_tlb_info(); } =20 +void flush_tlb_kernel_range(unsigned long start, unsigned long end) +{ + __flush_tlb_kernel_range(NULL, start, end); +} + +#ifdef CONFIG_TRACK_CR3 +static bool flush_tlb_kernel_cond(int cpu, void *info) +{ + return housekeeping_cpu(cpu, HK_TYPE_KERNEL_NOISE) || + per_cpu(kernel_cr3_loaded, cpu); +} + +void flush_tlb_kernel_range_deferrable(unsigned long start, unsigned long = end) +{ + __flush_tlb_kernel_range(flush_tlb_kernel_cond, start, end); +} +#endif + /* * This can be used from process context to figure out what the value of * CR3 is without needing to do a (slow) __read_cr3(). diff --git a/mm/vmalloc.c b/mm/vmalloc.c index e286c2d2068cb..55b7bafe26016 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -501,6 +501,26 @@ void vunmap_range_noflush(unsigned long start, unsigne= d long end) __vunmap_range_noflush(start, end); } =20 +/* + * !!! BIG FAT WARNING !!! + * + * The CPU is free to cache any part of the paging hierarchy it wants at a= ny + * time. It's also free to set accessed and dirty bits at any time, even f= or + * instructions that may never execute architecturally. + * + * This means that deferring a TLB flush affecting freed page-table-pages = (IOW, + * keeping them in a CPU's paging hierarchy cache) is a recipe for disaste= r. + * + * This isn't a problem for deferral of TLB flushes in vmalloc, because + * page-table-pages used for vmap() mappings are never freed - see how + * __vunmap_range_noflush() walks the whole mapping but only clears the le= af PTEs. + * If this ever changes, TLB flush deferral will cause misery. + */ +void __weak flush_tlb_kernel_range_deferrable(unsigned long start, unsigne= d long end) +{ + flush_tlb_kernel_range(start, end); +} + /** * vunmap_range - unmap kernel virtual addresses * @addr: start of the VM area to unmap @@ -514,7 +534,7 @@ void vunmap_range(unsigned long addr, unsigned long end) { flush_cache_vunmap(addr, end); vunmap_range_noflush(addr, end); - flush_tlb_kernel_range(addr, end); + flush_tlb_kernel_range_deferrable(addr, end); } =20 static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, @@ -2366,7 +2386,7 @@ static bool __purge_vmap_area_lazy(unsigned long star= t, unsigned long end, =20 nr_purge_nodes =3D cpumask_weight(&purge_nodes); if (nr_purge_nodes > 0) { - flush_tlb_kernel_range(start, end); + flush_tlb_kernel_range_deferrable(start, end); =20 /* One extra worker is per a lazy_max_pages() full set minus one. */ nr_purge_helpers =3D atomic_long_read(&vmap_lazy_nr) / lazy_max_pages(); @@ -2469,7 +2489,7 @@ static void free_unmap_vmap_area(struct vmap_area *va) flush_cache_vunmap(va->va_start, va->va_end); vunmap_range_noflush(va->va_start, va->va_end); if (debug_pagealloc_enabled_static()) - flush_tlb_kernel_range(va->va_start, va->va_end); + flush_tlb_kernel_range_deferrable(va->va_start, va->va_end); =20 free_vmap_area_noflush(va); } @@ -2916,7 +2936,7 @@ static void vb_free(unsigned long addr, unsigned long= size) vunmap_range_noflush(addr, addr + size); =20 if (debug_pagealloc_enabled_static()) - flush_tlb_kernel_range(addr, addr + size); + flush_tlb_kernel_range_deferrable(addr, addr + size); =20 spin_lock(&vb->lock); =20 @@ -2981,7 +3001,7 @@ static void _vm_unmap_aliases(unsigned long start, un= signed long end, int flush) free_purged_blocks(&purge_list); =20 if (!__purge_vmap_area_lazy(start, end, false) && flush) - flush_tlb_kernel_range(start, end); + flush_tlb_kernel_range_deferrable(start, end); mutex_unlock(&vmap_purge_lock); } =20 --=20 2.52.0